Digital twin three-dimensional scene construction method and system for urban security area

Abstract

The application relates to the technical field of three-dimensional reconstruction, and particularly relates to a digital twin three-dimensional scene construction method and system for a city security area.The method comprises the following steps: obtaining an independent deviation value of each object under each view angle according to video data and space data of multiple view angles; analyzing the difference of the independent deviation values of each object under multiple view angles to obtain a comprehensive deviation value of each object; screening according to the comprehensive deviation value of each object to obtain a target area; obtaining a conduction risk value of each rendering area according to the spatial distance between each rendering area and the target area and the maximum comprehensive deviation value corresponding to the target area; comprehensively evaluating the abnormal change value of each rendering area and the conduction risk values of multiple rendering areas to obtain a rendering priority of each rendering area, and performing three-dimensional rendering according to multiple rendering priorities.The application can improve the efficiency of digital twin three-dimensional scene construction for a city security area.

Description

Technical Field

[0001] This invention relates to the technical field of three-dimensional reconstruction, and specifically to a method and system for constructing digital twin three-dimensional scenes for urban security areas. Background Technology

[0002] By constructing a three-dimensional digital twin scene using surveillance video and spatial data, staff can intuitively determine the location of risk events and their surrounding environment, thereby effectively improving the intelligence level and emergency response capabilities of urban security.

[0003] In related technologies, the main approach is to apply the same rendering precision to the entire urban security area, that is, to perform undifferentiated, high-resolution rendering on all areas within the scene. However, this method has low efficiency in constructing digital twin 3D scenes of urban security areas. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for constructing digital twin 3D scenes in urban security areas, which solves the problem of low efficiency in constructing digital twin 3D scenes in urban security areas in related technologies.

[0005] In a first aspect, embodiments of the present invention provide a method for constructing a digital twin 3D scene for urban security areas, the method comprising: Behavioral analysis is performed based on video and spatial data from multiple perspectives to obtain the independent deviation value of each object in each perspective. The independent deviation value is used to characterize the degree to which the behavior of the corresponding object deviates from the average level of the group in the corresponding perspective. The differences in independent deviation values ​​of each object under multiple perspectives are analyzed to obtain the comprehensive deviation value of each object. The comprehensive deviation value is used to characterize the degree to which the behavior of the corresponding object deviates from the group after verification from multiple perspectives. Multiple regions to be rendered are filtered based on the overall deviation value of each object to obtain the target region. The target region is used to indicate the region to be rendered where the object with the largest overall deviation value is located. Based on the spatial distance between each region to be rendered and the target region, and the maximum comprehensive deviation value corresponding to the target region, data analysis is performed to obtain the transmission risk value of each region to be rendered. The transmission risk value is used to characterize the degree of risk of the corresponding region to be rendered being affected by the target region. A comprehensive evaluation is conducted based on the abnormal change values ​​of each region to be rendered and the propagation risk values ​​of multiple regions to be rendered to obtain the rendering priority of each region to be rendered. 3D rendering is then performed based on the rendering priority of each region to be rendered. The rendering priority is used to indicate the rendering accuracy of the corresponding region to be rendered. The rendering accuracy of the target region is higher than that of any region to be rendered. The abnormal change value of each region to be rendered is obtained by analyzing the change in the comprehensive deviation value of the multiple objects it includes within adjacent time periods.

[0006] In one embodiment, the step of performing behavioral analysis based on video data and spatial data from multiple perspectives to obtain an independent deviation value for each object under each perspective includes: Target tracking is performed based on video data and spatial data from multiple perspectives to obtain the orientation similarity value of each object in each perspective. The orientation similarity value is used to characterize the degree of similarity between the corresponding object and other objects in the corresponding perspective. The differences in directional similarity values ​​of multiple objects under each viewpoint are analyzed to obtain the independent deviation value of each object under each viewpoint. The independent deviation value is used to characterize the degree to which the behavior of the corresponding object deviates from the average level of the group under the corresponding viewpoint.

[0007] In one embodiment, the step of performing target tracking based on video data and spatial data from multiple perspectives to obtain the directional similarity value of each object in each perspective includes: Target detection is performed based on video data from multiple perspectives to obtain multiple objects from each perspective. For each object, optical flow tracing is performed based on its video data at each viewpoint to obtain the motion direction of each object at each viewpoint. Analyze the differences between the motion direction of each object and the motion direction of other objects from each viewpoint to obtain the directional similarity value of each object from each viewpoint.

[0008] In one embodiment, analyzing the differences in directional similarity values ​​of multiple objects under each viewpoint to obtain an independent deviation value for each object under each viewpoint includes: For each object, calculate the mean of the directional similarity values ​​of multiple objects under the target viewpoint to obtain the average directional similarity value corresponding to the target viewpoint, where the target viewpoint is any one of the multiple viewpoints; For each object, a difference analysis is performed based on its directional similarity value under the target viewpoint and the average directional similarity value corresponding to the target viewpoint to obtain the independent deviation value of each object under the target viewpoint. Among them, the independent deviation value of each object under the target viewpoint is positively correlated with the absolute value of the difference between its directional similarity value under the target viewpoint and the average directional similarity value corresponding to the target viewpoint.

[0009] In one embodiment, analyzing the differences in independent deviation values ​​of each object across multiple viewpoints to obtain a comprehensive deviation value for each object includes: The fluctuation of the independent deviation values ​​of each object under multiple perspectives is analyzed to obtain the confidence value corresponding to each object. The confidence value is used to characterize the reliability of the independent deviation values ​​of the corresponding object under multiple perspectives. Based on the independent deviation value of each object under each viewpoint and the confidence value corresponding to each viewpoint, the product is calculated to obtain multiple candidate deviation values ​​for each object. Each candidate deviation value of each object is positively correlated with its independent deviation value under the corresponding viewpoint, and each candidate deviation value of each object is positively correlated with the confidence value corresponding to the corresponding viewpoint. Analyze the relative magnitudes of multiple candidate deviation values ​​for each object to obtain the comprehensive deviation value for each object. The comprehensive deviation value for each object is the largest candidate deviation value among its multiple candidate deviation values.

[0010] In one embodiment, the step of analyzing data based on the spatial distance between each region to be rendered and the target region, and the maximum comprehensive deviation value corresponding to the target region, to obtain the transmission risk value for each region to be rendered includes: Calculate the ratio of the spatial distance between the candidate region and the target region to a preset distance threshold to obtain the distance attenuation factor corresponding to the candidate region, where the candidate region is any one of multiple regions to be rendered; Data analysis is performed based on the distance attenuation factor corresponding to the candidate region and the maximum comprehensive deviation value corresponding to the target region to obtain the transmission risk value of the candidate region. The transmission risk value of the candidate region is negatively correlated with the distance attenuation factor corresponding to the candidate region, and positively correlated with the maximum comprehensive deviation value corresponding to the target region.

[0011] In one embodiment, the step of comprehensively evaluating the abnormal change value of each region to be rendered and the propagation risk value of multiple regions to be rendered to obtain the rendering priority of each region to be rendered includes: Based on the comprehensive deviation values ​​of multiple objects included in the candidate region and the historical comprehensive deviation values ​​of multiple objects, a difference analysis is performed to obtain the change in the comprehensive deviation value of each object in the candidate region. Among them, the historical comprehensive deviation value of each object is its comprehensive deviation value in the historical adjacent time period. Calculate the cumulative sum of the propagation risk values ​​of multiple regions to be rendered to obtain the total propagation risk value; The risk weight of a candidate region is obtained by calculating the ratio of its transmission risk value to the total transmission risk value. The rendering priority of a candidate region is calculated by multiplying the sum of the changes in the comprehensive deviation value of each object in the candidate region with the risk weight of the candidate region.

[0012] In one embodiment, before performing behavioral analysis based on video data and spatial data from multiple perspectives to obtain an independent deviation value for each object under each perspective, the method further includes: Collect spatial data and video data from multiple perspectives of urban security areas; Using a 3D Gaussian sputtering algorithm, based on the spatial data and video data from multiple perspectives, a 3D reconstruction of the urban security area is performed to obtain a 3D map of the urban security area, wherein multiple areas to be rendered are marked in the 3D map.

[0013] In one embodiment, before performing behavioral analysis based on video data and spatial data from multiple perspectives to obtain an independent deviation value for each object under each perspective, the method further includes: Multiple perspective video data are spatially registered with the 3D map of the urban security area to obtain the mapping relationship between video frame pixel coordinates and 3D scene spatial coordinates.

[0014] Secondly, embodiments of the present invention provide a digital twin 3D scene construction system for urban security areas, the system comprising: The behavior analysis module is used to perform behavior analysis based on video data and spatial data from multiple perspectives to obtain the independent deviation value of each object in each perspective. The independent deviation value is used to characterize the degree to which the behavior of the corresponding object deviates from the average level of the group in the corresponding perspective. The difference analysis module is used to analyze the differences in independent deviation values ​​of each object from multiple perspectives, and obtain the comprehensive deviation value of each object. The comprehensive deviation value is used to characterize the degree to which the behavior of the corresponding object deviates from the group after multi-perspective verification. The region filtering module is used to filter multiple regions to be rendered based on the comprehensive deviation value of each object to obtain the target region. The target region is used to indicate the region to be rendered where the object with the largest comprehensive deviation value is located. The comprehensive analysis module is used to perform data analysis based on the spatial distance between each region to be rendered and the target region, as well as the maximum comprehensive deviation value corresponding to the target region, to obtain the transmission risk value of each region to be rendered. The transmission risk value is used to characterize the degree of risk of the corresponding region to be rendered being affected by the target region. The 3D rendering module is used to comprehensively evaluate the abnormal change values ​​of each region to be rendered and the transmission risk values ​​of multiple regions to be rendered, to obtain the rendering priority of each region to be rendered, and to perform 3D rendering based on the rendering priority of each region to be rendered. The rendering priority is used to indicate the rendering accuracy of the corresponding region to be rendered. The rendering accuracy of the target region is higher than that of any region to be rendered. The abnormal change value of each region to be rendered is obtained by analyzing the change of the comprehensive deviation value of the multiple objects it includes in adjacent time periods.

[0015] The present invention has the following beneficial effects: This invention performs behavioral analysis based on video and spatial data from multiple perspectives to obtain the independent deviation value of each object under each perspective. It then analyzes the differences in these independent deviation values ​​across multiple perspectives to obtain the comprehensive deviation value for each object. Based on this comprehensive deviation value, multiple areas to be rendered are filtered to obtain target areas. Subsequently, data analysis is performed based on the spatial distance between each area to be rendered and the target area, as well as the maximum comprehensive deviation value corresponding to the target area, to obtain the transmission risk value for each area to be rendered. Finally, a comprehensive evaluation is conducted based on the abnormal change values ​​of each area to be rendered and the transmission risk values ​​of multiple areas to be rendered, resulting in the rendering priority for each area. 3D rendering is then performed according to this priority. This invention quantifies the degree of abnormal human behavior in each area to be rendered by analyzing the comprehensive deviation value of each object, thereby identifying high-risk areas and performing differentiated rendering for high-risk and low-risk areas. This avoids the problems of excessive computational resource consumption and slow scene loading caused by applying the same high rendering precision to all areas, thus improving the efficiency of constructing digital twin 3D scenes for urban security areas. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a method for constructing a digital twin 3D scene for urban security areas, provided by an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a digital twin 3D scene construction system for urban security areas provided in an embodiment of the present invention. Detailed Implementation

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0018] The following description, in conjunction with the accompanying drawings, details the specific scheme of the digital twin 3D scene construction method and system for urban security areas provided by this invention.

[0019] This invention proposes a method for constructing digital twin 3D scenes for urban security areas. The method provided by this invention can be executed by a digital twin 3D scene construction system for urban security areas, specifically by the controller within that system. Please refer to [link to relevant documentation]. Figure 1 The diagram illustrates a flowchart of a method for constructing a digital twin 3D scene for urban security areas, according to an embodiment of the present invention. The method includes the following steps: Step S1: Perform behavioral analysis based on video data and spatial data from multiple perspectives to obtain the independent deviation value of each object under each perspective.

[0020] The independent deviation value characterizes the degree to which the behavior of a corresponding object deviates from the average level of the group from a given viewpoint. Objects include static and dynamic objects; static objects include, for example, stationary buildings and greenery, while dynamic objects include, for example, moving pedestrians. In this embodiment of the invention, behavioral analysis based on video and spatial data from multiple viewpoints yields the independent deviation value for each object at each viewpoint. This means that behavioral analysis based on video and spatial data from multiple viewpoints yields the independent deviation value for each dynamic object at each viewpoint, thereby determining the degree of abnormality of each dynamic object and assessing the rendering priority of each region to be rendered.

[0021] Multiple perspectives refer to multiple data acquisition points observing the same urban security area from different spatial locations and angles. In other words, each perspective is equipped with video acquisition equipment to collect video data from that perspective (e.g., video data from 10 minutes prior to the current moment) and send the collected video data to the controller. The controller can timestamp and synchronize the video data from multiple perspectives, obtaining synchronized video data from multiple perspectives. Based on this synchronized video data, it then performs the subsequent process of obtaining the independent deviation value of each object under each perspective. Examples of video acquisition equipment include: drones equipped with gimbal cameras, fixed surveillance cameras deployed in urban security areas, and mobile devices such as law enforcement recorders carried by patrol personnel.

[0022] Spatial data includes the three-dimensional coordinates, boundary ranges, and physical area of ​​each zone within the urban security zone. The controller can obtain spatial data by capturing images of the urban security zone along a preset flight path using a drone equipped with a gimbal camera.

[0023] In this embodiment of the invention, before performing behavioral analysis based on video data and spatial data from multiple perspectives to obtain the independent deviation value of each object under each perspective, the method further includes: Collect spatial data and video data from multiple perspectives of urban security areas; Using a 3D Gaussian sputtering algorithm, based on the spatial data and video data from multiple perspectives, a 3D reconstruction of the urban security area is performed to obtain a 3D map of the urban security area, wherein multiple areas to be rendered are marked in the 3D map.

[0024] As described above, the controller collects spatial data and video data from multiple perspectives of the urban security area. It also stores a 3D Gaussian sputtering algorithm and can use this algorithm to perform 3D reconstruction of the video data from multiple perspectives, then fuse it with the spatial data to obtain a 3D map of the urban security area. This 3D map is unrendered; it only contains the geometric structure and static texture information of the urban security area, and the locations of each area to be rendered are marked on the map.

[0025] This invention uses a train station as an example of an urban security area to illustrate the method provided by this invention. The areas to be rendered are, for example, security checkpoint areas, waiting hall areas, ticket sales areas, and entrance / exit areas.

[0026] Optionally, after the 3D reconstruction is completed, a 3D octree construction algorithm can be used to lightweight organize and construct a spatial index for the reconstructed 3D map, thereby reducing the amount of data and accelerating spatial queries. In the subsequent rendering stage, the data required for the current rendering area can be loaded and rendered on demand according to the spatial index, avoiding memory overflow and rendering lag caused by loading all data at once, so as to achieve smooth display and interaction on the Web.

[0027] In this embodiment of the invention, before performing behavior analysis based on video data and spatial data from multiple perspectives to obtain the independent deviation value of each object under each perspective, the method further includes: Multiple perspective video data are spatially registered with the 3D map of the urban security area to obtain the mapping relationship between video frame pixel coordinates and 3D scene spatial coordinates.

[0028] The controller can establish a spatial coordinate system based on a 3D map of the urban security area. For example, the controller can define any point on the 3D map as the origin O of the spatial coordinate system, with the longitude direction (e.g., east) of the urban security area as the X-axis, the latitude direction (e.g., north) as the Y-axis, and the Z-axis perpendicular to the XOY plane. The controller can obtain the intrinsic parameter matrix (e.g., focal length and distortion coefficients) and extrinsic parameter matrix (e.g., rotation matrix and translation vector) of the video acquisition device for each viewpoint through camera calibration. It then back-projects the video frame pixel coordinates onto the 3D scene space plane using frustum projection, obtaining the mapping relationship between the video frame pixel coordinates and the 3D scene space coordinates. This mapping relationship is used to map each object detected in the video data to its specific spatial location in the 3D scene, thereby determining the rendering area to which each object belongs and providing a data basis for subsequent calculations of object density and directional similarity values ​​for each rendering area.

[0029] Analysis revealed that abnormal individual behavior (such as going against the flow of traffic) can easily trigger a chain reaction in dense crowds, leading to public safety accidents. Therefore, the degree of abnormality of individual behavior can be quantified by analyzing how much each object's behavior deviates from the average level of the group, in order to identify whether there is a risk of public safety accidents in the area to be rendered, and thus determine the rendering priority of the area to be rendered.

[0030] Specifically, the behavioral analysis based on video and spatial data from multiple perspectives to obtain independent deviation values ​​for each object at each perspective includes: Target tracking is performed based on video data and spatial data from multiple perspectives to obtain the orientation similarity value of each object in each perspective. The orientation similarity value is used to characterize the degree of similarity between the corresponding object and other objects in the corresponding perspective. The differences in directional similarity values ​​of multiple objects under each viewpoint are analyzed to obtain the independent deviation value of each object under each viewpoint. The independent deviation value is used to characterize the degree to which the behavior of the corresponding object deviates from the average level of the group under the corresponding viewpoint.

[0031] In one example, the process of tracking targets based on video data and spatial data from multiple perspectives to obtain the directional similarity value of each object in each perspective includes: Target detection is performed based on video data from multiple perspectives to obtain multiple objects from each perspective. For each object, optical flow tracing is performed based on its video data at each viewpoint to obtain the motion direction of each object at each viewpoint. Analyze the differences between the motion direction of each object and the motion direction of other objects from each viewpoint to obtain the directional similarity value of each object from each viewpoint.

[0032] The controller pre-stores object detection algorithms (such as the YOLO series). For video data from each viewpoint, the controller can use the object detection algorithm to detect objects included in the video data from each viewpoint, thus obtaining multiple objects for each viewpoint. The controller can extract the 3D spatial coordinates of the boundary points of each region to be rendered based on the location of each region in the 3D map, calculate the area of ​​the polygon enclosed by each boundary point, and obtain the area of ​​each region to be rendered.

[0033] The controller also stores sparse optical flow algorithms (such as the Lucas-Kanade optical flow method). For video data at each viewpoint, the controller can use a sparse optical flow algorithm to perform optical flow tracing on each object in the video data at each viewpoint, obtaining the motion direction of each object at each viewpoint. Here, the motion direction refers to the displacement direction of the corresponding object between consecutive video frames of the video data, that is, the motion vector direction of the object on the image plane, which is used to characterize the motion trend of the corresponding object at the corresponding viewpoint.

[0034] In one example, the process of analyzing the differences in directional similarity values ​​of multiple objects under each viewpoint to obtain the independent deviation value of each object under each viewpoint includes: For each object, calculate the mean of the directional similarity values ​​of multiple objects under the target viewpoint to obtain the average directional similarity value corresponding to the target viewpoint, where the target viewpoint is any one of the multiple viewpoints; For each object, a difference analysis is performed based on its directional similarity value under the target viewpoint and the average directional similarity value corresponding to the target viewpoint to obtain the independent deviation value of each object under the target viewpoint. Among them, the independent deviation value of each object under the target viewpoint is positively correlated with the absolute value of the difference between its directional similarity value under the target viewpoint and the average directional similarity value corresponding to the target viewpoint.

[0035] For example, at time t, the independent deviation value of the i-th object under the f-th viewpoint. It can be represented as: In the formula, At time t, the directional similarity value of the i-th object from the f-th viewpoint. Let be the mean of the directional similarity values ​​of multiple objects at time t, under the f-th viewpoint, that is, the average directional similarity value corresponding to the f-th viewpoint. A very small positive number pre-stored for the controller, for example The value is 0.001. In the embodiments of the present invention... For the normalized values, the controller can use maximum-minimum normalization. Normalize, Normalized to [0,1]. Controller pair The maximum and minimum values ​​used for normalization can also be determined based on historical extreme values ​​from prior experiments.

[0036] Step S2: Analyze the differences in independent deviation values ​​of each object from multiple perspectives to obtain the comprehensive deviation value of each object.

[0037] Among them, the comprehensive deviation value is used to characterize the degree to which the behavior of the corresponding object deviates from the group after verification from multiple perspectives.

[0038] In urban security scenarios, the same area to be rendered may be simultaneously covered by multiple video capture devices from different angles. This means the same area will appear in video data from multiple perspectives, implying that the same object will be observed from multiple viewpoints at the same time. For each object, if the fluctuation of its independent deviation value across multiple viewpoints is small, it indicates that the behavioral judgments of the object from different viewpoints corroborate each other, meaning the behavioral deviation judgments are reliable. Based on this, the controller can analyze the differences in the independent deviation values ​​of each object across multiple viewpoints to obtain the comprehensive deviation value for each object.

[0039] Specifically, the analysis of the differences in independent deviation values ​​of each object under multiple viewpoints to obtain the comprehensive deviation value of each object includes: The fluctuation of the independent deviation values ​​of each object under multiple perspectives is analyzed to obtain the confidence value corresponding to each object. The confidence value is used to characterize the reliability of the independent deviation values ​​of the corresponding object under multiple perspectives. Based on the independent deviation value of each object under each viewpoint and the confidence value corresponding to each viewpoint, the product is calculated to obtain multiple candidate deviation values ​​for each object. Each candidate deviation value of each object is positively correlated with its independent deviation value under the corresponding viewpoint, and each candidate deviation value of each object is positively correlated with the confidence value corresponding to the corresponding viewpoint. Analyze the relative magnitudes of multiple candidate deviation values ​​for each object to obtain the comprehensive deviation value for each object. The comprehensive deviation value for each object is the largest candidate deviation value among its multiple candidate deviation values.

[0040] For example, the comprehensive deviation value of the i-th object It can be represented as: In the formula, Let be the independent deviation value of the i-th object under the f-th viewpoint at time t. To determine the confidence value of the i-th object at time t, the controller can calculate the variance of the independent deviation values ​​of the i-th object across multiple viewpoints. This variance is then defined as... . Let be the candidate deviation value of the i-th object under the f-th viewpoint at time t.

[0041] Step S3: Filter multiple regions to be rendered based on the comprehensive deviation value of each object to obtain the target region.

[0042] The target region is used to indicate the rendering area containing the object with the largest overall deviation value. In other words, the target region is the source of risk. The controller can compare the overall deviation value of each object, obtain the object with the largest overall deviation value, and determine the rendering area containing that object as the target region.

[0043] Step S4: Analyze the data based on the spatial distance between each region to be rendered and the target region, as well as the maximum comprehensive deviation value corresponding to the target region, to obtain the transmission risk value of each region to be rendered.

[0044] Among them, the transmission risk value is used to characterize the degree of risk that the corresponding area to be rendered will be affected by the target area.

[0045] Analysis reveals that public safety incidents are not isolated events, and the closer an area is to the risk source, the higher the likelihood of secondary incidents or risk spread. In other words, public safety incidents exhibit significant spatial transmission characteristics, with areas closer to the risk source experiencing greater impact. Furthermore, the impact is related to the intensity of the anomaly at the risk source (i.e., the target area). The greater the anomaly intensity of objects within the target area—that is, the larger the maximum comprehensive deviation value corresponding to the target area—the stronger the anomaly intensity of that target area. Based on this, the controller can perform data analysis based on the spatial distance between each area to be rendered and the target area, as well as the maximum comprehensive deviation value corresponding to the target area, to obtain the transmission risk value for each area to be rendered, thereby determining the degree to which each area is affected by the risk from the target area.

[0046] Specifically, the process of analyzing data based on the spatial distance between each region to be rendered and the target region, and the maximum comprehensive deviation value corresponding to the target region, to obtain the transmission risk value for each region to be rendered includes: Calculate the ratio of the spatial distance between the candidate region and the target region to a preset distance threshold to obtain the distance attenuation factor corresponding to the candidate region, where the candidate region is any one of multiple regions to be rendered; Data analysis is performed based on the distance attenuation factor corresponding to the candidate region and the maximum comprehensive deviation value corresponding to the target region to obtain the transmission risk value of the candidate region. The transmission risk value of the candidate region is negatively correlated with the distance attenuation factor corresponding to the candidate region, and positively correlated with the maximum comprehensive deviation value corresponding to the target region.

[0047] For example, suppose the candidate region is the x-th region to be rendered, and the target region is the b-th region to be rendered. The propagation risk value of the candidate region... It can be represented as: In the formula, This is the maximum composite deviation value among the composite deviation values ​​of multiple objects in the b-th region to be rendered, i.e. This represents the maximum comprehensive deviation value corresponding to the target area. Let x be the distance decay factor corresponding to the x-th region to be rendered. Let D be the spatial distance between the x-th and b-th regions to be rendered. The controller calculates the 3D Euclidean distance between the center points of the x-th and b-th regions on the 3D map, and defines this Euclidean distance as the spatial distance between them. D is a preset distance threshold stored in the controller's memory; preferably, this preset distance threshold is 50 meters.

[0048] Understandably, in When D is less than or equal to, Follow It decreases as it increases, and The larger the value, the more severely the x-th region to be rendered is affected by the target region. When the value is greater than D, the controller can The value is set directly to 0, which means that the x-th region to be rendered is not affected by the target region.

[0049] Optionally, the controller can obtain the object density of each region to be rendered and weight its transmission risk value based on the object density of each region as a weighting coefficient, thereby obtaining the final transmission risk value of the region to be rendered. The controller can calculate the ratio of the number of objects included in the region to the area of ​​the region to obtain the object density of the region, where the area of ​​the region to be rendered is calculated based on spatial data. It can be understood that the object density of the region to be rendered is a normalized value. The controller can use maximum-minimum normalization to normalize the object density of the region to be rendered to [0,1], and the maximum and minimum values ​​used by the controller when normalizing the object density of the region to be rendered can be determined based on the historical extreme values ​​of pre-experimentation. In this way, the influence of crowd density on the risk diffusion intensity can be considered, thereby improving the accuracy and applicability of the transmission risk value in different density scenarios.

[0050] Step S5: Based on the abnormal change value of each region to be rendered and the transmission risk value of multiple regions to be rendered, a comprehensive evaluation is performed to obtain the rendering priority of each region to be rendered, and 3D rendering is performed according to the rendering priority of each region to be rendered.

[0051] Among them, the rendering priority is used to indicate the rendering precision of the corresponding region to be rendered. The rendering precision of the target region is higher than that of any region to be rendered. The abnormal change value of each region to be rendered is obtained by analyzing the change of the comprehensive deviation value of the multiple objects it includes in adjacent time periods.

[0052] Objects exhibiting anomalous behavior show their anomalous behavior becoming more pronounced over time, leading to greater changes in their overall deviation value. This indicates a rapid accumulation of risk in the rendering region where the object resides. Furthermore, the higher the risk of a rendering region, the greater its propagated risk value and its proportion in the total propagated risk value of all rendering regions. Therefore, for each rendering region, a comprehensive assessment can be conducted based on the change in the overall deviation value of each object within it, as well as its proportion in the total propagated risk value of all rendering regions. This yields the rendering priority for each region. This approach considers both the risk evolution trend of each rendering region and its relative importance within the overall context, allowing for a more comprehensive assessment of the actual risk of each region and improving the accuracy of the obtained rendering priority.

[0053] Specifically, the step of comprehensively evaluating the abnormal change values ​​of each region to be rendered and the propagation risk values ​​of multiple regions to be rendered to obtain the rendering priority of each region to be rendered includes: Based on the comprehensive deviation values ​​of multiple objects included in the candidate region and the historical comprehensive deviation values ​​of multiple objects, a difference analysis is performed to obtain the change in the comprehensive deviation value of each object in the candidate region. Among them, the historical comprehensive deviation value of each object is its comprehensive deviation value in the historical adjacent time period. Calculate the cumulative sum of the propagation risk values ​​of multiple regions to be rendered to obtain the total propagation risk value; The risk weight of a candidate region is obtained by calculating the ratio of its transmission risk value to the total transmission risk value. The rendering priority of a candidate region is calculated by multiplying the sum of the changes in the comprehensive deviation value of each object in the candidate region with the risk weight of the candidate region.

[0054] For example, at the current time (let's say time t), assuming the candidate region is the x-th region to be rendered, the rendering priority of the candidate region is... It can be represented as: In the formula, nx is the total number of objects included in the x-th region to be rendered at time t. Let be the overall deviation value of the i-th object in the x-th region to be rendered at time t. Let be the historical aggregated deviation value of the i-th object in the x-th region to be rendered, i.e. This represents the overall deviation value of the i-th object in the x-th region to be rendered at time t-1 (i.e., the adjacent historical time period). The risk value is the propagation value for the x-th region to be rendered. This represents the total transmission risk value. A very small positive number pre-stored for the controller, for example, The value of is 0.001. exp[ ] represents an exponential function with the natural constant e as the base.

[0055] In this embodiment of the invention, if there are no historical adjacent time periods, i.e., when performing the comprehensive evaluation for the first time, the controller can directly set the change in the comprehensive deviation value of each object in the candidate region to 0, that is... .

[0056] Optionally, in this embodiment of the invention, the controller pre-stores a deviation threshold (e.g., 0.6). The controller compares the overall deviation value of each object with the deviation threshold. If the overall deviation value of an object is greater than or equal to the deviation threshold, the controller identifies the object as an abnormal person and determines the area to be rendered where the abnormal person is located as a candidate area. If the overall deviation value of an object is less than the deviation threshold, the controller identifies the object as a normal person.

[0057] When multiple abnormal individuals are in the same candidate area, the controller can obtain the maximum value among the comprehensive deviation values ​​of the multiple abnormal individuals and determine that candidate area as the target area. Then, the rendering priority of each region to be rendered can be calculated using the method described above. When multiple abnormal individuals are in different candidate areas, for each region to be rendered, the controller can sequentially determine each candidate area as the target area and calculate the rendering priority for each region to be rendered for each determined target area. In this case, each region to be rendered receives multiple rendering priorities, and the number of rendering priorities for each region to be rendered is equal to the number of candidate areas. Then, for each region to be rendered, the controller can obtain the maximum rendering priority among its multiple rendering priorities and determine the maximum rendering priority as its final rendering priority.

[0058] The process of 3D rendering based on the rendering priority of each region is as follows: The controller determines the rendering precision of each region based on its rendering priority. Rendering precision includes at least texture resolution and frame rate. Furthermore, the higher the rendering priority of a region, the higher its rendering precision, correspondingly higher texture resolution and frame rate, and higher priority in the allocation of computing resources. This means that the region receives more GPU computing power, video memory bandwidth, and rendering pipeline resources during rendering. The controller can allocate the corresponding texture resolution and frame rate according to the rendering priority of each region and use a 3D Gaussian sputtering algorithm to perform 3D rendering on each region.

[0059] In this embodiment of the invention, when the controller detects a change in the rendering priority of any region to be rendered, it re-executes the 3D rendering to achieve dynamic recycling and reallocation of computing resources.

[0060] In summary, this invention, through behavioral analysis based on video and spatial data from multiple perspectives, obtains the independent deviation value of each object under each perspective. It then analyzes the differences in the independent deviation values ​​of each object across multiple perspectives to obtain the comprehensive deviation value of each object. Based on the comprehensive deviation value of each object, multiple areas to be rendered are filtered to obtain the target area. Subsequently, based on the spatial distance between each area to be rendered and the target area, and the maximum comprehensive deviation value corresponding to the target area, data analysis is performed to obtain the transmission risk value of each area to be rendered. Then, based on the abnormal change value of each area to be rendered and the transmission risk values ​​of multiple areas to be rendered, a comprehensive evaluation is conducted to obtain the rendering priority of each area to be rendered. 3D rendering is then performed based on the rendering priority of each area to be rendered. This invention, by analyzing the comprehensive deviation value of each object to quantify the degree of abnormal human behavior in each area to be rendered, identifies high-risk areas and performs differentiated rendering for high-risk and low-risk areas. This avoids the problems of excessive computational resource consumption and slow scene loading caused by using the same high rendering precision for all areas, thus improving the efficiency of constructing digital twin 3D scenes for urban security areas.

[0061] This invention also proposes a digital twin 3D scene construction system for urban security areas; please refer to [link / reference]. Figure 2 , Figure 2 This diagram illustrates a structural schematic of a digital twin 3D scene construction system 200 for urban security areas, provided by an embodiment of the present invention. The system includes: The behavior analysis module 201 is used to perform behavior analysis based on video data and spatial data from multiple perspectives to obtain the independent deviation value of each object in each perspective. The independent deviation value is used to characterize the degree to which the behavior of the corresponding object deviates from the average level of the group in the corresponding perspective. The difference analysis module 202 is used to analyze the differences in independent deviation values ​​of each object from multiple perspectives to obtain the comprehensive deviation value of each object. The comprehensive deviation value is used to characterize the degree to which the behavior of the corresponding object deviates from the group after multi-perspective verification. The region filtering module 203 is used to filter multiple regions to be rendered based on the comprehensive deviation value of each object to obtain the target region. The target region is used to indicate the region to be rendered where the object with the largest comprehensive deviation value is located. The comprehensive analysis module 204 is used to perform data analysis based on the spatial distance between each region to be rendered and the target region, as well as the maximum comprehensive deviation value corresponding to the target region, to obtain the transmission risk value of each region to be rendered. The transmission risk value is used to characterize the degree of risk of the corresponding region to be rendered being affected by the target region. The 3D rendering module 205 is used to comprehensively evaluate the abnormal change value of each region to be rendered and the transmission risk value of multiple regions to be rendered to obtain the rendering priority of each region to be rendered, and to perform 3D rendering according to the rendering priority of each region to be rendered. The rendering priority is used to indicate the rendering accuracy of the corresponding region to be rendered. The rendering accuracy of the target region is higher than that of any region to be rendered. The abnormal change value of each region to be rendered is obtained by analyzing the change of the comprehensive deviation value of the multiple objects it includes in adjacent time periods.

[0062] It should be noted that the system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the digital twin 3D scene construction system for urban security areas provided in the above embodiments and the digital twin 3D scene construction method for urban security areas belong to the same concept. The specific implementation process is detailed in the method embodiments and will not be repeated here.

[0063] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for constructing a digital twin 3D scene for urban security areas, characterized in that, The method includes: Behavioral analysis is performed based on video and spatial data from multiple perspectives to obtain the independent deviation value of each object in each perspective. The independent deviation value is used to characterize the degree to which the behavior of the corresponding object deviates from the average level of the group in the corresponding perspective. The differences in independent deviation values ​​of each object under multiple perspectives are analyzed to obtain the comprehensive deviation value of each object. The comprehensive deviation value is used to characterize the degree to which the behavior of the corresponding object deviates from the group after verification from multiple perspectives. Multiple regions to be rendered are filtered based on the overall deviation value of each object to obtain the target region. The target region is used to indicate the region to be rendered where the object with the largest overall deviation value is located. Based on the spatial distance between each region to be rendered and the target region, and the maximum comprehensive deviation value corresponding to the target region, data analysis is performed to obtain the transmission risk value of each region to be rendered. The transmission risk value is used to characterize the degree of risk of the corresponding region to be rendered being affected by the target region. A comprehensive evaluation is conducted based on the abnormal change values ​​of each region to be rendered and the propagation risk values ​​of multiple regions to be rendered to obtain the rendering priority of each region to be rendered. 3D rendering is then performed based on the rendering priority of each region to be rendered. The rendering priority is used to indicate the rendering accuracy of the corresponding region to be rendered. The rendering accuracy of the target region is higher than that of any region to be rendered. The abnormal change value of each region to be rendered is obtained by analyzing the change in the comprehensive deviation value of the multiple objects it includes within adjacent time periods. The step of comprehensively evaluating the abnormal change value of each region to be rendered and the propagation risk value of multiple regions to be rendered to obtain the rendering priority of each region to be rendered includes: Based on the comprehensive deviation values ​​of multiple objects included in the candidate region and the historical comprehensive deviation values ​​of multiple objects, a difference analysis is performed to obtain the change in the comprehensive deviation value of each object in the candidate region. Among them, the historical comprehensive deviation value of each object is its comprehensive deviation value in the historical adjacent time period. Calculate the cumulative sum of the propagation risk values ​​of multiple regions to be rendered to obtain the total propagation risk value; The risk weight of a candidate region is obtained by calculating the ratio of its transmission risk value to the total transmission risk value. The rendering priority of a candidate region is calculated by multiplying the sum of the changes in the comprehensive deviation value of each object in the candidate region with the risk weight of the candidate region.

2. The method for constructing a digital twin 3D scene for urban security areas according to claim 1, characterized in that, The behavior analysis based on video and spatial data from multiple perspectives yields independent deviation values ​​for each object at each perspective, including: Target tracking is performed based on video data and spatial data from multiple perspectives to obtain the orientation similarity value of each object in each perspective. The orientation similarity value is used to characterize the degree of similarity between the corresponding object and other objects in the corresponding perspective. The differences in directional similarity values ​​of multiple objects under each viewpoint are analyzed to obtain the independent deviation value of each object under each viewpoint. The independent deviation value is used to characterize the degree to which the behavior of the corresponding object deviates from the average level of the group under the corresponding viewpoint.

3. The method for constructing a digital twin 3D scene for urban security areas according to claim 2, characterized in that, The target tracking based on video and spatial data from multiple perspectives, obtaining the directional similarity value of each object in each perspective, includes: Target detection is performed based on video data from multiple perspectives to obtain multiple objects from each perspective. For each object, optical flow tracing is performed based on its video data at each viewpoint to obtain the motion direction of each object at each viewpoint. Analyze the differences between the motion direction of each object and the motion direction of other objects from each viewpoint to obtain the directional similarity value of each object from each viewpoint.

4. The method for constructing a digital twin 3D scene for urban security areas according to claim 2, characterized in that, The analysis of the differences in directional similarity values ​​of multiple objects under each viewpoint yields an independent deviation value for each object under each viewpoint, including: For each object, calculate the mean of the directional similarity values ​​of multiple objects under the target viewpoint to obtain the average directional similarity value corresponding to the target viewpoint, where the target viewpoint is any one of the multiple viewpoints; For each object, a difference analysis is performed based on its directional similarity value under the target viewpoint and the average directional similarity value corresponding to the target viewpoint to obtain the independent deviation value of each object under the target viewpoint. Among them, the independent deviation value of each object under the target viewpoint is positively correlated with the absolute value of the difference between its directional similarity value under the target viewpoint and the average directional similarity value corresponding to the target viewpoint.

5. The method for constructing a digital twin 3D scene for urban security areas according to claim 1, characterized in that, The analysis examines the differences in independent deviation values ​​for each object across multiple viewpoints to obtain a comprehensive deviation value for each object, including: The fluctuation of the independent deviation values ​​of each object under multiple perspectives is analyzed to obtain the confidence value corresponding to each object. The confidence value is used to characterize the reliability of the independent deviation values ​​of the corresponding object under multiple perspectives. Based on the independent deviation value of each object under each viewpoint and the confidence value corresponding to each viewpoint, the product is calculated to obtain multiple candidate deviation values ​​for each object. Each candidate deviation value of each object is positively correlated with its independent deviation value under the corresponding viewpoint, and each candidate deviation value of each object is positively correlated with the confidence value corresponding to the corresponding viewpoint. Analyze the relative magnitudes of multiple candidate deviation values ​​for each object to obtain the comprehensive deviation value for each object. The comprehensive deviation value for each object is the largest candidate deviation value among its multiple candidate deviation values.

6. The method for constructing a digital twin 3D scene for urban security areas according to claim 1, characterized in that, The process involves data analysis based on the spatial distance between each region to be rendered and the target region, as well as the maximum comprehensive deviation value corresponding to the target region, to obtain the transmission risk value for each region to be rendered, including: Calculate the ratio of the spatial distance between the candidate region and the target region to a preset distance threshold to obtain the distance attenuation factor corresponding to the candidate region, where the candidate region is any one of multiple regions to be rendered; Data analysis is performed based on the distance attenuation factor corresponding to the candidate region and the maximum comprehensive deviation value corresponding to the target region to obtain the transmission risk value of the candidate region. The transmission risk value of the candidate region is negatively correlated with the distance attenuation factor corresponding to the candidate region, and positively correlated with the maximum comprehensive deviation value corresponding to the target region.

7. The method for constructing a digital twin 3D scene for urban security areas according to claim 1, characterized in that, Before performing behavior analysis based on video and spatial data from multiple perspectives to obtain the independent deviation value of each object under each perspective, the method further includes: Collect spatial data and video data from multiple perspectives of urban security areas; Using a 3D Gaussian sputtering algorithm, based on the spatial data and video data from multiple perspectives, a 3D reconstruction of the urban security area is performed to obtain a 3D map of the urban security area, wherein multiple areas to be rendered are marked in the 3D map.

8. The method for constructing a digital twin 3D scene for urban security areas according to claim 7, characterized in that, Before performing behavior analysis based on video and spatial data from multiple perspectives to obtain the independent deviation value of each object under each perspective, the method further includes: Multiple perspective video data are spatially registered with the 3D map of the urban security area to obtain the mapping relationship between the video frame pixel coordinates and the 3D scene spatial coordinates.

9. A digital twin 3D scene construction system for urban security areas, characterized in that, The system includes: The behavior analysis module is used to perform behavior analysis based on video data and spatial data from multiple perspectives to obtain the independent deviation value of each object in each perspective. The independent deviation value is used to characterize the degree to which the behavior of the corresponding object deviates from the average level of the group in the corresponding perspective. The difference analysis module is used to analyze the differences in independent deviation values ​​of each object from multiple perspectives, and obtain the comprehensive deviation value of each object. The comprehensive deviation value is used to characterize the degree to which the behavior of the corresponding object deviates from the group after multi-perspective verification. The region filtering module is used to filter multiple regions to be rendered based on the comprehensive deviation value of each object to obtain the target region. The target region is used to indicate the region to be rendered where the object with the largest comprehensive deviation value is located. The comprehensive analysis module is used to perform data analysis based on the spatial distance between each region to be rendered and the target region, as well as the maximum comprehensive deviation value corresponding to the target region, to obtain the transmission risk value of each region to be rendered. The transmission risk value is used to characterize the degree of risk of the corresponding region to be rendered being affected by the target region. The 3D rendering module is used to comprehensively evaluate the abnormal change values ​​of each region to be rendered and the transmission risk values ​​of multiple regions to be rendered to obtain the rendering priority of each region to be rendered, and to perform 3D rendering based on the rendering priority of each region to be rendered. The rendering priority is used to indicate the rendering accuracy of the corresponding region to be rendered. The rendering accuracy of the target region is higher than that of any region to be rendered. The abnormal change value of each region to be rendered is obtained by analyzing the change of the comprehensive deviation value of the multiple objects it includes in adjacent time periods. The step of comprehensively evaluating the abnormal change values ​​of each region to be rendered and the propagation risk values ​​of multiple regions to be rendered to obtain the rendering priority of each region to be rendered includes: Based on the comprehensive deviation values ​​of multiple objects included in the candidate region and the historical comprehensive deviation values ​​of multiple objects, a difference analysis is performed to obtain the change in the comprehensive deviation value of each object in the candidate region. Among them, the historical comprehensive deviation value of each object is its comprehensive deviation value in the historical adjacent time period. Calculate the cumulative sum of the propagation risk values ​​of multiple regions to be rendered to obtain the total propagation risk value; The risk weight of a candidate region is obtained by calculating the ratio of its transmission risk value to the total transmission risk value. The rendering priority of a candidate region is calculated by multiplying the sum of the changes in the comprehensive deviation value of each object in the candidate region with the risk weight of the candidate region.

Images