A data statistics-based scene semantic automatic labeling method and system

Abstract

The present application relates to the technical field of autonomous mobile robot environment perception and semantic understanding, and particularly relates to a scene semantic automatic labeling method and system based on data statistics, which realizes automatic, efficient and interpretable labeling of scene semantics by performing lightweight statistics and physical feature extraction on pedestrian spatio-temporal behavior data, does not need manual pre-configuration of semantic map, and enables the robot to autonomously learn and construct a scene semantic model through initial motion data, has strong environmental adaptive ability, the statistical features used all have clear physical meanings, the semantic labeling result is easy to understand, verify and debug, the credibility of the system is enhanced, and the traditional geometric clustering is abandoned, a clustering method based on motion-residence function dichotomy and flow field topology analysis is adopted, the main motion trend is simulated through streamline tracking, region growing is performed in combination with feature similarity, and physical consistent division of the motion channel and the residence area is realized.

Description

Technical Field

[0001] This invention relates to the field of environmental perception and semantic understanding technology for autonomous mobile robots, specifically to a method and system for automatic scene semantic annotation based on data statistics. Background Technology

[0002] With the increasing prevalence of robots in large commercial venues, their need for environmental understanding has gone beyond geometric obstacle avoidance and risen to the functional semantic level (such as distinguishing passageways, rest areas, and queuing areas). The main solutions include the following: (1) Solutions based on predefined semantic maps: Before deployment, the area attributes (such as corridors and counters) are marked on the map by humans or special tools. When the robot moves, it only queries the preset tags based on its own location. Some solutions combine simple rules (such as the threshold of pedestrian traffic) to trigger behavior; (2) Solutions based on deep learning visual recognition: Using convolutional neural networks (CNN) or visual Transformer models, the images or point clouds collected by the robot are directly analyzed end-to-end to output pixel-level or region-level semantic tags (such as chairs and cash registers). Some solutions introduce temporal information and process dynamic changes through RNN / LSTM.

[0003] However, all of the above solutions have the following obvious shortcomings:

[0004] High deployment and maintenance costs and lack of adaptability: Predefined solutions require a large amount of manpower upfront and cannot adapt to dynamic changes in scene layout and customer flow patterns (such as store renovations and changes in customer flow patterns during holidays), making maintenance difficult;

[0005] The model is black box, with poor interpretability and credibility: Deep learning solutions rely on massive amounts of labeled data, the model decision-making process is not transparent, it is difficult to verify in safety-critical scenarios, and the generalization ability to scenarios outside the training set is weak.

[0006] Lack of physical semantic understanding at the behavioral level: Existing solutions focus more on the category recognition of static objects and fail to fully explore the scene functional information contained in pedestrian dynamic behavior data (such as consistency of movement direction, speed distribution, and tendency to stay).

[0007] High computational complexity and limited real-time performance: Complex deep learning models are difficult to deploy on embedded mobile platforms and cannot meet the low latency requirements of robot real-time decision-making. To address this, a method and system for automatic scene semantic annotation based on data statistics is proposed. Summary of the Invention

[0008] To address the technical problem in existing technologies of achieving automatic, efficient, and interpretable annotation of scene semantics through lightweight statistical analysis and physical feature extraction of pedestrian spatiotemporal behavior data, this invention provides a method and system for automatic annotation of scene semantics based on data statistics. By performing lightweight statistical analysis and physical feature extraction on pedestrian spatiotemporal behavior data, it achieves automatic, efficient, and interpretable annotation of scene semantics.

[0009] To address the aforementioned technical problems, this invention provides the following technical solution: a method and system for automatic scene semantic annotation based on data statistics, comprising the following steps:

[0010] S1, Obtain the basic grid map and divide it into multiple grids. ;

[0011] S2, counts pedestrian targets falling into the grid within a sampling period. Dynamic pedestrian information within;

[0012] S3, calculates the value of each grid cell based on the statistical dynamic pedestrian information. Four-dimensional feature information;

[0013] S4 aggregates discrete grids into a few continuous regions with clear semantic features;

[0014] S5 performs semantic annotation based on the four-dimensional feature information of each continuous region.

[0015] Preferably, in step S1, the basic grid map constructed based on the robot SLAM is divided into square grids of fixed size, with each square grid having a side length of... The calculation formula is:

[0016] ;

[0017] in, This represents the average walking speed of pedestrians. Indicates the system sampling period. Indicates the adjustment coefficient;

[0018] Based on the length and width of the base raster map, a detailed grid map is derived. ,in, This represents the grid cell in the i-th row and j-th column.

[0019] Preferably, in step S2, the perceptual clustering tracking module obtains the number of pedestrian targets falling into the grid within a sampling period. Dynamic pedestrian information within the system, including:

[0020] ;

[0021] in, The ID representing a dynamic pedestrian; This represents the x-coordinate of a dynamic pedestrian in the global coordinate system. This represents the y-coordinate of a dynamic pedestrian in the global coordinate system. This represents the absolute speed of a moving pedestrian. This represents the absolute velocity component of a dynamic pedestrian in the x-direction in the global coordinate system. This represents the absolute velocity component of a dynamic pedestrian in the y-direction in the global coordinate system.

[0022] Preferably, in step S3, the four-dimensional feature information includes the regional dwell tendency index (DPI), the average flow velocity index (MTVI), the grid area main direction index (MD), and the direction consistency index (DCI). The specific calculation steps include:

[0023] S31, Calculates the grid using dynamic pedestrian information. Domain residency tendency index The calculation formula is:

[0024] ;

[0025] in, This indicates the number of people staying within the grid during the sampling period. The number of pedestrians in the area who have triggered a stop event at least once, 2 consecutive times. Within a time ,but Each stop is counted once; regardless of how many times a pedestrian stops, it is counted only once. This represents the total number of people passing through the grid during the sampling period. Number of pedestrians in the area, per pedestrian It is unique, as long as the pedestrian's position ( , Falling into the grid Then it will be included Recorded only once;

[0026] S32, calculates the grid using dynamic pedestrian information. Average traffic flow velocity index The calculation formula is:

[0027] ;

[0028] in, express The total number of valid movement points in the game; Represents the set of moving points that fall onto the grid during the sampling period. All points within the range are recorded. This represents the instantaneous absolute velocity of the pedestrian at the moment of data collection. This indicates the threshold for determining the speed of pedestrian movement. Greater than At that time, the collected movement points are counted. ; Indicates the speed after processing; Represents the set of moving points that fall onto the grid during the sampling period. All points within the range are recorded.

[0029] The expression is:

[0030] ;

[0031] in, This represents the k-th point collected;

[0032] The expression is:

[0033] ;

[0034] in, Indicates the upper limit threshold for speed;

[0035] S33, via the set of moving points Computational grid Main directional index of the grid area within The distribution pattern (mode) and fine-scale directional partition histograms were obtained. ;

[0036] S34, using distribution pattern (mode) and fine-scale directional partitioning histogram Computational grid Internal directional consistency index The calculation formula is:

[0037] ;

[0038] in, Indicates the point of movement The unit direction vector; , These represent the number of movement points in each direction in bidirectional mode; unordered mode represents the grid. The direction of motion within the system does not show a statistically significant central tendency; the unidirectional mode calculates the average vector of all directional vectors and takes the magnitude of the average vector; the bidirectional mode calculates the proportion of vectors contained in the two dominant reverse flows to the total number; the multi-mode calculates the proportion of vectors covered by all identified effective main directions.

[0039] Preferably, in step S31, The value range of is [0,1]. This indicates that almost all pedestrians simply pass through quickly; this is a pure passageway. This indicates that about half of the people will stay here, and the other half will pass through directly; this is a mixed-use area. This indicates that almost everyone who enters will stay here, making it a strong attraction or resistance point.

[0040] Preferably, step S33 includes the following steps:

[0041] S331, Coarse-scale directional partitioning: Divide the 360-degree direction into multiple intervals and calculate the set of moving points. The significant direction interval to which the moving point belongs is expressed as:

[0042] ];

[0043] ;

[0044] ;

[0045] in, This indicates the direction of movement of the k-th point in the set of moving points. ; Indicates the width of the partition in the coarse-scale direction; The number of partitions in the coarse-scale direction is represented by... Decide; Indicates that the index is The histogram of the number of moving points in the coarse-scale directional partition, i.e., the number of points falling on the first... The number of moving points within each directional partition; This represents the scaling factor, which controls the strictness of the interval selection. The larger the value, the fewer significant directional intervals are selected. The threshold representing the salient directional interval is a dynamic threshold. If the number of moving points within a coarse-scale directional partition exceeds this value, it is considered a salient directional interval. Represents the set of indexes for significant directional intervals;

[0046] S332, Fine-scale directional partitioning: Calculate the peak direction of the moving points within each significant directional interval obtained in step S331, and initialize the histogram of the number of moving points in the fine-scale directional partitioning. The expression is:

[0047] ;

[0048] in, Indicates that the index is The histogram of the number of moving points within the fine-scale directional partition, i.e., the number of points falling on the first... The number of moving points within each directional partition; Indicates the number of partitions in the fine-scale direction;

[0049] Then, statistics are performed for each significant directional interval, with the known index being... The angular range of the significant direction interval is ], then for each The calculation formula is:

[0050] ;

[0051] ;

[0052] in, Indicates that the index is The angular range of the significant directional interval; Indicates the width of the partition in the fine-scale direction; Indicates in Angle of moving point within range Indexes relative to subdivided scale partitions;

[0053] Then, based on the principle of larger adjacent ratios, the set of peak angle partitions within the significant directional interval based on fine-scale directional partitions is calculated. The calculation formula is as follows:

[0054] ;

[0055] in, The set of peak angle partitions representing fine-scale directional partitions;

[0056] S333, Peak Significance Filtering: Filtering out peak angle partitions with a number of moving points greater than [a certain value]. The set of thresholds is expressed as:

[0057] ;

[0058] in, This represents the set after peak angle partitioning. A threshold is set based on the number of moving points within a certain peak angle partition. If the number of moving points within that peak angle partition exceeds a certain threshold, then... Record only when necessary; Indicates the scaling factor; The threshold for judging the number of moving points within the fine-scale peak angle partition is a dynamic threshold used to filter out false peak moving points generated by a few noise points.

[0059] Furthermore, through The set of peak center angles for fine-scale directional partitions is calculated. The calculation formula is:

[0060] ;

[0061] in, Represents a set The angle is processed to obtain the centerline direction of each subdivision scale directional partition; | represents a set The number of elements contained;

[0062] Therefore, according to The number of elements and the corresponding peak center angle calculation region distribution pattern (mode) are used to derive the grid. Pedestrian walking direction information:

[0063] ;

[0064] in, This indicates that when the mesh mode is determined to be unidirectional, the mesh... The main direction angle of the region; This indicates that when the grid mode is determined to be bidirectional, the grid... The two main directions of the region; This indicates that when the mesh mode is determined to be multi-directional, the mesh... Multiple main directional angles.

[0065] Preferably, step S4 includes the following steps:

[0066] S41, traverse all discrete meshes Set a dwell tendency threshold The motion candidate set is obtained based on the DPI value of the grid. With residency candidate set Specifically:

[0067] ;

[0068] in, ;

[0069] S42, in the motion candidate set In the middle, subcategories are formed based on speed differences;

[0070] S43, from the non-motion candidate set, identify continuous regions with significant dwell behavior characteristics: traverse all discrete grids. All The grid markers are high-resident-prone seeds, among which Starting from each seed grid, grow towards its 8-connected neighbor grids. Merging will proceed if the following conditions are met: Furthermore, its difference from the average DPI value of the currently merged region is less than the threshold. ,Right now:

[0071] ;

[0072] After the merger, the area of ​​residence was obtained. .

[0073] Preferably, step S42 includes the following steps:

[0074] S421, Identify the candidate set of motions All grids with DCI values ​​being local maxima ,get The expression is:

[0075] ;

[0076] in, ;

[0077] S422, Set seed grid The coordinates of the center point are By seed grid The MD information of eight adjacent grids is weighted to obtain the weighted average direction. The calculation formula is:

[0078] ;

[0079] ;

[0080] Then along Heading forward Distance, point :

[0081] ;

[0082] in, Indicates the length of a grid;

[0083] Then calculate the point The grid it is located in And so on, until... The grid in question does not belong to Then the streamline generation ends;

[0084] S423, After completing the forward exploration in step S422, then from the coordinates... Perform reverse exploration, setting the step size to... Repeat the above process until the end, resulting in a grid. Explored skeletal path After traversing all the seed grids, a set of skeleton paths is obtained. ;

[0085] S424, Set skeleton path The initial region formed by the traversed grid is Determine adjacent grids Should it be merged into the region? The formula for judgment is:

[0086] ;

[0087] in: Indicates adjacent grids The strength of directional consistency; Indicates the region The average DCI across all grids; Indicates adjacent grids The main direction; Indicates the region All grids The average value; The tolerance scale for angular deviation in the main motion direction; Indicates the weight parameter, adjustment and The proportion of the two items; Indicates adjacent grids The index of tendency to reside in the area; Indicates the weight of the residency attribute penalty;

[0088] If adjacent grid Belongs to the candidate set of motion And it may not necessarily be classified under any In the middle, the calculation is performed by judging the formula, if Among them, the merging threshold Then the grid will be Merged into the region Update and iterate The adjacent grids are merged until no new qualified grids can be merged, thus obtaining multiple initial motion regions. ;

[0089] The initial movement areas are further merged to obtain a larger area. Specifically, for any two adjacent regions and If the following two conditions are met, then they will be merged.

[0090] ;

[0091] in, ;

[0092] Calculate the adjacency relationships between all pairs of regions. For each pair of adjacent regions, calculate their orientation difference and average DCI difference. If both conditions are met, merge the two regions into a new region. Repeat this process until no more pairs of regions meet the merging conditions. This yields one or more merged motion regions. Each region contains four-dimensional feature information, expressed as:

[0093] .

[0094] Preferably, in step S5, when , , , When it is in a single direction or two approximately opposite directions, it is a forward motion channel; when , , When null or the number of directions is greater than 2, it is a mixed / converging motion channel; when , , , When null or size > 2, it is an active resident area; when , , When null or the number of directions is greater than 2, it is a static dwell area.

[0095] This invention also discloses an automatic scene semantic annotation system based on data statistics, comprising:

[0096] The environment discretization module is used to acquire and divide the basic grid map into multiple grids. ;

[0097] The statistics module is used to count the number of pedestrian targets that fall into the grid within a given period. Dynamic pedestrian information within;

[0098] The spatiotemporal feature calculation module is used to calculate the features of each grid based on the statistically derived dynamic pedestrian information. Four-dimensional feature information;

[0099] An adaptive clustering module is used to aggregate discrete grids into a few contiguous regions with clear semantic features;

[0100] The semantic annotation module is used to perform semantic annotation based on the four-dimensional feature information of each continuous region.

[0101] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0102] 1. This invention does not require manual pre-configuration of semantic maps. The robot can autonomously learn and build a scene semantic model through initial motion data, which has a strong environmental adaptability. The statistical features used, such as direction consistency, dwell tendency, and typical speed, all have clear physical meanings. The semantic annotation results are easy to understand, verify, and debug, which enhances the credibility of the system.

[0103] 2. This invention abandons traditional geometric clustering and adopts a clustering method based on motion-dwelling functional dichotomy and flow field topology analysis. It simulates the mainstream motion trend by streamline tracing and combines feature similarity to perform region growth, thereby achieving a physical consistency division of motion channels and dwelling areas.

[0104] 3. This invention establishes a lightweight rule engine based on multi-dimensional feature combinations of DCI, DPI, MTVI, and MD, which maps the statistical feature space to high-level semantic labels, such as efficient channels, intersection areas, and active residence areas, to achieve automatic semantic generation without training. Attached Figure Description

[0105] Figure 1 This is a schematic diagram of the automatic scene semantic annotation method of the present invention. Detailed Implementation

[0106] The present invention will be further described below with reference to the accompanying drawings and embodiments, which illustrate the above and other technical features and advantages of the present invention. However, the following embodiments are merely preferred embodiments of the present invention and are not exhaustive.

[0107] Example:

[0108] like Figure 1 As shown, this invention provides a method and system for automatic scene semantic annotation based on data statistics, including the following steps:

[0109] S1, Obtain the basic grid map and divide it into multiple grids. ;

[0110] S2, counts pedestrian targets falling into the grid within a sampling period. Dynamic pedestrian information within;

[0111] S3, calculates the value of each grid cell based on the statistical dynamic pedestrian information. Four-dimensional feature information;

[0112] S4 aggregates discrete grids into a few continuous regions with clear semantic features;

[0113] S5 performs semantic annotation based on the four-dimensional feature information of each continuous region.

[0114] In this embodiment, in step S1, the basic grid map constructed based on the robot SLAM is divided into square grids of fixed size, with each square grid having a side length of... The calculation formula is:

[0115] ;

[0116] in, This indicates the average walking speed of pedestrians (default is 1.5m / s). Indicates the system sampling period (default is 3s). This indicates the adjustment factor (default is 1.5).

[0117] Based on the length and width of the base raster map, a detailed grid map is derived. ,in, This represents the grid cell in the i-th row and j-th column.

[0118] In this embodiment, in step S2, the perceptual clustering tracking module obtains the number of pedestrian targets falling into the grid within a sampling period. Dynamic pedestrian information within the system, including:

[0119] ;

[0120] in, The ID of a dynamic pedestrian (obtained by clustering and tracking by the perception module; it is a unique attribute of an object). This represents the x-coordinate of a dynamic pedestrian in the global coordinate system. This represents the y-coordinate of a dynamic pedestrian in the global coordinate system. This represents the absolute speed of a moving pedestrian. This represents the absolute velocity component of a dynamic pedestrian in the x-direction in the global coordinate system. This represents the absolute velocity component of a dynamic pedestrian in the y-direction in the global coordinate system.

[0121] In this embodiment, step S3, the four-dimensional feature information includes the regional dwell tendency index DPI, the average flow velocity index MTVI, the grid area main direction index MD, and the direction consistency index DCI. The specific calculation steps include:

[0122] S31, Calculates the grid using dynamic pedestrian information. Domain residency tendency index Used to measure entry into the grid The proportion of pedestrians who stop in a given area reflects the strength of the attributes within that area that prompt pedestrians to change from passing through to stopping. The calculation formula is as follows:

[0123] ;

[0124] in, This indicates the number of people staying within the grid during the sampling period. The number of pedestrians in the area who have triggered a stop event at least once, 2 consecutive times. Within a time , but Each stop is counted once; regardless of how many times a pedestrian stops, it is counted only once. This represents the total number of people passing through the grid during the sampling period. The number of pedestrians in the area (whose trajectory points fall into the grid), and the number of pedestrians per person. It is unique, as long as the pedestrian's position ( , Falling into the grid Then it will be included Recorded only once;

[0125] The value range of is [0,1]. This indicates that almost all pedestrians simply pass through quickly; this is a pure passageway. This indicates that about half of the people will stay here, and the other half will pass through directly; this is a mixed-use area (such as the exhibition hall entrance). This indicates that almost everyone who enters will stop here, making it a strong attraction or obstacle (such as rest seats or popular booths).

[0126] S32, calculates the grid using dynamic pedestrian information. Average traffic flow velocity index This metric measures the typical speed of pedestrians moving within a specific area, reflecting the smoothness and efficiency of traffic flow in that area. It excludes the influence of static or quasi-static behaviors such as stopping or loitering. The calculation formula is as follows:

[0127] ;

[0128] in, express The total number of valid movement points in the game; Represents the set of moving points that fall onto the grid during the sampling period. All points within the range are recorded. This represents the instantaneous absolute velocity of the pedestrian at the moment of data collection. This indicates the speed threshold for determining pedestrian movement; the default value is 0.2 m / s. Greater than At that time, the collected movement points are counted. ; Indicates the speed after processing;

[0129] The expression is:

[0130] ;

[0131] in, This represents the k-th point collected;

[0132] The expression is:

[0133]

[0134] in, This indicates the upper limit threshold for speed, which defaults to 2.5 m / s. This operation will truncate abnormally high speeds (such as running or data noise) to the upper limit to ensure the robustness of the indicator.

[0135] S33, via the set of moving points Computational grid Main directional index of the grid area within The main direction refers to the most representative angle among the movement directions of all moving points in a grid within the statistical period. It represents the mainstream trend of pedestrian movement in that area. The direction of pedestrian movement is based on... and The calculations are as follows:

[0136] S331, coarse-scale directional partitioning, statistically identifying significant directional intervals: dividing the 360-degree direction into multiple intervals, and calculating the set of moving points. The significant direction interval to which the moving point belongs, where there are one or more significant direction intervals, is expressed as:

[0137] ];

[0138] ;

[0139] ;

[0140] in, This indicates the direction of movement of the k-th point in the set of moving points. ; Indicates the width of the partition in the coarse-scale direction, the default value is... ; The number of partitions in the coarse-scale direction is represented by... Decide; Indicates that the index is The histogram of the number of moving points in the coarse-scale directional partition, i.e., the number of points falling on the first... The number of movement points within each directional partition, Indicates the sequence number of the coarse-scale directional partition; This represents the scaling factor, which defaults to 0.1 and controls the strictness of the interval filtering. The larger the value, the fewer significant directional intervals are selected; The threshold representing the salient directional interval is a dynamic threshold. If the number of moving points within a coarse-scale directional partition exceeds this value, it is considered a salient directional interval. Represents the set of indexes for significant directional intervals;

[0141] S332, Fine-scale orientation partitioning, determining the peak direction within significant orientation intervals: Calculate the peak direction of the moving point within each significant orientation interval obtained in step S331, and initialize the fine-scale orientation partitioning histogram: Then for each significant directional interval ( ) Perform detailed statistics, the known index is The angular range of the significant direction interval is ], then for each The calculation formula is:

[0142] ;

[0143] ;

[0144] in, Indicates that the index is The angular range of the significant direction intervals, and the coarse-scale partitions share a common index with the significant direction intervals; Indicates the width of the fine-scale partition, the default value is... ; Indicates in Angle of moving point within range Indexes relative to subdivided scale partitions; Indicates the number of partitions in the fine-scale direction. ;

[0145] Histograms of the number of moving points for fine-scale directional partitions within all salient directional intervals were obtained. Then, based on the principle of larger adjacent ratio (i.e., the number of histograms for this partition is greater than the number of histograms for its neighboring partitions), the set of peak angle partitions based on fine-scale directional partitions within the salient directional intervals was calculated. The calculation formula is as follows:

[0146] ;

[0147] in, The set of peak angle partitions representing fine-scale directional partitions; Indicates that the index is The histogram of the number of moving points in the fine-scale directional partitions, i.e., those falling on the th... The number of moving points within each directional partition;

[0148] S333, Significance screening of candidate peaks, and based on their quantity and geometric relationship, the final directional distribution pattern of the decision region is determined: Peak significance screening, selecting peaks with a number of moving points greater than [a certain value] within the peak angle partition. The set of thresholds:

[0149] ;

[0150] Furthermore, through The set of peak center angles for fine-scale directional partitions is calculated:

[0151] ;

[0152] Therefore, according to The distribution pattern (mode) of the region can be calculated from the number of peaks and the corresponding peak center angles. The pseudocode is as follows:

[0153] switch(| |):

[0154] Case 0: mode = unordered

[0155] Case 1: mode = one-way

[0156] Case 2:

[0157] Calculate the angle difference between the two peaks. .

[0158] If mode = bidirectional

[0159] Else: mode = multi-directional

[0160] Default(>2): mode = multi-directional

[0161] Therefore, the mesh can be obtained. Pedestrian walking direction information:

[0162] ;

[0163] in, This represents the set after peak angle partitioning. A threshold is set based on the number of moving points within a certain peak angle partition. If the number of moving points within that peak angle partition exceeds a certain threshold, then... Record only when necessary; This represents the scaling factor, which defaults to 0.1 and controls the strictness of peak filtering. The threshold for judging the number of moving points within the fine-scale peak angle partition is a dynamic threshold used to filter out false peak moving points generated by a few noise points. Represents a set Processing the angles within the range involves obtaining the centerline direction of each subdivision's directional partition; | represents a set The number of elements contained; Indicates when | When |, calculate the difference between these two angles, which is mainly used to determine whether it is a dual-channel system.

[0164] S34, using distribution pattern (mode) and fine-scale directional partitioning histogram Computational grid Internal directional consistency index This scalar value, used to quantify the concentration or uniformity of motion directions within a local region, ranges from 0 (completely disordered) to 1 (completely uniform). The calculation formula is:

[0165] ;

[0166] in, Indicates the point of movement The unit direction vector; , In bidirectional mode, the number of movement points in each direction is represented respectively; in unordered mode, the number of movement points in each direction is represented by the grid. The movement direction within the channel shows no statistically significant concentration trend, so the DCI is set to zero, indicating that the direction is completely dispersed and there is no dominant flow direction. The unidirectional mode calculates the average vector of all direction vectors, which measures the concentration of the direction distribution and takes the magnitude of the average vector. When the magnitude of the average vector is ≈1, all vector directions are almost identical (high-intensity unidirectional flow). When the magnitude of the average vector is ≈0, the vector directions are evenly dispersed (although it is identified as having a main peak, the consistency is very weak). The bidirectional mode calculates the proportion of vectors contained in two dominant reverse flows (such as back-and-forth pedestrian flow in a channel) to the total number of vectors, which measures the purity of the bidirectional structure. A high proportion indicates that almost all movement is concentrated in these two opposite directions, while a low proportion indicates that a considerable portion of the movement is not in the main direction, indicating the presence of intersection, lateral movement, or noise in the channel. The multi-mode calculates the proportion of vectors covered by all identified effective main directions, which measures the concentration of movement directions at multi-directional intersections. A high proportion indicates that the movement is highly concentrated in a limited number of main directions. A low proportion indicates that the movement directions are very dispersed, and even if multiple peaks are identified, each direction only accounts for a small portion.

[0167] In this embodiment, step S4 includes the following steps:

[0168] S41, traverse all discrete meshes According to the grid The core functions (pass through or stay) are initially divided to provide input for different subsequent clustering strategies, and a dwell tendency threshold is set. The motion candidate set is obtained based on the DPI value of the grid. With residency candidate set Specifically:

[0169] ;

[0170] in, .

[0171] S42, in the motion candidate set In the process, continuous and consistent motion channels are identified, and subclasses are made based on speed differences. The specific steps are as follows:

[0172] S421, find the grid with the highest directional consistency located at the core of the pure motion channel. As the starting point for region growth, specifically: finding the candidate set of motions. All grids with DCI values ​​being local maxima (That is, the DCI of this point is higher than all its adjacent grids), in order to exclude grids located at the end point of the movement (such as the exit of the channel). Additional requirement: seed mesh The DPI must be very low, with the following constraints:

[0173] ;

[0174] in, Used to ensure seed grid It hardly causes any pauses; it is the starting point of pure movement.

[0175] S422, each seed grid ( Each of these has known center point coordinates, DPI, MTVI, MD information, setting seed grid The coordinates of the center point are By seed grid 8 adjacent grids The MD information is weighted to obtain the weighted average direction. :

[0176] ;

[0177] ;

[0178] Then along Heading forward Distance, point :

[0179] ;

[0180] in, Represents a grid Length;

[0181] Then calculate the point The forward grid And so on, until point The forward grid Not belonging to Then the streamline generation ends;

[0182] S423, After completing the forward exploration in step S422, then from the coordinates... Perform reverse exploration, setting the step size to... Repeat the above steps to perform reverse exploration until the end, thus obtaining a path from the seed grid. Explored skeletal path , It records a series of coordinate points, traversing all seed grids. Then, the skeleton path set is obtained. ;

[0183] S424, for the skeleton path set For each skeleton path in the image, move a similar mesh around it. Aggregate to form a continuous region.

[0184] First, set the skeleton path. Grids that pass through The initial region formed is Determine adjacent grids Should it be merged into the region? The formula for judgment is:

[0185] ;

[0186] in: Indicates adjacent grids The strength of directional consistency; Indicates the region The average DCI across all grids; Indicates adjacent grids The main direction; Indicates the region All grids The average value (obtained by averaging the region vectors; for unidirectional or bidirectional dominant regions, calculate the average value in both directions). This indicates the tolerance scale for angular deviation in the main motion direction, defaulting to... ; Indicates the weight parameter, adjustment and The ratio of the two items is 0.5 by default; Indicates adjacent grids The index of tendency to reside in the area; Indicates the weight of the residency attribute penalty, defaulting to 0.3;

[0187] If adjacent grid Belongs to the candidate set of motion And it may not necessarily be classified under any In the middle, the calculation is performed by judging the formula, if (Merge Threshold) ), then adjacent grids Merged into the region Updates and iterations Adjacent grids until no new qualified mesh is available. Until it can be merged in.

[0188] Then, the regions are merged to obtain multiple potentially adjacent preliminary motion regions. ;

[0189] Further merging the initial motion areas yields a larger area. For any two adjacent regions and If the following two conditions are met, then merge them:

[0190] ;

[0191] in, ;

[0192] Calculate the adjacency relationships between all pairs of regions. For each pair of adjacent regions, calculate their directional difference and average DCI difference. If both conditions are met, merge the two regions into a new region. Repeat this process until no more pairs of regions meet the merging conditions.

[0193] At this point, one or more merged motion regions are obtained. Each region contains four key indicators, expressed as follows:

[0194] .

[0195] S43 identifies continuous regions with significant dwell behavior characteristics from the non-motion candidate set. These regions typically correspond to rest areas, waiting areas, and viewpoints. The main method is density clustering based on spatial location and DPI values, specifically: traversing all discrete grids. All The grid markers are high-resident-prone seeds, among which, Starting from each seed grid, grow towards its 8-connected neighbor grids. Merging will proceed if the following conditions are met: Furthermore, its difference from the average DPI value of the currently merged region is less than the threshold. , where, default ,Right now:

[0196] ;

[0197] Thus, a series of residence areas were obtained. .

[0198] In this embodiment, in step S5, based on the four indicators of each continuous region... The specific physical semantics of the region are defined, as shown in Table 1.

[0199] Table 1 Semantic Tagging - Final Classification and Naming Table

[0200]

[0201] In this embodiment, a computer-readable storage medium is also disclosed, on which a computer program is stored, which, when executed by a processor, implements the steps of an automatic scene semantic annotation method based on data statistics.

[0202] The above description is merely a preferred embodiment of the present invention and is illustrative rather than restrictive. Those skilled in the art will understand that many changes, modifications, and even equivalents can be made within the spirit and scope defined by the claims of the present invention, all of which will fall within the protection scope of the present invention.

Claims

1. A method for automatic semantic annotation of scenes based on data statistics, characterized in that, Includes the following steps: S1, Obtain the basic grid map and divide it into multiple grids. ; S2, counts pedestrian targets falling into the grid within a sampling period. Dynamic pedestrian information within; S3, calculates the value of each grid cell based on the statistical dynamic pedestrian information. Four-dimensional feature information; S4 aggregates discrete grids into continuous regions with clear semantic features; S5 performs semantic annotation based on the four-dimensional feature information of each continuous region.

2. The automatic scene semantic annotation method based on data statistics as described in claim 1, characterized in that, In step S1, the basic grid map constructed based on the robot SLAM is divided into square grids of fixed size, with each square grid having a side length of... The calculation formula is: ； in, This represents the average walking speed of pedestrians. Indicates the system sampling period. Indicates the adjustment coefficient; Based on the length and width of the base raster map, a detailed grid map is derived. ,in, This represents the grid cell in the i-th row and j-th column.

3. The automatic scene semantic annotation method based on data statistics as described in claim 1, characterized in that, In step S2, the perceptual clustering tracking module obtains the number of pedestrian targets falling into the grid within a sampling period. Dynamic pedestrian information within the system, including: ； in, The ID representing a dynamic pedestrian; This represents the x-coordinate of a dynamic pedestrian in the global coordinate system. This represents the y-coordinate of a dynamic pedestrian in the global coordinate system. This represents the absolute speed of a moving pedestrian. This represents the absolute velocity component of a dynamic pedestrian in the x-direction in the global coordinate system. This represents the absolute velocity component of a dynamic pedestrian in the y-direction in the global coordinate system.

4. The automatic scene semantic annotation method based on data statistics as described in claim 1, characterized in that, Step S3 involves four-dimensional feature information, including the regional dwell tendency index (DPI), the average flow velocity index (MTVI), the grid area main direction index (MD), and the direction consistency index (DCI). The specific calculation steps include: S31, Calculates the grid using dynamic pedestrian information. Domain residency tendency index The calculation formula is: ； in, This indicates the number of people staying within the grid during the sampling period. The number of pedestrians in the area who have triggered a stop event at least once, 2 consecutive times. Within a time ,but Each stop is counted once; regardless of how many times a pedestrian stops, it is counted only once. This represents the total number of people passing through the grid during the sampling period. Number of pedestrians in the area, per pedestrian It is unique, as long as the pedestrian's position ( , Falling into the grid Then it will be included Recorded only once; S32, calculates the grid using dynamic pedestrian information. Average traffic velocity index The calculation formula is: ； in, express The total number of valid movement points in the game; Represents the set of moving points that fall onto the grid during the sampling period. All points within the range are recorded. This represents the instantaneous absolute velocity of the pedestrian at the moment of data collection. This indicates the threshold for determining the speed of pedestrian movement. Greater than At that time, the collected movement points are counted. ; Indicates the speed after processing; Represents the set of moving points that fall onto the grid during the sampling period. All points within the range are recorded. The expression is: ； in, This represents the k-th point collected; The expression is: ； in, Indicates the upper limit threshold for speed; S33, via the set of moving points Computational grid Main directional index of the grid area within The distribution pattern (mode) and fine-scale directional partition histograms were obtained. ; S34, using distribution pattern (mode) and fine-scale directional partitioning histogram Computational grid Internal directional consistency index The calculation formula is: ； in, Indicates the point of movement The unit direction vector; , These represent the number of movement points in each direction in bidirectional mode; unordered mode represents the grid. The direction of motion within the system does not show a statistically significant central tendency; the unidirectional mode calculates the average vector of all directional vectors and takes the magnitude of the average vector; the bidirectional mode calculates the proportion of vectors contained in the two dominant reverse flows to the total number; the multi-mode calculates the proportion of vectors covered by all identified effective main directions.

5. The automatic scene semantic annotation method based on data statistics as described in claim 4, characterized in that, In step S31 The value range is [0,1]. This indicates that all pedestrians are simply passing through quickly, making it a pure passageway; This indicates that half of the people will stay here, and the other half will pass through directly; it is a mixed-use area. This indicates that everyone who enters will stay here, making it a strong attraction point or a resistance point.

6. The automatic scene semantic annotation method based on data statistics as described in claim 4, characterized in that, Step S33 includes the following steps: S331, Coarse-scale directional partitioning: Divide the 360-degree direction into multiple intervals and calculate the set of moving points. The significant direction interval to which the moving point belongs is expressed as: ]； ； ； in, This indicates the direction of movement of the k-th point in the set of moving points. ; Indicates the width of the partition in the coarse-scale direction; The number of partitions in the coarse-scale direction is represented by... Decide; Indicates that the index is The histogram of the number of moving points in the coarse-scale directional partition, i.e., the number of points falling on the first... The number of movement points within each directional partition, Indicates the sequence number of the coarse-scale directional partition; This represents the scaling factor, which controls the strictness of the interval selection. The larger the value, the fewer significant directional intervals are selected. The threshold representing the salient directional interval is a dynamic threshold. If the number of moving points within a coarse-scale directional partition exceeds this value, it is considered a salient directional interval. Represents the set of indexes for significant directional intervals; S332, Fine-scale directional partitioning: Calculate the peak direction of the moving points within each significant directional interval obtained in step S331, and initialize the histogram of the number of moving points in the fine-scale directional partitioning. The expression is: ； in, Indicates that the index is The histogram of the number of moving points within the fine-scale directional partition, i.e., the number of points falling on the first... The number of movement points within each directional partition; Indicates the number of partitions in the fine-scale direction; Then, statistics are performed on each significant directional interval, with the known index being... The angular range of the significant direction interval is ], then for each The calculation formula is: ； ； in, Indicates that the index is The angular range of the significant directional interval; Indicates the width of the partition in the fine-scale direction; Indicates in Angle of moving point within range Indexes relative to subdivided scale partitions; Then, based on the principle of larger adjacent ratios, the set of peak angle partitions within the significant directional intervals based on fine-scale directional partitions is calculated. The calculation formula is as follows: ； in, The set of peak angle partitions representing fine-scale directional partitions; S333, Peak Significance Filtering: Filtering out peak angle partitions with a number of moving points greater than [a certain value]. The set of thresholds is expressed as: ； in, This represents the set after peak angle partitioning. A threshold is set based on the number of moving points within a certain peak angle partition. If the number of moving points within that peak angle partition exceeds a certain threshold, then... Record only when necessary; Indicates the scaling factor; The threshold for judging the number of moving points within the fine-scale peak angle partition is a dynamic threshold used to filter out false peak moving points generated by a few noise points. Furthermore, through The set of peak center angles for fine-scale directional partitions is calculated. The calculation formula is: ； in, Represents a set The angle is processed to obtain the centerline direction of each subdivision scale directional partition; | represents a set The number of elements contained; Therefore, according to The number of elements and the corresponding peak center angle calculation region distribution pattern (mode) are used to derive the grid. Pedestrian walking direction information: ； in, This indicates the main direction angle of the grid region when the grid mode is determined to be unidirectional; This indicates the angles of the principal directions of the two regions of the mesh when the mesh mode is determined to be bidirectional; This indicates the multiple principal direction angles of the mesh when the mesh mode is determined to be multi-directional.

7. The automatic scene semantic annotation method based on data statistics as described in claim 1, characterized in that, Step S4 includes the following steps: S41, traverse all discrete meshes Set a dwell tendency threshold According to the grid The DPI is worth the motion candidate set With residency candidate set Specifically: ； in, ; S42, in the motion candidate set In the middle, subcategories are formed based on speed differences; S43, from the non-motion candidate set, identify continuous regions with significant dwell behavior characteristics: traverse all discrete grids. All The grid markers are high-resident-prone seeds, among which Starting from each seed grid, grow towards its 8-connected neighbor grids. The following conditions must be met for a merger to proceed: Furthermore, its difference from the average DPI value of the currently merged region is less than the threshold. ,Right now: ； After the merger, the area of ​​residence was obtained. .

8. The automatic scene semantic annotation method based on data statistics as described in claim 1, characterized in that, Step S42 includes the following steps: S421, Identify the candidate set of motions All grids with DCI values ​​being local maxima ,get The expression is: ； in, ; S422, Set seed grid The coordinates of the center point are By seed grid 8 adjacent grids The MD information is weighted to obtain the weighted average direction. The calculation formula is: ； ； Then along Heading forward Distance, point : ； in, Represents a grid Length; Then calculate the point The forward grid And so on, until... The forward grid Not belonging to Then the streamline generation ends; S423, After completing the forward exploration in step S422, then from the coordinates... Perform reverse exploration, setting the step size to... Repeat the above process until the end, resulting in a grid formed by the seed mesh. Explored skeletal path Traverse all seed grids Then, the skeleton path set is obtained. ; S424, Set skeleton path Grids that pass through The initial region formed is Determine adjacent grids Should it be merged into the region? The formula for judgment is: ； in: Indicates adjacent grids The strength of directional consistency; Indicates the area The average DCI across all grids; Indicates adjacent grids The main direction; Indicates the area All grids The average value; The tolerance scale for angular deviation in the main motion direction; Indicates the weight parameter, adjustment and The proportion of the two items; Indicates adjacent grids The index of tendency to reside in the area; Indicates the weight of the residency attribute penalty; If adjacent grid Belongs to the candidate set of motion And it may not necessarily be classified under any In the middle, the calculation is performed by judging the formula, if Then the adjacent grids will be Merged into the region Among them, the merging threshold Update and iterate Adjacent grids until no new qualified mesh is available. Multiple initial motion regions can be obtained by merging them in. ; The initial motion regions are further merged to obtain the region. Specifically, for any two adjacent regions and If the following two conditions are met, then they will be merged. ； in, ; Calculate the adjacency relationships between all pairs of regions. For each pair of adjacent regions, calculate their orientation difference and average DCI difference. If both conditions are met, merge the two regions into a new region. Repeat this process until no more pairs of regions meet the merging conditions. This yields one or more merged motion regions. Each region contains four-dimensional feature information, expressed as: 。 9. The automatic scene semantic annotation method based on data statistics as described in claim 1, characterized in that, In step S5, when , , , When it is in a single direction or two approximately opposite directions, it is a forward motion channel; when , , When null or the number of directions is greater than 2, it is a mixed / converging motion channel; when , , , When null or size > 2, it is an active resident area; when , , When null or the number of directions is greater than 2, it is a static dwell area.

10. An automatic scene semantic annotation system, characterized in that, The automatic scene semantic annotation method based on data statistics as described in any one of claims 1-9 includes: The environment discretization module is used to acquire and divide the basic grid map into multiple grids. ; The statistics module is used to count the number of pedestrian targets that fall into the grid within a given period. Dynamic pedestrian information within; The spatiotemporal feature calculation module is used to calculate the features of each grid based on the statistically derived dynamic pedestrian information. Four-dimensional feature information; An adaptive clustering module is used to aggregate discrete grids into continuous regions with well-defined semantic features; The semantic annotation module is used to perform semantic annotation based on the four-dimensional feature information of each continuous region.

Images