Intelligent public space interaction design method and system based on space perception

Abstract

The application relates to the technical field of intelligent interaction design, and discloses an intelligent public space interaction design method and system based on space perception, which realizes real-time data collection in a public space through a multi-source sensor, adopts an LIO-SAM algorithm for fusion to obtain multi-source fusion data, adopts a MaskR-CNN model for semantic segmentation on the multi-source fusion data, and constructs a three-dimensional dynamic space model; a behavior prediction model is constructed based on a Social-STGCNN-Transformer network, future behavior trends of personnel are predicted based on the multi-source fusion data, and user behavior analysis results are obtained; intelligent interaction strategies are generated according to the three-dimensional dynamic space model and the user behavior analysis results, the intelligent interaction strategies are executed through interaction equipment in the public space; feedback data are acquired, and the intelligent interaction strategies are evaluated and optimized based on the feedback data by adopting a multi-target Bayesian optimization algorithm; and the application realizes dynamic interaction optimization of the public space.

Description

Technical Field

[0001] This invention relates to the field of intelligent interactive design technology, specifically to an intelligent public space interactive design method and system based on spatial perception. Background Technology

[0002] With the acceleration of urbanization and the increasing demand for public space experiences, intelligent public space interaction design is receiving more and more attention. Traditional public space interaction design often lacks the ability to perceive and dynamically respond to the spatial environment and user behavior in real time. Most existing interaction designs are based on preset rules and fixed patterns, and cannot make adaptive adjustments to the interaction design according to the number, location, and behavior of people in the space, as well as changes in the spatial environment, such as light, temperature, and air quality. This limitation results in the interaction experience of public spaces being not personalized and intelligent enough, making it difficult to meet the increasingly diverse needs of users, and also failing to fully realize the functionality and efficiency of public spaces. Summary of the Invention

[0003] The purpose of this invention is to solve the above problems by designing an intelligent public space interaction design method and system based on spatial perception.

[0004] The first aspect of this invention provides a method for designing intelligent public space interaction based on spatial perception, the method comprising the following steps:

[0005] Data is collected in real time in public spaces using multiple source sensors and fused using the LIO-SAM algorithm to obtain multi-source fused data.

[0006] The Mask R-CNN model was used for semantic segmentation of multi-source fused data, and a three-dimensional dynamic spatial model was constructed.

[0007] A behavior prediction model is built based on the Social-STGCNN-Transformer network. Based on multi-source fusion data, the future behavior trends of people are predicted, and user behavior analysis results are obtained.

[0008] Based on the three-dimensional dynamic space model and user behavior analysis results, intelligent interaction strategies are generated and executed through interactive devices in public spaces.

[0009] Obtain feedback data and use a multi-objective Bayesian optimization algorithm to evaluate and optimize the intelligent interaction strategy based on the feedback data.

[0010] Optionally, in a first implementation of the first aspect of the present invention, the step of collecting data in real time in a public space using multiple source sensors and fusing it using the LIO-SAM algorithm to obtain multi-source fused data includes:

[0011] Data is collected in real time in public spaces using multi-source sensors, and the collected data is preprocessed with timestamp alignment.

[0012] The feature points extracted by the lidar are matched with the map to obtain the constraint information of the lidar pose, forming lidar factors.

[0013] Based on the pose information calculated from IMU data and the relationship between poses at adjacent time points, IMU factors are constructed.

[0014] In each iteration, based on the current pose estimation value, the errors of the lidar factor and IMU factor are recalculated, and the pose estimation values ​​of the nodes in the factor graph are continuously adjusted to minimize the lidar point cloud registration error and the IMU inertial measurement error, thereby obtaining multi-source fusion data.

[0015] Optionally, in a second implementation of the first aspect of the present invention, the LIO-SAM algorithm is based on a factor graph optimization framework, wherein the factor graph consists of nodes and edges, the nodes represent variables including the LiDAR pose and the IMU pose, and the edges represent the constraint relationships between the variables.

[0016] Optionally, in a third implementation of the first aspect of the present invention, the step of using the Mask R-CNN model to perform semantic segmentation on the multi-source fused data and constructing a three-dimensional dynamic spatial model includes:

[0017] Multi-source fused data is input into the MaskR-CNN model, and features are extracted from the multi-source fused data through a convolutional neural network. A region proposal network is used to generate candidate regions that may contain the target.

[0018] The ROIAlign layer is used to align the features of the candidate regions for classification, bounding box regression and mask prediction of each target, thus obtaining the semantic segmentation result.

[0019] Based on the semantic segmentation results and combined with the 3D coordinate information of LiDAR point cloud data, a 3D dynamic spatial model is constructed using 3D modeling technology.

[0020] Optionally, in the fourth implementation of the first aspect of the present invention, the step of constructing a behavior prediction model based on the Social-STGCNN-Transformer network, predicting future behavioral trends of individuals based on multi-source fusion data, and obtaining user behavior analysis results includes:

[0021] Multi-source fusion data is preprocessed to extract the motion characteristics of people and the relative positional relationships between people, and a spatiotemporal map is constructed.

[0022] The spatiotemporal graph is input into the behavior prediction model, and the spatiotemporal graph convolutional layer extracts features from the spatiotemporal graph to capture the local features of human behavior in the spatial and temporal dimensions.

[0023] The Transformer layer uses a multi-head attention mechanism to model long-distance dependencies between different people and learns the social interaction patterns between them.

[0024] The behavioral prediction model predicts the behavioral trends of people in the next 10-30 seconds and outputs user behavior analysis results, including movement direction, probability of staying and likelihood of interacting with others.

[0025] Optionally, in a fifth implementation of the first aspect of the present invention, the step of generating an intelligent interaction strategy based on a three-dimensional dynamic spatial model and user behavior analysis results, and executing the intelligent interaction strategy through an interactive device in a public space, includes:

[0026] Different interactive objects in the public space are set as multiple intelligent agents, and a state space is constructed for each intelligent agent based on a three-dimensional dynamic space model and user behavior analysis results.

[0027] A policy network is constructed based on a multilayer perceptron. The policy network takes the agent's state space as input and outputs the probability distribution of each possible action in the current state.

[0028] Based on the action probability distribution output by the policy network, the agent greedily selects an action to execute. When all agents have selected and executed the action, a joint action is formed, and a corresponding reward signal is generated in the common space.

[0029] Based on the obtained reward signals and state information, the policy network of the agent is optimized using proximal policy optimization. The action selection policy output by the policy network of each agent is integrated to obtain the intelligent interaction policy.

[0030] Optionally, in a sixth implementation of the first aspect of the present invention, the step of obtaining feedback data and evaluating and optimizing the intelligent interaction strategy based on the feedback data using a multi-objective Bayesian optimization algorithm includes:

[0031] By acquiring feedback data through interactive devices, a probabilistic model of the objective function is constructed. Through continuous sampling and evaluation of new intelligent interaction strategies, the optimal strategy combination is gradually found to balance multiple optimization objectives, including at least user satisfaction, space utilization efficiency, and interaction response speed.

[0032] A second aspect of the present invention provides a space-aware intelligent public space interaction design system, the system comprising:

[0033] The data acquisition module is used to collect data in real time in public spaces using multi-source sensors and to fuse the data using the LIO-SAM algorithm to obtain multi-source fused data.

[0034] The module is used to perform semantic segmentation on multi-source fused data using the Mask R-CNN model and to construct a three-dimensional dynamic spatial model.

[0035] The prediction module is used to build a behavior prediction model based on the Social-STGCNN-Transformer network, predict the future behavior trends of people based on multi-source fusion data, and obtain user behavior analysis results.

[0036] The execution module is used to generate intelligent interaction strategies based on the three-dimensional dynamic spatial model and user behavior analysis results, and to execute the intelligent interaction strategies through interactive devices in the public space.

[0037] The optimization module is used to acquire feedback data and evaluate and optimize the intelligent interaction strategy based on the feedback data using a multi-objective Bayesian optimization algorithm.

[0038] A third aspect of the present invention provides a space-aware intelligent public space interaction design device, the space-aware intelligent public space interaction design device comprising a memory and at least one processor, the memory storing instructions; the at least one processor invokes the instructions in the memory to cause the space-aware intelligent public space interaction design device to perform the various steps of the space-aware intelligent public space interaction design method as described in any of the preceding claims.

[0039] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of the spatially aware intelligent public space interaction design method as described in any of the preceding claims.

[0040] The technical solution provided by this invention involves real-time data collection via multi-source sensors in public spaces, followed by fusion using the LIO-SAM algorithm to obtain multi-source fused data. The multi-source fused data is then semantically segmented using the Mask R-CNN model, and a three-dimensional dynamic spatial model is constructed. A behavior prediction model is built based on the Social-STGCNN-Transformer network to predict future behavioral trends of individuals based on the multi-source fused data, yielding user behavior analysis results. Based on the three-dimensional dynamic spatial model and the user behavior analysis results, an intelligent interaction strategy is generated and executed through interactive devices within the public space. Feedback data is acquired, and based on this feedback data, multiple... The objective Bayesian optimization algorithm evaluates and optimizes intelligent interaction strategies. This invention can perceive environmental changes and user behavior in public spaces in real time, dynamically generate interaction strategies, and achieve adaptive adjustments to the interaction design of public spaces. Compared with traditional design methods, it has a higher level of intelligence and greatly enhances the personalized interaction experience of users in public spaces. It also optimizes space utilization efficiency: through the analysis and prediction of user behavior and the guidance of intelligent interaction strategies, it can reasonably adjust the distribution of people, optimize the use of facilities in public spaces, improve the overall utilization efficiency of public spaces, alleviate problems such as space congestion, realize dynamic interaction optimization of public spaces, and enhance the intelligence, personalization, and energy efficiency of public spaces. Attached Figure Description

[0041] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0042] Figure 1 A flowchart illustrating the spatially-aware intelligent public space interaction design method provided in this embodiment of the invention;

[0043] Figure 2 A schematic diagram of the structure of an intelligent public space interaction design system based on spatial perception provided in an embodiment of the present invention;

[0044] Figure 3 This is a structural schematic diagram of an intelligent public space interaction design device based on spatial perception, provided in an embodiment of the present invention. Detailed Implementation

[0045] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0046] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The flowchart of the intelligent public space interaction design method based on spatial awareness provided in this embodiment of the invention includes the following steps:

[0047] Step 101: Collect data in real time in public spaces using multi-source sensors and fuse the data using the LIO-SAM algorithm to obtain multi-source fused data;

[0048] In this embodiment, multiple sensors, including LiDAR, visual cameras, inertial measurement units (IMUs), microphone arrays, and environmental sensors, are deployed in the public space. Each sensor collects data in real time at different frequencies. The LiDAR collects spatial point cloud data at a frequency of 10Hz-20Hz, the visual cameras collect image data at a frame rate of 30fps, and the IMU acquires motion attitude data at a frequency of 100Hz.

[0049] In this embodiment, data is collected in real time in the public space using multi-source sensors. The collected data undergoes timestamp alignment preprocessing. Feature points extracted by the lidar are matched with the map to obtain lidar pose constraint information, forming lidar factors. IMU factors are constructed based on pose information calculated from IMU data and the relationship between poses at adjacent time points. In each iteration, the errors of lidar factors and IMU factors are recalculated based on the current pose estimation value. The pose estimation values ​​of nodes in the factor graph are continuously adjusted to minimize lidar point cloud registration errors and IMU inertial measurement errors, resulting in multi-source fused data.

[0050] In this embodiment, the point cloud data collected by the lidar undergoes preprocessing. The algorithm removes outliers caused by factors such as occlusion and noise. Simultaneously, based on the measurement principle of the lidar, the data is transformed to unify it into the global coordinate system of the public space. Then, the algorithm extracts features from the processed point cloud data, mainly planar feature points and edge feature points. Planar feature points come from relatively flat areas in the public space, such as the ground and walls; edge feature points come from the boundaries and corners of objects. These feature points can effectively represent the geometric structure information of the space.

[0051] For the data collected by the inertial measurement unit, the algorithm first filters it to eliminate noise caused by sensor errors and external interference. The IMU data includes the measurements of the accelerometer and gyroscope, which reflect the acceleration and angular velocity of the object, respectively. Through integration, the changes in acceleration and angular velocity over time are converted into the position, velocity and attitude information of the object. However, due to the drift problem of the IMU, the calculated position and other information will have a large error over time. Therefore, it is necessary to fuse and correct it with the LiDAR data in the future.

[0052] When fusing LiDAR and IMU data, the LIO-SAM algorithm operates based on a factor graph optimization framework. The factor graph consists of nodes and edges, where nodes represent variables such as the pose of the LiDAR and IMU, and edges represent the constraints between these variables. The algorithm matches the feature points extracted by the LiDAR with the constructed map to obtain the constraint information of the LiDAR pose, forming LiDAR factors. Simultaneously, based on the pose information calculated from the IMU data and the relationships between poses at adjacent time points, IMU factors are constructed.

[0053] The LIO-SAM algorithm continuously adjusts the pose estimation values ​​of nodes in the factor graph through iterative optimization. The goal is to minimize the registration error of the LiDAR point cloud and the inertial measurement error of the IMU. In each iteration, the errors of the LiDAR factor and IMU factor are recalculated based on the current pose estimation value. Then, the pose is adjusted using the optimization algorithm to continuously reduce the overall error. After multiple iterations, a high-precision pose estimation of the fused LiDAR and IMU is obtained, which in turn yields high-precision multi-source fusion data, enabling accurate perception of the public space environment and the position and movement information of people.

[0054] Step 102: Use the Mask R-CNN model to perform semantic segmentation on the multi-source fusion data and construct a three-dimensional dynamic spatial model;

[0055] In this embodiment, multi-source fused data is input into the MaskR-CNN model, and features are extracted from the multi-source fused data through a convolutional neural network. A region proposal network is used to generate candidate regions that may contain targets. The ROIAlign layer is used to align the features of the candidate regions to classify each target, perform bounding box regression and mask prediction, and obtain semantic segmentation results. Based on the semantic segmentation results, combined with the three-dimensional coordinate information of the LiDAR point cloud data, a three-dimensional dynamic spatial model is constructed using three-dimensional modeling technology.

[0056] In this embodiment, after the multi-source fused data is input into the MaskR-CNN model, the convolutional neural network (CNN) begins to play a crucial role. The CNN consists of multiple convolutional layers, pooling layers, and activation function layers. It extracts features from the multi-source fused data step by step according to a hierarchical structure. First, the data enters the first convolutional layer, and the convolutional kernel slides on the data, extracting basic features such as edges and textures through convolution operations. As the data is continuously passed through the network, subsequent convolutional layers will further extract more abstract and representative high-level semantic features based on the basic features extracted earlier. For example, in a public space scene, it gradually identifies the features of objects such as chairs, tables, and pedestrians, starting from initially recognizing lines and small areas. In this process, the pooling layer reduces the data dimensionality and computational load through downsampling operations, while retaining important features. The activation function gives the network non-linear capabilities, enabling the network to learn complex feature relationships, and finally obtains a feature map containing rich spatial semantic information.

[0057] Using the Region Proposal Network (RPN) based on the feature map extracted by the CNN, candidate regions that may contain targets are generated. The RPN slides a small network on the feature map, generating multiple anchor boxes of different scales and aspect ratios centered at each sliding position. These anchor boxes are like pre-set search boxes in space, covering various possible target shapes and sizes. Then, the RPN evaluates each anchor box to determine whether it contains a target and predicts the offset between the anchor box and the real target. By setting an appropriate score threshold, anchor boxes with higher scores are selected as candidate regions. For example, in public spaces, the RPN can quickly find areas where pedestrians or facilities may exist.

[0058] After obtaining candidate regions, the ROIAlign layer performs feature alignment on the candidate regions to achieve classification, bounding box regression, and mask prediction for each target, thereby obtaining semantic segmentation results. The main function of the ROIAlign layer is to accurately align and sample the feature maps corresponding to the candidate regions at a fixed size. It first determines the corresponding feature region based on the position of the candidate region in the original feature map, and then divides the region into several small units. Through methods such as bilinear interpolation, it accurately extracts features within each small unit, avoiding the feature bias problem caused by quantization operations in traditional ROI pooling. After feature alignment, the obtained features are input into the classification branch, bounding box regression branch, and mask prediction branch, respectively. The classification branch determines which type of object the target belongs to, such as pedestrians, seats, trash cans, etc.; the bounding box regression branch further adjusts the position and size of the candidate regions to make them more closely match the real targets; the mask prediction branch generates pixel-level masks for each target, accurately outlining the target's contour, and finally achieving accurate semantic segmentation of various targets in public spaces, clearly distinguishing different objects and people.

[0059] Based on the semantic segmentation results and combined with the 3D coordinate information of LiDAR point cloud data, a 3D dynamic spatial model is constructed using 3D modeling technology. The semantic segmentation results clarify the category and 2D contour of each target in the space, while the LiDAR point cloud data provides accurate 3D spatial location information. Combining the two, firstly, each semantically segmented target is assigned a corresponding 3D coordinate to determine its actual position and shape in space. Then, a suitable 3D modeling algorithm, such as triangulation algorithm, is used to transform the discrete point cloud data and semantic information into a continuous 3D geometric model. During the modeling process, considering the dynamic characteristics of the public space, the model is updated in real time. For example, when people move or the position of objects changes, newly collected multi-source data will be reprocessed through the above steps to promptly correct and update the 3D model, ensuring that it always accurately reflects the real-time state of the public space.

[0060] Step 103: Construct a behavior prediction model based on the Social-STGCNN-Transformer network, predict future behavior trends of people based on multi-source fusion data, and obtain user behavior analysis results;

[0061] In this embodiment, multi-source fusion data is preprocessed to extract the motion features of people and their relative positional relationships, constructing a spatiotemporal graph. The spatiotemporal graph is then input into a behavior prediction model, where a spatiotemporal graph convolutional layer extracts features, capturing local features of people's behavior in the spatial and temporal dimensions. The Transformer layer uses a multi-head attention mechanism to model long-distance dependencies between different people and learns social interaction patterns among them. The behavior prediction model then predicts the behavioral trends of people within the next 10-30 seconds and outputs user behavior analysis results, where behavioral trends include movement direction, probability of dwelling, and likelihood of interaction with others.

[0062] Step 104: Based on the three-dimensional dynamic space model and user behavior analysis results, generate intelligent interaction strategies and execute the intelligent interaction strategies through interactive devices in the public space;

[0063] In this embodiment, different interactive objects within the public space are defined as multiple agents. Based on a three-dimensional dynamic space model and user behavior analysis results, a state space is constructed for each agent. A policy network is constructed based on a multilayer perceptron. The policy network takes the agent's state space as input and outputs the probability distribution of each possible action in the current state. The agent selects an action to execute based on the action probability distribution output by the policy network using a greedy selection method. When all agents have selected and executed actions, a joint action is formed, and the public space generates a corresponding reward signal. Based on the obtained reward signal and state information, the policy network of the agents is optimized using proximal policy optimization. The action selection strategies output by the policy network of each agent are integrated to obtain an intelligent interaction strategy.

[0064] In this embodiment, different interactive objects or interactive targets in the public space are set as multiple intelligent agents. For example, displays, voice broadcasting systems, and smart lighting devices in different areas are regarded as independent intelligent agents. Each intelligent agent has autonomous decision-making capabilities. Based on the three-dimensional dynamic space model and user behavior analysis results, a state space is constructed for each intelligent agent. The state space includes the spatial location of the intelligent agent, surrounding environmental information, user behavior characteristics, and the state of the intelligent agent itself. Surrounding environmental information includes personnel density and the usage of nearby facilities. User behavior characteristics include the behavior trends and needs of nearby users. The state of the intelligent agent itself includes the current working mode of the device and remaining resources. This information is presented in a structured data form, enabling the intelligent agent to perceive the current environmental state.

[0065] Each agent possesses a policy network built upon a multilayer perceptron (MLP). The policy network takes the agent's state space as input and, through the computation of multiple neurons and the action of activation functions, outputs the probability distribution of each possible action in the current state. The action space encompasses all possible interactive actions that the agent may take, such as switching the displayed content on a screen, playing specific prompts in a voice broadcasting system, and adjusting the brightness and color of smart lighting devices. Based on the action probability distribution output by the policy network, the agent selects an action to execute using methods such as random sampling or greedy selection. For example, it may select the action with the highest probability with a certain probability, or randomly select an action according to the probability distribution, thereby enabling interaction with the public space environment and users.

[0066] Once all agents have selected and executed actions, these actions collectively constitute a joint action, which acts on the public space environment. The public space environment changes according to the joint action, generating corresponding feedback information. This feedback information includes the user's response to the interactive action and changes in the spatial state. Response information includes whether the user pays attention to the display screen content and whether they act according to the voice prompts. Changes in the spatial state include whether the distribution of people is more reasonable and whether the efficiency of facility use has improved. This feedback information is quantified into reward values, which are used to measure the execution effect of the joint action. Positive rewards indicate that the action has a positive effect on achieving the goal, while negative rewards indicate that the action has an adverse effect. Each agent receives its own reward signal and updated state information based on the environmental feedback.

[0067] Based on the obtained reward signals and state information, the MAPPO algorithm uses the idea of ​​proximal policy optimization to optimize the agent's policy network. The core idea is to maximize the cumulative reward while ensuring that the policy update is not too drastic. The algorithm calculates the difference between the current policy and the previous policy. By limiting the magnitude of policy updates, it avoids drastic fluctuations in the policy during the optimization process that could lead to performance degradation. During the optimization process, the advantage function is used to evaluate the merits of each action. The advantage function combines the reward of the current action and the expected reward in the future, which can more accurately reflect the value of the action. Based on the advantage function and the action probability output by the policy network, the loss function is calculated. The parameters of the policy network are updated through the backpropagation algorithm, so that the policy network gradually learns the policy that can obtain higher rewards. After multiple rounds of training and optimization, when the policy network converges to a certain extent, the action selection policies output by the policy network of each agent are integrated to form an intelligent interaction policy for the current public space state and user behavior, which is used to guide the actual operation of interactive devices in the public space.

[0068] Step 105: Obtain feedback data and evaluate and optimize the intelligent interaction strategy based on the feedback data using a multi-objective Bayesian optimization algorithm.

[0069] In this embodiment, feedback data is obtained through feedback buttons on the interactive device, facial expression recognition by the camera, and monitoring of changes in user behavior. The feedback data includes direct feedback from users on the interaction strategy, such as whether they click to confirm or choose to ignore, as well as indirect feedback, such as whether they change their route after accepting guidance or whether the efficiency of facility use has improved.

[0070] In this embodiment, feedback data is obtained through interactive devices to construct a probabilistic model of the objective function. By continuously sampling and evaluating new intelligent interaction strategies, the optimal strategy combination is gradually found to balance multiple optimization objectives, including at least user satisfaction, space utilization efficiency, and interaction response speed.

[0071] Please see Figure 2 A schematic diagram of the structure of the intelligent public space interaction design system based on spatial perception provided in this embodiment of the invention. The system includes:

[0072] The data acquisition module is used to collect data in real time in public spaces using multi-source sensors and to fuse the data using the LIO-SAM algorithm to obtain multi-source fused data.

[0073] The module is used to perform semantic segmentation on multi-source fused data using the Mask R-CNN model and to construct a three-dimensional dynamic spatial model.

[0074] The prediction module is used to build a behavior prediction model based on the Social-STGCNN-Transformer network, predict the future behavior trends of people based on multi-source fusion data, and obtain user behavior analysis results.

[0075] The execution module is used to generate intelligent interaction strategies based on the three-dimensional dynamic spatial model and user behavior analysis results, and to execute the intelligent interaction strategies through interactive devices in the public space.

[0076] The optimization module is used to acquire feedback data and evaluate and optimize the intelligent interaction strategy based on the feedback data using a multi-objective Bayesian optimization algorithm.

[0077] Figure 3This is a schematic diagram of the structure of a space-aware intelligent public space interaction design device 300 provided in an embodiment of the present invention. The space-aware intelligent public space interaction design device 300 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the space-aware intelligent public space interaction design device 300. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the space-aware intelligent public space interaction design device 300 to implement the method provided in the above embodiment.

[0078] The space-aware intelligent public space interaction design device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating devices 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3 The spatially-aware intelligent public space interaction design device structure shown does not constitute a limitation on the computer device provided by the present invention. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0079] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the various steps of the space-aware intelligent public space interaction design method provided in the above embodiments.

[0080] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described equipment or apparatus / unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0081] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A spatially-aware intelligent public space interaction design method, characterized in that, The method includes the following steps: Data is collected in real time in public spaces using multiple source sensors and fused using the LIO-SAM algorithm to obtain multi-source fused data. The Mask R-CNN model was used for semantic segmentation of multi-source fused data, and a three-dimensional dynamic spatial model was constructed. A behavior prediction model is built based on the Social-STGCNN-Transformer network. Based on multi-source fusion data, the future behavior trends of people are predicted, and user behavior analysis results are obtained. Based on the three-dimensional dynamic space model and user behavior analysis results, intelligent interaction strategies are generated and executed through interactive devices in public spaces. Obtain feedback data, and use a multi-objective Bayesian optimization algorithm to evaluate and optimize the intelligent interaction strategy based on the feedback data; The process involves real-time data collection via multi-source sensors within the public space, followed by fusion using the LIO-SAM algorithm to obtain multi-source fused data, including: Data is collected in real time in public spaces using multi-source sensors, and the collected data is preprocessed with timestamp alignment. The feature points extracted by the lidar are matched with the map to obtain the constraint information of the lidar pose, forming lidar factors. Based on the pose information calculated from IMU data and the relationship between poses at adjacent time points, IMU factors are constructed. In each iteration, based on the current pose estimation value, the errors of the lidar factor and IMU factor are recalculated, and the pose estimation values ​​of the nodes in the factor graph are continuously adjusted to minimize the lidar point cloud registration error and the IMU inertial measurement error, thereby obtaining multi-source fusion data.

2. The intelligent public space interaction design method based on spatial perception as described in claim 1, characterized in that, The LIO-SAM algorithm is based on a factor graph optimization framework, where the factor graph consists of nodes and edges. Nodes represent variables containing LiDAR pose and IMU pose, and edges represent the constraint relationships between variables.

3. The intelligent public space interaction design method based on spatial perception as described in claim 1, characterized in that, The process of semantic segmentation of multi-source fused data using the Mask R-CNN model and constructing a three-dimensional dynamic spatial model includes: Multi-source fused data is input into the MaskR-CNN model, and features are extracted from the multi-source fused data through a convolutional neural network. A region proposal network is used to generate candidate regions that may contain the target. The ROIAlign layer is used to align the features of the candidate regions for classification, bounding box regression and mask prediction of each target, thus obtaining the semantic segmentation result. Based on the semantic segmentation results and combined with the 3D coordinate information of LiDAR point cloud data, a 3D dynamic spatial model is constructed using 3D modeling technology.

4. The intelligent public space interaction design method based on spatial perception as described in claim 1, characterized in that, The behavior prediction model built on the Social-STGCNN-Transformer network predicts future behavioral trends based on multi-source fusion data, yielding user behavior analysis results, including: Multi-source fusion data is preprocessed to extract the motion characteristics of people and the relative positional relationships between people, and a spatiotemporal map is constructed. The spatiotemporal graph is input into the behavior prediction model, and the spatiotemporal graph convolutional layer extracts features from the spatiotemporal graph to capture the local features of human behavior in the spatial and temporal dimensions. The Transformer layer uses a multi-head attention mechanism to model long-distance dependencies between different people and learns the social interaction patterns between them. The behavioral prediction model predicts the behavioral trends of people in the next 10-30 seconds and outputs user behavior analysis results, including movement direction, probability of staying and likelihood of interacting with others.

5. The intelligent public space interaction design method based on spatial perception as described in claim 1, characterized in that, The process of generating intelligent interaction strategies based on a three-dimensional dynamic spatial model and user behavior analysis results, and executing these strategies through interactive devices within the public space, includes: Different interactive objects in the public space are set as multiple intelligent agents, and a state space is constructed for each intelligent agent based on a three-dimensional dynamic space model and user behavior analysis results. A policy network is constructed based on a multilayer perceptron. The policy network takes the agent's state space as input and outputs the probability distribution of each possible action in the current state. Based on the action probability distribution output by the policy network, the agent greedily selects an action to execute. When all agents have selected and executed the action, a joint action is formed, and a corresponding reward signal is generated in the common space. Based on the obtained reward signals and state information, the policy network of the agent is optimized using proximal policy optimization. The action selection policy output by the policy network of each agent is integrated to obtain the intelligent interaction policy.

6. The intelligent public space interaction design method based on spatial perception as described in claim 1, characterized in that, The process of acquiring feedback data and evaluating and optimizing the intelligent interaction strategy based on the feedback data using a multi-objective Bayesian optimization algorithm includes: By acquiring feedback data through interactive devices, a probabilistic model of the objective function is constructed. Through continuous sampling and evaluation of new intelligent interaction strategies, the optimal strategy combination is gradually found to balance multiple optimization objectives, including at least user satisfaction, space utilization efficiency, and interaction response speed.

7. A spatially-aware intelligent public space interaction design system, characterized in that, The system includes: The data acquisition module is used to collect data in real time in public spaces using multi-source sensors and fuse it using the LIO-SAM algorithm to obtain multi-source fused data. The process involves: collecting data in real time using multi-source sensors in public spaces; preprocessing the collected data by aligning timestamps; matching feature points extracted by the LiDAR with the map to obtain LiDAR pose constraint information, forming LiDAR factors; constructing IMU factors based on pose information calculated from IMU data and the relationship between poses at adjacent time points; and in each iteration, recalculating the errors of the LiDAR factors and IMU factors based on the current pose estimation value, continuously adjusting the pose estimation values ​​of nodes in the factor graph, minimizing the LiDAR point cloud registration error and the IMU inertial measurement error, and obtaining multi-source fused data. The module is used to perform semantic segmentation on multi-source fused data using the Mask R-CNN model and to construct a three-dimensional dynamic spatial model. The prediction module is used to build a behavior prediction model based on the Social-STGCNN-Transformer network, predict the future behavior trends of people based on multi-source fusion data, and obtain user behavior analysis results. The execution module is used to generate intelligent interaction strategies based on the three-dimensional dynamic spatial model and user behavior analysis results, and to execute the intelligent interaction strategies through interactive devices in the public space. The optimization module is used to acquire feedback data and evaluate and optimize the intelligent interaction strategy based on the feedback data using a multi-objective Bayesian optimization algorithm.

8. A spatially-aware intelligent public space interaction design device, characterized in that, The space-aware intelligent public space interaction design device includes a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the space-aware intelligent public space interaction design device to perform each step of the space-aware intelligent public space interaction design method as described in any one of claims 1-6.

9. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the steps of the spatially aware intelligent public space interaction design method as described in any one of claims 1-6.

Images