Public space social behavior and environment influence identification method based on perception equipment network

Abstract

The invention discloses a public space social behavior and environment influence identification method based on a sensing equipment network. The method comprises the following steps of public space area selection, measuring point layout and sensing equipment distribution, public space physical environment monitoring, public space social behavior identification, behavior environment index influence analysis and influence rating and output. Based on the network layout of the sensing equipment, the physical environment of the public space is monitored through the environment sensing equipment, the social behaviors of the public space are identified through the behavior sensing equipment, the influence relation between the environment indexes and the behavior indexes is comprehensively analyzed, and the influence degrees are sequenced and output. The method can be used for identifying the behavior environment influence relationship in the urban intelligent sensing field, and is beneficial to building a more efficient and comfortable urban public space.

Description

Technical Field

[0001] The present invention belongs to the field of urban intelligent perception technology, and in particular relates to a method for identifying social behaviors and environmental impacts in public spaces based on a perception device network. Background Art

[0002] In the context of high-density urban development, open urban public spaces are home to most outdoor social activities, and their physical environment is directly related to the quality of life and social interaction of residents. The creation of a reasonable public space physical environment helps promote communication and interaction among residents, promote connections and mutual assistance within the community, and enhance community cohesion and a sense of belonging.

[0003] At present, the main monitoring methods for the physical environment of public spaces are remote sensing and ground observation. Among them, the remote sensing method is affected by atmospheric interference and surface coverage, and the data analysis error is large; the ground observation method is more suitable for micro-urban spaces such as streets and squares, but the current technology focuses more on the single dimension of thermal environment and wind environment, and rarely conducts comprehensive measurement of the physical environment. Behavior recognition in public spaces is mainly through image recognition or inertial measurement methods. Among them, the image recognition method is less dependent on wearable devices and is more suitable for long-term monitoring of social behavior in public spaces. However, the current technology focuses more on application scenarios such as driving safety and indoor security, and lacks technical methods for urban social behavior recognition.

[0004] Therefore, it is urgent to solve the above problems. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a method for identifying social behaviors and environmental impacts in public spaces based on a perception device network. The present invention is based on a perception device network that integrates environmental perception devices and behavior perception devices to achieve monitoring, identification, and analysis of the impact relationship between social behaviors in public spaces and the physical environment.

[0006] Technical solution: To achieve the above objectives, the present invention discloses a method for identifying social behaviors and environmental impacts in public spaces based on a sensing device network, comprising the following steps:

[0007] (1) Selecting a public space area through an interactive map on a web page interface, and recording the geographic location information of the public space area based on the user's selection input;

[0008] (2) Determine the number of sensing devices and the layout of measurement points based on the geographic location information of the public space area in step (1), deploy a sensing device network in the public space area, the sensing device network includes environmental sensing devices and behavior sensing devices, test the working status and coverage of the devices, and transmit continuous monitoring data to the data center;

[0009] (3) The physical environment data of the public space area collected by the environmental sensing device is aggregated into the corresponding spatial grid unit according to the time node, and a grid data matrix with time series information is generated. The continuous environmental data distribution of the public space area is generated by the spatial interpolation method, and the average value of the environmental data within the unit time interval is calculated and stored in the database in the form of structured data;

[0010] (4) Use convolutional neural networks to process video data from behavior sensing devices in real time, detect and track individuals in the crowd, and obtain individual position coordinates and motion trajectories; use multi-camera data fusion technology to eliminate the impact of occlusions and video angles, and merge redundant data; combine social distance, facial orientation, and body movement recognition to determine the individual's social status and social behavior type, add spatial raster data to social behavior indicators, and store them in the database in the form of structured data;

[0011] (5) Based on the structured data of the physical environment in step (3) and the structured data of social behavior in step (4), the effects of fixed spatial locations and time nodes on social behavior are eliminated through spatial regression models and time series decomposition, thereby obtaining a social behavior data set that only retains the effects of environmental factors; correlation analysis and multivariate regression analysis are combined to determine whether various environmental indicators have an impact on social behavior, and to determine the mode and extent of the impact;

[0012] (6) Based on the regression analysis results in step (5), the degree of influence of various environmental indicators of public spaces on social behavior is ranked and output in three-dimensional display.

[0013] Optionally, step (1) specifically includes the following steps: loading and displaying an interactive map on a web page interface through an HTML container element, receiving user selection input on the map, where the selection input refers to selecting a public space area in the map; based on the user's selection input, recording geographic location information of the public space area, where the geographic location information includes all vertex coordinate geometric information, the geometric center point geographic coordinates and the geographic area of ​​the public space area.

[0014] Optionally, the environmental sensing device in step (2) integrates temperature and humidity sensors, light sensors and wind environment sensors, which are evenly distributed in the public space area in a 20m×20m grid; the behavior sensing device integrates a portable power supply and a network camera, and the effective sensing range of the behavior sensing device is a plane area with a radius of 50m and an angle of 80° centered on the device. The boundary and geometric center of the public space area are determined, and the public space area is divided into 5m×5m grid units. The number of devices and the location of the points that achieve full grid unit coverage and minimum overlap are calculated through a genetic algorithm.

[0015] Optionally, the structured data in step (3) includes a grid unit identifier, a timestamp, an environmental indicator type and a data average value of the data. The environmental indicator types include light environment indicators, wind environment indicators and thermal environment indicators. The light environment indicators include sunshine hours per minute and per hour. The sunshine hours are calculated cumulatively based on the time when the direct radiation measured by the light sensor is less than or equal to 120 W / ㎡. The wind environment indicators include the average wind speed and average wind direction per minute. The average wind speed and average wind direction per minute are calculated based on the wind speed and wind direction measured by the wind environment sensor. The thermal environment indicators include the average temperature and relative humidity per minute. The average temperature and relative humidity per minute are calculated based on the temperature and humidity measured by the temperature and humidity sensor.

[0016] Optionally, the multi-camera data fusion technology in step (4) refers to combining the position coordinates (x c ,y c ,z c ) and shooting direction (θ x ,θ y ,θ z ), the coordinates of the center point of the individual image in the photographic image (c x ,c y ) and the focal lengths f on the x-axis and y-axis on the photographic image plane determined by the vertical angle of the camera and the sensor size of the camera x and f y , construct the camera's internal parameter matrix and the external parameter matrix [R|T], where the rotation matrix R = R z (θ z )·R y (θ y )·R x (θ x ), where R x , R y , R z The rotation matrix and translation matrix are respectively the rotation angles around the x, y, and z axes. where t x ,t y and t z The x, y, and z axes are the translations of the center coordinates of the individual image in the photographic image, respectively. The projection matrix P = K · [R|T] of the camera is then defined. The individual image coordinates are converted into spatial plane coordinates through the projection matrix. The trajectory of each individual in the camera is recorded based on the converted spatial plane coordinates to obtain a series of grid coordinate points. The trajectory data of each camera is matched by timestamp and spatial position, and the distance between two trajectories in the same time period is calculated. Where) i,t ,Y i,t ) represents the coordinate position of individual i at time frame t, (X j,t,Y j,t ) represents the coordinate position of individual j at time frame t; the distance threshold is set to 2m, and when d<2m, the individual similarity is identified through cross-device pedestrian re-identification (Re-Identification); the ResNet deep learning model is used to extract the feature vector of the individual image taken by each camera, and the cosine similarity is used to calculate the feature vector similarity between different individuals in the same time period under different cameras:

[0017]

[0018] Where: x·y is the dot product of the feature vectors, ||x|| and ||y|| are the Euclidean norms of vectors x and y respectively; the cosine similarity threshold is set to 0.5, and the feature vectors with cosine similarity>0.5 are regarded as the same object and assigned the same identity ID; the trajectories with the same identity ID in different cameras are associated, and the matched trajectories in each camera are merged to remove duplicate trajectories and form unified trajectory data; the matched trajectories in different cameras are merged into a continuous trajectory sequence, and the plane coordinates, video frame images, and identity ID information in each time frame are retained. The merged plane grid coordinates (X t ,Y t ) and the trajectory data of the corresponding video images are stored in the database in the form of structured data.

[0019] Optionally, in step (4), the social status and social behavior type of an individual are determined by combining social distance, facial orientation and body movement recognition. First, the database is sorted, the motion trajectory of the individual is segmented according to the timestamp, and the video frame is extracted at a time interval of 0.5 s; by analyzing the motion trajectory, according to the distance between any two individuals at different time points in the plane coordinate data, for each timestamp t, the Euclidean distance d between individual i and individual j is calculated. ij (t):

[0020]

[0021] Where: x i (t), y i (t) is the plane coordinate of individual i at time t, based on the preset social distance threshold D threshold , if d ij (t)≤D threshold If the duration Δt exceeds the threshold, it is judged that there is social interaction between two or more individuals; the face area is located by the MTCNN face detection algorithm, and the Euler angles (α, β, γ) of the head are calculated through head posture estimation to match the facing directions between individuals. If the yaw angle β of two or more individuals isi and β j The difference between them is within a certain threshold Δβ threshold Inner, that is, |β i -β j |≤Δβ threshold , and the relative distance meets the social distance threshold, it is judged that they are engaging in social interaction; the OpenPose human posture estimation algorithm is used to extract the key node coordinates P in each video frame = {p1, p2...p n}, by analyzing the time series of joint points, the motion trajectory and speed of key limb parts are calculated, and the movement distance of joint point k between time t and t+Δt is:

[0022]

[0023] The extracted feature vector v = {v1,v2...v n} is input into the SVM classification model and the social distance d is integrated ij (t), facial orientation β, and limb movement features v are comprehensively analyzed:

[0024] F = concat(d ij (t),β i ,β j ,v)

[0025] The probability distribution P(class|F) is output by the random forest model, and the three behavior types of fitness, entertainment and general social are calculated based on the feature vector F that is concatenated from multiple features. The final output is a structured data set including individual ID, movement trajectory, social status, social behavior type, time and spatial location of the behavior, and corresponding movement status detailed information.

[0026] Optionally, the structured data in step (4) includes a grid unit identifier, a timestamp, a behavior indicator type and an indicator data value of the data, the behavior indicator type includes a social status and a social behavior type, the social status includes two data values, yes and no, and the social behavior type includes three data values, namely, fitness, entertainment and general social.

[0027] Optionally, in step (5), the effects of fixed spatial locations and time nodes on social behavior are eliminated by using a spatial regression model and time series decomposition. First, the spatial autocorrelation of the structured social behavior data is calculated to construct a spatial weight matrix W for capturing the relationship between spatial locations, where the elements of the matrix W are ij Represent the spatial interaction between location i and location j, and construct a spatial autoregressive model SAR:

[0028] Y ij =ρWY ij+X ij β+∈ ij

[0029] Where: Y ij is the social behavior of individual i at location j, ρ is the spatial autocorrelation coefficient, X ij is the environmental indicator, β is the coefficient vector of the independent variable, ∈ ij is the error term;

[0030] Use maximum likelihood estimation MLE to estimate the model parameters ρ and β. According to the estimated ρ, calculate the spatial autocorrelation part of the social behavior data:

[0031] Y spatial =ρWY

[0032] Remove the spatial autocorrelation part from the original data to obtain the non-spatial autocorrelation part, that is, the residual Y residual :

[0033] Y residual =YY spatial -Xβ

[0034] Residual Y residual That is, after removing spatial autocorrelation, social behavior data at fixed spatial locations are excluded;

[0035] Secondly, perform time series decomposition calculation on structured social behavior data:

[0036] Y t =T t +S t +R t

[0037] Among them, Y t is social behavior data, T t is the trend component, S t is the seasonal component, R t is the residual component; removing the trend and seasonality from the original data, we get the residual R t =Y t -T t -S t , which is the social behavior data without the influence of time nodes; the residual data Y obtained by the spatial autocorrelation model residual The residual data R obtained by time series decomposition t Combined, we get the social behavior dataset Y after comprehensive spatial and temporal adjustment adj :

[0038] Y adj =Y residual +R t

[0039] Dataset Y adj The influence of fixed spatial locations and time nodes on social behavior is eliminated, and the characteristics of behavioral changes related to the environment are retained.

[0040] Optionally, in step (5), combining correlation analysis and multivariate regression analysis to determine whether various environmental indicators have an impact on social behavior and determine the impact mode and degree refers to standardizing the environmental indicator data in the structured data of the physical environment in step (3) and converting the social behavior data set Y adj The social status and three behavior type data in the data were coded as binary variables, in which "0" represents "no" and "1" represents "yes". The correlation coefficients between the environmental indicator data and the social status data and the three behavior type data were calculated by point-by-point serial correlation analysis, and the environmental indicators significantly correlated with the social status and behavior type were screened out. The significant correlation refers to the absolute value of the correlation coefficient being greater than 0.3. The data of the screened environmental indicators were extracted and regressed with the binary coded social status and three behavior type data by logistic regression method to obtain the regression coefficient β and OR value. The way and degree of the impact of the environmental indicator on the social behavior indicator were judged according to the threshold range to which it belonged. The threshold ranges were as follows: weak positive impact, 0≤β<0.1 and 1.0≤OR<1.1; moderate positive impact, 0.1≤β<0.5 and 1.1≤OR<2.0; strong positive impact, 0.5≤β and 2.0≤OR; weak negative impact, -0.1≤β<0 and 0.9≤OR<1.0; moderate negative impact, -0.5≤β<-0.1 and 0.5≤OR<0.9; strong negative impact, β<-0.5 and OR<0.5.

[0041] Optionally, in step (6), a holographic digital projector and a visualization system are used to display and output in three dimensions the time-space thermal plane of each environmental indicator, the social behavior thermal plane, and the environmental indicator impact ranking table.

[0042] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: the present invention combines the specific characteristics of the target public space, integrates the layout of environmental perception devices and behavior perception devices through the perception device network, realizes the precise coupling of physical environment data and social behavior data, and improves the accuracy of behavior environment impact analysis and data collection efficiency; the present invention integrates temperature and humidity sensors, light sensors and wind environment sensors into environmental perception devices to realize the comprehensive measurement of the physical environment, more comprehensively evaluate the impact of the physical environment on social behavior, and obtain the impact rating of each physical environment indicator; the present invention analyzes whether an individual is in a social state and the type of social behavior by identifying social distance, facial orientation and body movements, thereby realizing the refinement and accuracy improvement of social behavior recognition; the present invention constructs a data correction method based on the spatiotemporal characteristics of the data, eliminates the influence of fixed spatial positions and time node interference items on social behavior, and makes the influence relationship between the physical environment and social behavior clearer and more explicit; the present invention calculates the number and location of perception devices, optimizes the coverage of devices, and improves the comprehensiveness and accuracy of data collection; the present invention uses HTML container elements to create interactive maps, so that users can easily select public space areas, improve user experience and participation. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0044] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0045] like Figure 1 As shown, a method for identifying social behaviors and environmental impacts in public spaces based on a sensing device network includes the following steps:

[0046] S1. Public space area selection: select a public space area through an interactive map frame on the web page interface, and record the geographic location information of the public space area based on the user's selection input;

[0047] An interactive map is loaded and displayed on the web page interface through an HTML container element, and the user's selection input on the map is received. The selection input refers to selecting a public space area in the map; based on the user's selection input, the geographic location information of the public space area is recorded. The geographic location information includes the geometric coordinate information of all vertices of the public space area, the geographic coordinates of the geometric center point, and the geographic area;

[0048] S2. Measurement point layout and sensing device allocation: Determine the number of sensing devices and measurement point layout based on the geographic location information of the public space area in S1, deploy a sensing device network in the public space area, the sensing device network includes environmental sensing devices and behavior sensing devices, determine the measurement point layout of environmental sensing devices and behavior sensing devices, install environmental sensing devices and behavior sensing devices, test the working status and coverage of the equipment, and transmit 24h continuous monitoring data to the data center;

[0049] The environmental sensing device integrates temperature and humidity sensors, light sensors, and wind environment sensors, and is evenly distributed in the public space area in a 20m×20m grid. The behavior sensing device integrates a mobile power supply and a network camera. The effective sensing range of the behavior sensing device is a plane area with a radius of 50m and an angle of 80° centered on the device. The boundary and geometric center of the public space area are determined, and the public space area is divided into 5m×5m grid units. The number of devices and the location of the points that achieve full grid unit coverage and minimum overlap are calculated through a genetic algorithm.

[0050] S3. Public space physical environment monitoring: Preprocess the environmental data collected by environmental sensing devices, that is, clean up outliers, aggregate the environmental data to the corresponding spatial grid units according to time nodes, generate a grid data matrix with time series information, and generate continuous environmental data distribution in the public space area through spatial interpolation methods, calculate the average value of environmental data within the unit time interval, and store it in the database in the form of structured data;

[0051] The structured data in step S3 includes a grid unit identifier, a timestamp, an environmental indicator type, and a data average value of the data. The environmental indicator type includes a light environment indicator, a wind environment indicator, and a thermal environment indicator. The light environment indicator includes sunshine hours per minute and per hour. The sunshine hours are calculated cumulatively according to the time when the direct radiation measured by the light sensor is less than or equal to 120 W / ㎡; the wind environment indicator includes an average wind speed and an average wind direction per minute. The average wind speed and the average wind direction per minute are calculated according to the wind speed and the wind direction measured by the wind environment sensor; the thermal environment indicator includes an average temperature and relative humidity per minute. The average temperature and relative humidity per minute are calculated according to the temperature and humidity measured by the temperature and humidity sensor;

[0052] S4. Social behavior recognition in public spaces: Use convolutional neural networks to process video data from behavior sensing devices in real time, detect and track individuals in the crowd, and obtain individual position coordinates and movement trajectories; use multi-camera data fusion technology to eliminate the impact of occlusions and video angles, and merge redundant data; combine social distance, facial orientation, and body movement recognition to determine the individual's social status and social behavior type, add spatial raster data to social behavior indicators, and store them in the database in the form of structured data;

[0053] The multi-camera data fusion technology in step S4 refers to combining the position coordinates (x c ,y c ,z c ) and shooting direction (θ x ,θ y ,θ z ), the coordinates of the center point of the individual image in the photographic image (c x ,c y ) and the focal lengths f on the x-axis and y-axis on the photographic image plane determined by the vertical angle of the camera and the sensor size of the camera x and f y , construct the camera's internal parameter matrix and the external parameter matrix [R|T], where the rotation matrix R = R z (θ z )·R y (θ y )·R x (θ x ), where R x , R y , R z The rotation matrix and translation matrix are respectively the rotation angles around the x, y, and z axes. where t x ,t y and t z The x, y, and z axes are the translations of the center coordinates of the individual image in the photographic image, respectively. The projection matrix P = K · [R|T] of the camera is then defined. The individual image coordinates are converted into spatial plane coordinates through the projection matrix. The trajectory of each individual in the camera is recorded based on the converted spatial plane coordinates to obtain a series of grid coordinate points. The trajectory data of each camera is matched by timestamp and spatial position, and the distance between two trajectories in the same time period is calculated. Where (X i,t ,Y i,t ) represents the coordinate position of individual i at time frame t, (X j,t ,Y j,t ) represents the coordinate position of individual j at time frame t; the distance threshold is set to 2m, and when d<2m, the individual similarity is identified through cross-device pedestrian re-identification (Re-Identification); the ResNet deep learning model is used to extract the feature vector of the individual image taken by each camera, and the cosine similarity is used to calculate the feature vector similarity between different individuals in the same time period under different cameras:

[0054]

[0055] Where: x·y is the dot product of the feature vectors, ||x|| and ||y|| are the Euclidean norms of vectors x and y respectively; the cosine similarity threshold is set to 0.5, and the feature vectors with cosine similarity>0.5 are regarded as the same object and assigned the same identity ID; the trajectories with the same identity ID in different cameras are associated, and the matched trajectories in each camera are merged to remove duplicate trajectories and form unified trajectory data; the matched trajectories in different cameras are merged into a continuous trajectory sequence, and the plane coordinates, video frame images, and identity ID information in each time frame are retained. The merged plane grid coordinates (X t ,Y t ) and the trajectory data of the corresponding video image are stored in the database in the form of structured data;

[0056] In step S4, the social status and social behavior type of an individual are determined by combining social distance, facial orientation, and body movement recognition. First, the database is sorted, the motion trajectory of the individual is segmented according to the timestamp, and the video frame is extracted at a time interval of 0.5s. By analyzing the motion trajectory, the distance between any two individuals at different time points in the plane coordinate data is calculated for each timestamp t. ij (t):

[0057]

[0058] Where: x i (t), y i (t) is the plane coordinate of individual i at time t, based on the preset social distance threshold D threshold , if d ij (t)≤D threshold If the duration Δt exceeds the threshold, it is judged that there is social interaction between two or more individuals; the face area is located by the MTCNN face detection algorithm, and the Euler angles (α, β, γ) of the head are calculated through head posture estimation to match the facing directions between individuals. If the yaw angle β of two or more individuals is i and β j The difference between them is within a certain threshold Δβ threshold Inner, that is, |β i -β j |≤Δβ threshold , and the relative distance meets the social distance threshold, it is judged that they are engaging in social interaction; the OpenPose human posture estimation algorithm is used to extract the key node coordinates P in each video frame = {p1, p2...p n}, by analyzing the time series of joint points, the motion trajectory and speed of key limb parts are calculated, and the movement distance of joint point k between time t and t+Δt is:

[0059]

[0060] The extracted feature vector v = {v1,v2...v n} is input into the SVM classification model and the social distance d is integrated ij (t), facial orientation β, and limb movement features v are comprehensively analyzed:

[0061] F = concat(d ij (t),β i ,β j ,v)

[0062] The random forest model outputs the probability distribution P(class|F), and based on the feature vector F concatenated from multiple features, the corresponding identification is divided into three types of behavior: fitness, entertainment, and general social. The final output includes a structured data set including individual ID, movement trajectory, social status, social behavior type, time and spatial location of the behavior, and corresponding movement status detailed information;

[0063] The structured data in step S4 includes a grid unit identifier, a timestamp, a behavior indicator type, and an indicator data value of the data, the behavior indicator type includes a social status and a social behavior type, the social status includes two data values ​​of yes and no, and the social behavior type includes three data values ​​of fitness, entertainment, and general social;

[0064] S5. Analysis of the impact of behavioral environmental indicators: Based on the structured data of the physical environment in S3 and the structured data of social behavior in S4, the spatial regression model and time series decomposition are used to eliminate the influence of fixed spatial locations and time nodes on social behavior, and obtain a social behavior data set that only retains the influence of environmental factors; Combine correlation analysis with multiple regression analysis to determine whether various environmental indicators have an impact on social behavior, and determine the mode and degree of impact;

[0065] In S5, the spatial regression model and time series decomposition are used to eliminate the influence of fixed spatial locations and time nodes on social behavior. First, the spatial autocorrelation of structured social behavior data is calculated, and a spatial weight matrix W is constructed to capture the relationship between spatial locations, where the elements of the matrix W ij Represent the spatial interaction between location i and location j, and construct a spatial autoregressive model SAR:

[0066] Y ij =ρWY ij +X ij β+∈ ij

[0067] Where: Y ijis the social behavior of individual i at location j, ρ is the spatial autocorrelation coefficient, X ij is the environmental indicator, β is the coefficient vector of the independent variable, ∈ ij is the error term;

[0068] Use maximum likelihood estimation MLE to estimate the model parameters ρ and β. According to the estimated ρ, calculate the spatial autocorrelation part of the social behavior data:

[0069] Y spatial =ρWY

[0070] Remove the spatial autocorrelation part from the original data to obtain the non-spatial autocorrelation part, that is, the residual Y residual :

[0071] Y residual =YY spatial -Xβ

[0072] Residual Y residual That is, after removing spatial autocorrelation, social behavior data at fixed spatial locations are excluded;

[0073] Secondly, perform time series decomposition calculation on structured social behavior data:

[0074] Y t =T t +S t +R t

[0075] Among them, Y t is social behavior data, T t is the trend component, S t is the seasonal component, R t is the residual component; removing the trend and seasonality from the original data, we get the residual R t =Y t -T t -S t , which is the social behavior data without the influence of time nodes; the residual data Y obtained by the spatial autocorrelation model residual The residual data R obtained by time series decomposition t Combined, we get the social behavior dataset Y after comprehensive spatial and temporal adjustment adj :

[0076] Y adj =Y residual +R t

[0077] Dataset Y adj The effects of fixed spatial locations and time nodes on social behavior are eliminated, and the characteristics of behavioral changes related to the environment are retained;

[0078] S5 combines correlation analysis and multiple regression analysis to determine whether various environmental indicators have an impact on social behavior, and to determine the mode and extent of the impact. This means that the environmental indicator data in the structured data of S3 is standardized and the social behavior dataset Y adj The social status and three behavior type data in the data were coded as binary variables, in which "0" represents "no" and "1" represents "yes". The correlation coefficients between the environmental indicator data and the social status data and the three behavior type data were calculated by point-by-point serial correlation analysis, and the environmental indicators significantly correlated with the social status and behavior type were screened out. The significant correlation refers to the absolute value of the correlation coefficient being greater than 0.3. The data of the screened environmental indicators were extracted and regressed with the binary coded social status and three behavior type data by logistic regression method to obtain the regression coefficient β and OR value. The way and degree of the impact of the environmental indicator on the social behavior indicator were judged according to the threshold range to which it belonged. The threshold ranges were as follows: weak positive impact, 0≤β<0.1 and 1.0≤OR<1.1; moderate positive impact, 0.1≤β<0.5 and 1.1≤OR<2.0; strong positive impact, 0.5≤β and 2.0≤OR; weak negative impact, -0.1≤β<0 and 0.9≤OR<1.0; moderate negative impact, -0.5≤β<-0.1 and 0.5≤OR<0.9; strong negative impact, β<-0.5 and OR<0.5.

[0079] S6. Impact rating and output: Combined with the regression analysis results in step S5, the impact of various environmental indicators in public spaces on social behavior is ranked, and the time-space thermal plane of each environmental indicator, the social behavior thermal plane, and the environmental indicator impact ranking table are displayed and output in three dimensions through a holographic digital projector and a visualization system.

Claims

1. A method for identifying social behaviors and environmental impacts in public spaces based on a sensory device network, characterized in that: The steps include: (1) Selecting a public space area through an interactive map on a web page interface, and recording the geographic location information of the public space area based on the user's selection input; (2) Determine the number of sensing devices and the layout of measurement points based on the geographic location information of the public space area in step (1), deploy a sensing device network in the public space area, the sensing device network includes environmental sensing devices and behavior sensing devices, test the working status and coverage of the devices, and transmit continuous monitoring data to the data center; (3) The physical environment data of the public space area collected by the environmental sensing device is aggregated into the corresponding spatial grid unit according to the time node, and a grid data matrix with time series information is generated. The continuous environmental data distribution of the public space area is generated by the spatial interpolation method, and the average value of the environmental data within the unit time interval is calculated and stored in the database in the form of structured data; (4) Use convolutional neural networks to process video data from behavior sensing devices in real time, detect and track individuals in the crowd, and obtain individual position coordinates and motion trajectories; use multi-camera data fusion technology to eliminate the impact of occlusions and video angles, and merge redundant data; combine social distance, facial orientation, and body movement recognition to determine the individual's social status and social behavior type, add spatial raster data to social behavior indicators, and store them in the database in the form of structured data; (5) Based on the structured data of the physical environment in step (3) and the structured data of social behavior in step (4), the effects of fixed spatial locations and time nodes on social behavior are eliminated through spatial regression models and time series decomposition, thereby obtaining a social behavior data set that only retains the effects of environmental factors; correlation analysis and multivariate regression analysis are combined to determine whether various environmental indicators have an impact on social behavior, and to determine the mode and extent of the impact; (6) Based on the regression analysis results in step (5), the degree of influence of various environmental indicators of public spaces on social behavior is ranked and output in three-dimensional display.

2. The method for identifying social behaviors and environmental impacts in public spaces based on a sensing device network according to claim 1, characterized in that: The step (1) specifically includes the following steps: loading and displaying an interactive map on a web page interface through an HTML container element, receiving a user's selection input on the map, wherein the selection input refers to selecting a public space area in the map; and recording geographic location information of the public space area based on the user's selection input, wherein the geographic location information includes all vertex coordinate geometric information, the geographic coordinates of the geometric center point, and the geographic area of ​​the public space area.

3. The method for identifying social behaviors and environmental impacts in public spaces based on a sensing device network according to claim 1, characterized in that: In step (2), the environmental sensing device integrates temperature and humidity sensors, light sensors and wind environment sensors, and is evenly distributed in the public space area in a 20m×20m grid; the behavior sensing device integrates a mobile power supply and a network camera, and the effective sensing range of the behavior sensing device is a plane area with a radius of 50m and an angle of 80° centered on the device. The boundary and geometric center of the public space area are determined, and the public space area is divided into 5m×5m grid units. The number of devices and the location of the points that achieve full grid unit coverage and minimum overlap are calculated through a genetic algorithm.

4. The method for identifying social behaviors and environmental impacts in public spaces based on a sensing device network according to claim 1, characterized in that: The structured data in step (3) includes a grid unit identifier, a timestamp, an environmental indicator type and a data average value of the data. The environmental indicator type includes a light environment indicator, a wind environment indicator and a thermal environment indicator. The light environment indicator includes the sunshine hours per minute and per hour. The sunshine hours are calculated cumulatively based on the time when the direct radiation measured by the light sensor is less than or equal to 120 W / ㎡. The wind environment indicator includes the average wind speed and average wind direction per minute. The average wind speed and average wind direction per minute are calculated based on the wind speed and wind direction measured by the wind environment sensor. The thermal environment indicator includes the average temperature and relative humidity per minute. The average temperature and relative humidity per minute are calculated based on the temperature and humidity measured by the temperature and humidity sensor.

5. The method for identifying social behaviors and environmental impacts in public spaces based on a sensing device network according to claim 1, characterized in that: The multi-camera data fusion technology in step (4) refers to combining the position coordinates (x c ,y c ,z c ) and shooting direction (θ x ,θ y ,θ z ), the coordinates of the center point of the individual image in the photographic image (c x ,c y ) and the focal lengths f on the x-axis and y-axis on the photographic image plane determined by the vertical angle of the camera and the sensor size of the camera x and f y , construct the camera's internal parameter matrix and the external parameter matrix [R|T], where the rotation matrix R = R z (θ z )·R y (θ y )·R x (θ x ), where R x , R y , R z The rotation matrix and translation matrix are respectively the rotation angles around the x, y, and z axes. where t x ,t y and t z The x, y, and z axes are the translations of the center coordinates of the individual image in the photographic image, respectively. The projection matrix P = K · [R|T] of the camera is then defined. The individual image coordinates are converted into spatial plane coordinates through the projection matrix. The trajectory of each individual in the camera is recorded based on the converted spatial plane coordinates to obtain a series of grid coordinate points. The trajectory data of each camera is matched by timestamp and spatial position, and the distance between two trajectories in the same time period is calculated. Where (X i,t ,Y i,t ) represents the coordinate position of individual i at time frame t, (X j,t ,Y j,t ) represents the coordinate position of individual j at time frame t; the distance threshold is set to 2m, and when d<2m, the individual similarity is identified through cross-device pedestrian re-identification (Re-Identification); the ResNet deep learning model is used to extract the feature vector of the individual image taken by each camera, and the cosine similarity is used to calculate the feature vector similarity between different individuals in the same time period under different cameras: Where: x·y is the dot product of the feature vectors, ||x|| and ||y|| are the Euclidean norms of vectors x and y respectively; the cosine similarity threshold is set to 0.5, and the feature vectors with cosine similarity>0.5 are regarded as the same object and assigned the same identity ID; the trajectories with the same identity ID in different cameras are associated, and the matched trajectories in each camera are merged to remove duplicate trajectories and form unified trajectory data; the matched trajectories in different cameras are merged into a continuous trajectory sequence, and the plane coordinates, video frame images, and identity ID information in each time frame are retained. The merged plane grid coordinates (X t ,Y t ) and the trajectory data of the corresponding video images are stored in the database in the form of structured data.

6. The method for identifying social behaviors and environmental impacts in public spaces based on a sensing device network according to claim 5, characterized in that: In the step (4), the social status and social behavior type of an individual are determined by combining social distance, facial orientation and body movement recognition. First, the database is sorted, the motion trajectory of the individual is segmented according to the timestamp, and the video frame is extracted at a time interval of 0.5 s; by analyzing the motion trajectory, according to the distance between any two individuals at different time points in the plane coordinate data, for each timestamp t, the Euclidean distance d between individual i and individual j is calculated. ij (t): Where: x i (t), y i (t) is the plane coordinate of individual i at time t, based on the preset social distance threshold D threshold , if d ij (t)≤D threshold If the duration Δt exceeds the threshold, it is judged that there is social interaction between two or more individuals; the face area is located by the MTCNN face detection algorithm, and the Euler angles (α, β, γ) of the head are calculated through head posture estimation to match the facing directions between individuals. If the yaw angle β of two or more individuals is i and β j The difference between them is within a certain threshold Δβ threshold Inner, that is, |β i -β j |≤Δβ threshold , and the relative distance meets the social distance threshold, it is judged that they are engaging in social interaction; the OpenPose human posture estimation algorithm is used to extract the key node coordinates P in each video frame = {p1, p2...p n }, by analyzing the time series of joint points, the motion trajectory and speed of key limb parts are calculated, and the movement distance of joint point k between time t and t+Δt is: The extracted feature vector v = {v1,v2...v n } is input into the SVM classification model and the social distance d is integrated ij (t), facial orientation β, and limb movement features v are comprehensively analyzed: F=concat(d ij (t),β i ,β j ,v) The probability distribution P(class|F) is output by the random forest model, and the three behavior types of fitness, entertainment and general social are calculated based on the feature vector F that is concatenated from multiple features. The final output is a structured data set including individual ID, movement trajectory, social status, social behavior type, time and spatial location of the behavior, and corresponding movement status detailed information.

7. The method for identifying social behaviors and environmental impacts in public spaces based on a sensing device network according to claim 6, characterized in that: The structured data in step (4) includes a grid unit identifier, a timestamp, a behavior indicator type and an indicator data value of the data. The behavior indicator type includes a social status and a social behavior type. The social status includes two data values, yes and no. The social behavior type includes three data values, namely, fitness, entertainment and general social.

8. The method for identifying social behaviors and environmental impacts in public spaces based on a sensing device network according to claim 1, characterized in that: In step (5), the influence of fixed spatial locations and time nodes on social behavior is eliminated by using a spatial regression model and time series decomposition. First, the spatial autocorrelation of the structured social behavior data is calculated to construct a spatial weight matrix W for capturing the relationship between spatial locations, where the elements of the matrix W are ij Represent the spatial interaction between location i and location j, and construct a spatial autoregressive model SAR: Y ij =ρWY ij +X ij β+∈ ij Where: Y ij is the social behavior of individual i at location j, ρ is the spatial autocorrelation coefficient, X ij is the environmental indicator, β is the coefficient vector of the independent variable, ∈ ij is the error term; Use maximum likelihood estimation MLE to estimate the model parameters ρ and β. According to the estimated ρ, calculate the spatial autocorrelation part of the social behavior data: Y spatial =ρWY Remove the spatial autocorrelation part from the original data to obtain the non-spatial autocorrelation part, that is, the residual Y residual : Y residual =Y-Y spatial -Xβ Residual Y residual That is, after removing spatial autocorrelation, social behavior data at fixed spatial locations are excluded; Secondly, perform time series decomposition calculation on structured social behavior data: Y t =T t +S t +R t Among them, Y t is social behavior data, T t is the trend component, S t is the seasonal component, R t is the residual component; removing the trend and seasonality from the original data, we get the residual R t =Y t -T t -S t , which is the social behavior data without the influence of time nodes; the residual data Y obtained by the spatial autocorrelation model residual The residual data R obtained by time series decomposition t Combined, we get the social behavior dataset Y after comprehensive spatial and temporal adjustment adj : Y adj =Y residual +R t Dataset Y adj The influence of fixed spatial locations and time nodes on social behavior is eliminated, and the characteristics of behavioral changes related to the environment are retained.

9. The method for identifying social behaviors and environmental impacts in public spaces based on a sensing device network according to claim 8, characterized in that: In step (5), the correlation analysis and multivariate regression analysis are combined to determine whether each environmental indicator has an impact on social behavior, and to determine the impact mode and degree, which refers to standardizing the environmental indicator data in the structured data of the physical environment in step (3) and converting the social behavior data set Y adj The social status and three behavior type data in the data were coded as binary variables, in which "0" represents "no" and "1" represents "yes". The correlation coefficients between the environmental indicator data and the social status data and the three behavior type data were calculated by point-by-point serial correlation analysis, and the environmental indicators significantly correlated with the social status and behavior type were screened out. The significant correlation refers to the absolute value of the correlation coefficient being greater than 0.

3. The data of the screened environmental indicators were extracted and regressed with the binary coded social status and three behavior type data by logistic regression method to obtain the regression coefficient β and OR value. The way and degree of the impact of the environmental indicator on the social behavior indicator were judged according to the threshold range to which it belonged. The threshold ranges were as follows: weak positive impact, 0≤β<0.1 and 1.0≤OR<1.1; moderate positive impact, 0.1≤β<0.5 and 1.1≤OR<2.0; strong positive impact, 0.5≤β and 2.0≤OR; weak negative impact, -0.1≤β<0 and 0.9≤OR<1.0; moderate negative impact, -0.5≤β<-0.1 and 0.5≤OR<0.9; strong negative impact, β<-0.5 and OR<0.

5.

10. The method for identifying social behaviors and environmental impacts in public spaces based on a sensing device network according to claim 1, characterized in that: In the step (6), a holographic digital projector and a visualization system are used to display and output in three dimensions the time-space thermal plane of each environmental indicator, the social behavior thermal plane, and the environmental indicator impact ranking table.

Images