A method, system and device for monitoring the vitality of landscape space based on human action recognition

Abstract

The present invention relates to the technical field of landscape space vitality monitoring, solving the current technical problem of being unable to effectively monitor the vitality status of landscape spaces. In particular, it relates to a landscape space vitality monitoring method based on human motion recognition. The monitoring method uses human motion recognition in a landscape space as a basis to monitor the vitality status of the landscape space. The monitoring method includes the following processes: obtaining a human motion feature vector set within at least a certain area of the landscape space within a time period T1; analyzing the human motion feature vector set and making a preliminary judgment on the type of human motion. The present invention achieves the purpose of effectively monitoring the vitality of the landscape space and providing management personnel with a basis for judging the rearrangement of the scene. It can improve the spatial vitality of the landscape space, increase tourists' popularity of the landscape space, and enhance tourists' experience and comfort in the landscape space.

Description

Technical Field

[0001] The present invention relates to the technical field of landscape space vitality monitoring, and in particular to a landscape space vitality monitoring method, system and device based on human motion recognition. Background Art

[0002] The form and function of landscape space are interrelated. Spatial form serves function, while function also shapes spatial form. For example, a landscape space created for residential use should be quiet and simple, furnished with essential lighting, seating, and activity areas. It should avoid large, ostentatious structures or strong, stimulating colors. Instead, it should feature a wide variety of evergreen and deciduous plants, including a rich variety of groundcovers, flowers, shrubs, and trees.

[0003] At present, in the landscape space, the ground cover, flowers, shrubs, trees and other landscapes in the space need regular maintenance. At the same time, the scene layout in the landscape space needs to be changed according to the actual situation. The rearrangement of the scene needs to be determined by the popularity of the landscape, that is, the vitality of the landscape space. Under normal circumstances, different layout methods can directly affect the vitality of the entire landscape space. The improvement of vitality will enhance the experience of people in the landscape space and improve comfort.

[0004] At the same time, the growth status of various vegetation in the landscape space can also affect the vitality of the entire space. That is, the vegetation growth trend is poor, or there is a lack of corresponding watering, fertilizing and pruning measures, which makes the corresponding vegetation unpopular among people, thereby reducing the vitality of the entire landscape space.

[0005] From the above factors, it can be found that the vitality of landscape space directly affects its popularity and has a serious impact on its functional services. However, there is currently a lack of corresponding technical means to monitor the vitality of landscape space, resulting in the inability to obtain timely evidence of changes related to the vitality of landscape space. As a result, managers are unable to rearrange scenes in a timely manner according to the vitality status of the space, which not only leads to a decline in the vitality of the landscape space, but also makes it easy for supervision to be inadequate and for idle space to become abandoned. Summary of the Invention

[0006] In response to the shortcomings of the existing technology, the present invention provides a landscape space vitality monitoring method, system and device based on human motion recognition, which solves the current technical problem that the vitality status of the landscape space cannot be effectively monitored, achieves the purpose of effectively monitoring the vitality of the landscape space and provides managers with a basis for judging the rearrangement of the scene, can improve the spatial vitality of the landscape space, increase the popularity of the landscape space among tourists, and improve the experience and comfort of tourists in the landscape space.

[0007] To solve the above technical problems, the present invention provides the following technical solution: a method for monitoring the vitality of a landscape space based on human motion recognition. The method uses human motion recognition in a landscape space as a basis to monitor the vitality of the landscape space. The method includes the following steps:

[0008] Obtaining a set of human motion feature vectors in at least a certain area of a landscape space within a time period T1;

[0009] Analyze the human motion feature vector set and make a preliminary judgment on the human motion type;

[0010] The spatial vitality value V1 of the area in the period is obtained based on the preliminary judgment of the human body movement type;

[0011] According to the human body movement type in the area in the next time period T2, T3...Tn, the spatial vitality value V2, V3...Vn in the time period is obtained;

[0012] Compare the spatial vitality values V1, V2, V3…Vn accumulated in different time periods within the area;

[0013] Based on the comparison results, it is determined whether the vitality of the landscape space in the area has been improved;

[0014] If the vitality of the landscape space increases, the specific landscape with the highest human movement frequency in the area is analyzed, and the specific location of the specific landscape and the specific characteristic vector of the human movement are listed. The specific characteristic vector is the human joint movement of the human body in the location;

[0015] If the vitality of a landscape space decreases, the area is marked as a key supervision area, and the reasons for the reduction of the specific feature vectors of the human body in this area are analyzed, including a landscape with fewer human joint movements in this area, and the individual vitality ranking of each landscape is listed according to the number of human joint movements.

[0016] Furthermore, the method for human action recognition includes the following steps:

[0017] Obtaining motion amplitude characteristics of each joint point of the subject's body within at least a certain time period;

[0018] Extracting human joint points with motion amplitude characteristics within the time period;

[0019] Analyze the correlation between various joints of the human body with motion amplitude characteristics;

[0020] Extract motion data features based on the motion amplitude features of human joint points;

[0021] Determine the human motion feature vector based on the correlation between each joint point and data characteristics;

[0022] A human motion feature vector set is constructed based on the accumulation of human feature vectors.

[0023] Furthermore, the method for human motion recognition includes:

[0024] The human joint point movements that occur within this time period are extracted one by one, and the features 1, 2, 3, ..., n corresponding to the human joint points are obtained;

[0025] The above-mentioned features 1, 2, 3…n are input into the three-dimensional model of human body motion to construct a dynamic process. The dynamic process is used to derive the degree of correlation between the motions of the various limbs of the target human body, thereby determining the combination of the motions of the human body joints and the specific motions of the limbs after the combination.

[0026] Furthermore, the motion data feature is a specific motion of each limb of the human body obtained by fusing feature 1, feature 2, feature 3...feature n.

[0027] Furthermore, in obtaining the human feature vector set, the human action feature vector set is a set of actions performed by human joints in the time period T1, which includes each single feature vector performed by the human body and is accumulated in the time period T1.

[0028] Furthermore, when obtaining the human body feature vector set, the area within the landscape space can be divided according to the actual on-site conditions of the landscape space. The total area is divided into multiple independent sub-areas, where a certain area corresponds to a sub-area, and cameras and infrared thermal imagers are deployed in the area to capture human joint movements, and a corresponding three-dimensional human movement model is established through the server.

[0029] Furthermore, when analyzing the human body motion feature vector set, the specific movements made by the human body are synchronously analyzed based on the accumulation and frequency of occurrence of individual feature vectors in the human body feature vector set, and combined with the human body joint points, thereby making corresponding judgments on the human body motion types.

[0030] Furthermore, in the comparison of spatial vitality values, the comparison of spatial vitality values needs to be compared with the values generated in different time periods in the specific area. In the three cumulative time periods of T1, T2 and T3, the corresponding spatial vitality values are V1, V2 and V3 respectively. At this time, V1 and V2 are compared to determine whether the spatial vitality has increased or decreased, and V2 and V3 are compared to determine whether the spatial vitality has increased or decreased.

[0031] A device for human motion recognition, comprising:

[0032] A first extraction module, the first extraction module is used to extract human joint points with motion amplitude characteristics within the time period;

[0033] An analysis module, the analysis module is used to analyze the correlation between various joint points of the human body with motion amplitude characteristics;

[0034] A second extraction module, the second extraction module is used to extract motion data features according to the motion amplitude features of the human body joint points;

[0035] A determination module, configured to determine a human motion feature vector based on the correlation between joint points and data features;

[0036] A construction module is used to construct a human motion feature vector set based on the accumulation of human feature vectors.

[0037] A landscape space vitality monitoring device based on human motion recognition, comprising:

[0038] An acquisition module, configured to acquire a set of human motion feature vectors in at least a certain area of a landscape space within a time period T;

[0039] An analysis and judgment module, which is used to analyze the human body motion feature vector set and make a preliminary judgment on the human body motion type;

[0040] A first spatial vitality value module, configured to obtain a spatial vitality value V1 of the region within the time period according to a preliminarily determined human motion type;

[0041] A second space vitality value module, which is used to obtain the space vitality values V2, V3...Vn in the next time period T2, T3...Tn according to the human body movement type in the area;

[0042] A comparison module, the comparison module is used to compare the spatial vitality values V1, V2, V3...Vn accumulated in different time periods within the area;

[0043] An evaluation module is used to evaluate whether the vitality of the landscape space in the area has been improved based on the comparison results.

[0044] A method for monitoring the vitality of a landscape space based on human motion recognition, comprising capturing a video image of human motion in a landscape space, and comprising the following steps:

[0045] Receiving a video image of a human body movement within a certain period of time uploaded by a shooting device;

[0046] Preprocess the human body motion video images and capture the joint points of the human body motion;

[0047] Human action recognition is performed based on the joint points of human actions, and a human action feature vector set is constructed;

[0048] Uploading the human motion feature vector set to the second server;

[0049] Send an upload signal of the video image of the next period to the shooting device.

[0050] Furthermore, pre-processing the human body motion video image includes setting the playback speed of the human body motion video image, reducing the frame rate of the human body motion video image to improve the clarity, and playing the human body motion video image at a slow speed reduced by 50-80%.

[0051] A landscape space vitality monitoring device based on human motion recognition, comprising:

[0052] A receiving module, configured to receive a video image of a human body movement within a certain period of time uploaded by a shooting device;

[0053] A preprocessing module, which is used to preprocess the human body action video image and complete the capture of the joint points of the human body action;

[0054] A human action recognition module, which is used to recognize human actions based on the joint points of human actions. The recognition of human actions is completed by a human action recognition method, and a human action feature vector set is constructed;

[0055] An uploading module, configured to upload the human motion feature vector set to a second server;

[0056] The sending module is used to send an upload signal of the video image of the next time period to the shooting device.

[0057] Furthermore, the device also includes a storage module, which is used to store local video images captured by the shooting device.

[0058] A landscape space vitality monitoring system based on human motion recognition includes: a shooting device, a first server, a second server and a terminal;

[0059] The shooting device establishes a communication connection with the first server, and the shooting device is used to shoot video images of human body movements in the landscape space;

[0060] The first server is used to receive the human body action video image, process it and send the human body action feature vector set to the second server;

[0061] The second server establishes a communication connection with the first server, the second server receives the human motion feature vector set and determines whether the landscape space vitality in the area is increased or decreased;

[0062] The second server establishes a communication connection with the terminal, and the terminal is used to display the vitality status of each area in the landscape space and can retrieve the space vitality status of each area in different time periods in the morning, noon and afternoon.

[0063] Furthermore, the shooting device is a camera and an infrared thermal imager.

[0064] A landscape space vitality monitoring system based on human motion recognition includes: a shooting device, a first server, a second server and a terminal;

[0065] The photographing device establishes a communication connection with the first server, and the photographing device is used to collect human motion data in the landscape space;

[0066] The first server is used to receive human motion data, perform human motion recognition on the extracted motion amplitude features using an SVM classification algorithm, and send the human motion recognition result to the second server;

[0067] The second server establishes a communication connection with the first server, and the second server receives the human motion recognition result and determines whether the vitality of the landscape space in the area is increased or decreased through comparison;

[0068] The second server establishes a communication connection with the terminal, and the terminal is used to display the vitality status of each area in the landscape space and can retrieve the space vitality status of each area in different time periods in the morning, noon and afternoon.

[0069] Furthermore, the shooting device is a non-contact somatosensory device Kinect.

[0070] Through the above technical solution, the present invention provides a method, system and device for monitoring the vitality of a landscape space based on human motion recognition, which has at least the following beneficial effects:

[0071] 1. The present invention monitors the vitality of the landscape space through human motion recognition, can monitor the spatial vitality of the entire landscape space in real time online during the day, and can make timely and effective evaluations based on the increase and decrease of spatial vitality, providing a basis for managers' scene arrangement and other measures. At the same time, it can make reminders when the spatial vitality continues to decrease, so that corresponding treatment measures can be made in a timely and rapid manner. After the landscape space is rearranged, it is verified through monitoring whether it is attractive enough to people and whether the update is effective. Therefore, the purpose of effectively monitoring the vitality of the landscape space and providing managers with a basis for judging the rearrangement of the scene is achieved, which can improve the spatial vitality of the landscape space, increase tourists' popularity of the landscape space, and improve tourists' experience and comfort in the landscape space.

[0072] 2. The present invention can quickly identify visitors' body movements by online monitoring of the landscape space, and determine the popularity of the vegetation layout in the space based on the frequency of the body movements. At the same time, it provides a reference for the individual popularity of each type of vegetation and the growth trend of each type of vegetation. Therefore, the present invention can effectively monitor the vitality of the landscape space, and avoid the phenomenon of inadequate supervision and the occurrence of idle space with waste gas.

[0073] 3. The present invention provides a monitoring basis for the vitality of landscape space by adopting the recognition of human movements. It can quickly determine the movement type of tourists through the human movement recognition method. At the same time, the SVM classification algorithm has excellent classification ability. It can have a higher classification ability when facing linear and nonlinear high-dimensional feature spaces, effectively avoiding the dimensionality disaster, thereby improving the accuracy of human movement recognition, and can quickly make a basis for judging the movement type, thereby improving the accuracy of monitoring the space vitality status. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0075] Figure 1 This is a flowchart of the landscape space vitality monitoring method of the present invention;

[0076] Figure 2 This is a functional block diagram of the landscape space vitality monitoring device of the present invention;

[0077] Figure 3 This is a schematic diagram of the identification points of the human body joints of the present invention;

[0078] Figure 4 A schematic diagram of dividing the total area within the landscape space of the present invention into sub-areas;

[0079] Figure 5 A flowchart of a method for human motion recognition according to the present invention;

[0080] Figure 6 This is a functional block diagram of a device for human motion recognition according to the present invention;

[0081] Figure 7 This is a functional block diagram of the landscape space vitality monitoring system of the present invention;

[0082] Figure 8 A flow chart showing the interaction of the landscape space vitality monitoring system of the present invention;

[0083] Figure 9 This is a flowchart of a method for monitoring the vitality of a landscape space in the second embodiment of the present invention;

[0084] Figure 10 This is a functional block diagram of a landscape space activity monitoring device in a second embodiment of the present invention;

[0085] Figure 11 This is a block diagram of the principle of feature extraction of motion data of the present invention;

[0086] Figure 12 This is a flow chart of the interaction of the landscape space vitality monitoring system in Example 3 of the present invention.

[0087] In the figure: 110, acquisition module; 120, analysis and judgment module; 130, first space vitality value module; 140, second space vitality value module; 150, comparison module; 160, evaluation module; 111, first extraction module; 112, analysis module; 113, second extraction module; 114, judgment module; 115, construction module; 100, shooting device; 200, first server; 300, second server; 400, terminal; 210, receiving module; 220, preprocessing module; 230, human motion recognition module; 240, upload module; 250, sending module. DETAILED DESCRIPTION

[0088] To make the above-mentioned objectives, features, and advantages of the present invention more clearly understood, the present invention is further described below in detail with reference to the accompanying drawings and specific embodiments. This will enable a full understanding of how this application uses technical means to solve technical problems and achieve technical effects, and to implement the invention accordingly.

[0089] Those skilled in the art will appreciate that all or part of the steps in the above-mentioned embodiment methods can be accomplished by instructing the relevant hardware through a program. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0090] Example 1

[0091] Please refer to Figure 1-8 , shows a method for monitoring the vitality of a landscape space based on human motion recognition according to a first embodiment of the present invention. The monitoring method uses human motion recognition in a landscape space as a basis for monitoring the vitality of the landscape space. The monitoring method includes the following steps:

[0092] S11, obtaining a human motion feature vector set in at least a certain area of the landscape space within a time period T1, the human motion feature vector set is a set of motions made by the human joints within the time period T1, which includes each single feature vector made by the human body and is accumulated within the time period T1, such as Figure 4 As shown, according to the actual situation of the landscape space, the area within the landscape space can be divided into multiple independent sub-areas, where a certain area corresponds to a sub-area, and cameras and infrared thermal imagers are deployed in the area to capture human joint movements, and a corresponding three-dimensional human movement model is established through the server;

[0093] S12, analyzing the human motion feature vector set and making a preliminary judgment on the human motion type, based on the accumulation and frequency of the individual feature vectors in the human motion feature vector set, and combining the human joint points to synchronously analyze the specific motions made by the human body, thereby making a corresponding judgment on the human motion type, such as Figure 3 As shown in the figure, by dynamically identifying whether a person is in a walking state at points A and B, it is possible to obtain basic point information such as the length of time a person stops in front of a certain landscape during the walking process;

[0094] The running state of the human body in the area is dynamically identified through the four points A, B, E, and F;

[0095] Identify the squatting and standing state of the human body through the height difference of point C;

[0096] The dynamic recognition of points C, D, and G is used to detect the human body's sniffing of vegetation after squatting;

[0097] The turning motion of the human head is recognized through the dynamic recognition of points D and G, and the basic point information such as the viewing frequency of the person is determined by combining the landscape in that direction;

[0098] Dynamically identify the jumping state of the human body through the five points A, B, C, E, and F;

[0099] The system dynamically identifies upper limb movements through the four points E, E1, F, and F1, such as picking, pulling branches to the head to sniff, and breaking and damaging vegetation. It then issues warnings when these actions occur.

[0100] S13. Based on the preliminarily determined human motion types, a spatial vitality value V1 of the area during the time period is calculated. Information such as the frequency, duration, and amplitude of a particular human motion type is used as a criterion for determining the spatial vitality value. For example, the occurrence of human motion types in the area may include: the duration of a person's pause in front of a certain landscape while walking, the number of times the gaze shifts to a landscape while running, the occurrence of a person leaping joyfully in front of a landscape, the squatting and sniffing action of a person, the duration of a person's stay in front of a landscape, the frequency of a person's head turning to look and upper limbs to reach for something, and the number of times a person takes a photo.

[0101] The more the above behaviors occur, the greater the spatial vitality value of the area is, indicating that the landscape in the area is popular among people, including the growth status of flowers and plants, the fragrance of flowers, the overall layout of the landscape, the status of vegetation, and the degree of popularization of the landscape, etc.

[0102] S14, according to the human body movement type in the area in the next time period T2, T3...Tn, the spatial vitality value V2, V3...Vn in the time period is obtained;

[0103] S15. Compare the spatial vitality values V1, V2, V3…Vn accumulated in different time periods within the area. The comparison of spatial vitality values needs to be based on the values generated in different time periods within the specific area. For example, in the three cumulative time periods T1, T2, and T3, the corresponding spatial vitality values are V1, V2, and V3, respectively. At this time, V1 and V2 are compared to determine whether the spatial vitality has increased or decreased, and V2 and V3 are compared to determine whether the spatial vitality has increased or decreased.

[0104] The space vitality value accumulated in the morning is marked as A, and the time period corresponding to A is set as AT1, AT2, AT3…ATn, and the space vitality value corresponding to this time period is set as AV1, AV2, AV3…AVn; the space vitality value accumulated at noon is marked as B, and the time period corresponding to B is set as BT1, BT2, BT3…BTn, and the space vitality value corresponding to this time period is set as BV1, BV2, BV3…BVn; the space vitality value accumulated in the afternoon is marked as C, and the time period corresponding to C is set as BT1, BT2, BT3…BTn, and the space vitality value corresponding to this time period is set as BV1, BV2, BV3…BVn. The corresponding time periods are set as CT1, CT2, CT3…CTn, and the spatial vitality values corresponding to the time periods are set as CV1, CV2, CV3…CVn. The ranking is obtained by comparing the comprehensive spatial vitality values in the three major time periods A, B, and C in a day. The comprehensive spatial vitality value is AV1+AV2+AV3+…AVn / n, taking the time period AT1, AT2, AT3…ATn corresponding to A as an example. The spatial vitality value corresponding to the time period is AV1+AV2+AV3+…AVn / n. This can be used to determine whether the vitality of the landscape space in the corresponding time period of a day is affected by the morning, noon, and afternoon.

[0105] S16. Evaluate whether the vitality of the landscape space in the area has increased based on the comparison results. If the vitality of the landscape space has increased, analyze the specific landscape in which the human body has the highest frequency of movement in the area, and list the specific locations of the specific landscapes and the specific characteristic vectors of the human body movements. The specific characteristic vectors are the human joint movements of the human body at the locations. If the vitality of the landscape space has decreased, mark the area as a key supervision area, and analyze the reasons for the decrease in the specific characteristic vectors of the human body in the area, including a landscape in which the human body has fewer human joint movements in the area, and list the individual vitality rankings of each landscape according to the number of human joint movements.

[0106] The method of human action recognition includes:

[0107] S21, obtaining motion amplitude characteristics of each joint point of the subject's human body within at least a certain time period, the motion amplitude characteristics being determined by the specific motion of each joint point of the subject's human body, constructing a three-dimensional human motion model based on the motion of the human joint points, and inputting the motion amplitude characteristics into the three-dimensional human motion model to construct a dynamic process of each joint point;

[0108] S22, extracting human joint points with motion amplitude characteristics within the time period, and extracting the human joint point movements occurring within the time period one by one, such as feature 1, feature 2, feature 3 ... feature n corresponding to the human joint points;

[0109] S23, analyzing the correlation between the joint points of the human body with the motion amplitude characteristics, inputting the above-mentioned features 1, 2, 3, ..., n into the three-dimensional human motion model to construct a dynamic process, and obtaining the degree of correlation between the motions of the subject's limbs from the dynamic process, thereby determining the combination of the human joint point motions and the specific motion of each limb after the combination;

[0110] S24. Extract motion data features based on the motion amplitude features of the human joint points. The motion data features are the specific motions of the human limbs after fusing features 1, 2, 3…n. The specific steps of motion data feature extraction are: constructing human motion vectors. The angles between these vectors are used to represent the rotation of the limbs, and the relative modulus ratio is used to represent the relative displacement of the limbs. The two types of parameters generate fusion feature extraction to describe the motion, and generate a fusion feature space vector for SVM to perform motion classification and recognition. The feature extraction process of motion data is as follows: Figure 11 shown.

[0111] S25. Determine a human motion feature vector based on the correlation between the joint points and the data features, and determine the human motion feature vector based on the correlation between the movements of the subject's limbs and the specific movements. That is, determine the specific movements made by the subject's limbs through the variables obtained by fusing the feature space.

[0112] S26. Construct a human motion feature vector set based on the accumulation of human feature vectors. The human motion feature vector set obtained based on the accumulation of human feature vectors is the specific type of human motion, such as squatting to smell, walking, running, etc.

[0113] The method of human action recognition uses the SVM classification algorithm to perform human action recognition on the extracted feature vectors.

[0114] A device for human motion recognition, comprising:

[0115] The first extraction module 111 is used to extract the human body joint points with motion amplitude characteristics within the time period;

[0116] An analysis module 112 is used to analyze the correlation between various joint points of the human body with motion amplitude characteristics;

[0117] The second extraction module 113 is used to extract motion data features based on the motion amplitude features of the human body joint points;

[0118] The determination module 114 is used to determine the human body motion feature vector according to the correlation between the joint points and the data features;

[0119] The construction module 115 is used to construct a human motion feature vector set according to the accumulation of human feature vectors.

[0120] A landscape space vitality monitoring system based on human motion recognition includes: a shooting device 100, a first server 200, a second server 300 and a terminal 400;

[0121] The shooting device 100 establishes a communication connection with the first server 200, and the shooting device 100 is used to shoot a video image of a human body movement in a landscape space;

[0122] The first server 200 is used to receive the human body action video image, process it and send the human body action feature vector set to the second server 300;

[0123] The second server 300 establishes a communication connection with the first server 200, and the second server 300 receives the human motion feature vector set and determines whether the landscape space vitality in the area is increased or decreased;

[0124] The second server 300 establishes a communication connection with the terminal 400. The terminal 400 is used to display the vitality status of each area in the landscape space and can retrieve the spatial vitality status of each area in different time periods in the morning, noon and afternoon according to needs.

[0125] The terminal 400 is any one of a PC, a notebook, a tablet or a mobile phone.

[0126] The shooting device 100 is a camera and an infrared thermal imager.

[0127] In this embodiment, the specific steps of the landscape space vitality monitoring system are as follows:

[0128] S1, shooting video images of human body movements in landscape space;

[0129] S2. Upload the recorded human action video image;

[0130] S3, receiving a video image of a human body movement within a certain period of time uploaded by a shooting device;

[0131] S4, pre-processing the human body action video image and completing the capture of the joint points of the human body action;

[0132] S5. Recognize human actions according to the joint points of human actions and construct a human action feature vector set;

[0133] S6. Uploading the human motion feature vector set to the second server;

[0134] S7, sending an upload signal of the video image of the next period to the shooting device;

[0135] S8, obtaining a set of human motion feature vectors in at least a certain area of the landscape space within a time period T1;

[0136] S9, analyzing the human motion feature vector set and making a preliminary judgment on the human motion type;

[0137] S10, obtaining a spatial vitality value V1 of the area during the period based on the preliminarily determined human motion type;

[0138] S11, according to the human body movement type in the area in the next time period T2, T3...Tn, the spatial vitality value V2, V3...Vn in the time period is obtained;

[0139] S12, comparing the spatial vitality values V1, V2, V3...Vn accumulated in different time periods within the area;

[0140] S13. Evaluate whether the vitality of the landscape space in the area has increased or decreased based on the comparison results;

[0141] S14. Display the vitality status of each area in the landscape space.

[0142] Through this embodiment, the vitality of the landscape space is monitored through human motion recognition, and the spatial vitality of the entire landscape space during the day can be monitored online in real time. It can also make timely and effective evaluations based on the increase and decrease of spatial vitality, providing a basis for managers' scene arrangement and other measures. At the same time, it can make reminders when the spatial vitality continues to decrease, so that corresponding treatment measures can be made in a timely and rapid manner. After the landscape space is rearranged, it is verified through monitoring whether it is attractive enough to people and whether the update is effective. Therefore, the purpose of effectively monitoring the vitality of the landscape space and providing managers with a basis for judging the rearrangement of the scene is achieved, which can improve the spatial vitality of the landscape space, increase tourists' popularity with the landscape space, and improve tourists' experience and comfort in the landscape space.

[0143] Example 2

[0144] Please refer to Figure 8-10 , shows a method for monitoring the vitality of a landscape space based on human motion recognition according to a second embodiment of the present invention, including a shooting device for shooting human motion video images in a landscape space and collecting human motion data. The method includes the following steps:

[0145] S31, receiving a video image of human body movements within a certain period of time uploaded by a shooting device, wherein the video image is recorded by a camera and an infrared thermal imager to complete the recording of human body movements, and the shooting device is provided with a storage module for storing the locally recorded video image;

[0146] S32, pre-processing the human body action video image and completing the capture of the joint points of the human body action, the pre-processing including setting the playback speed of the human body action video image, reducing the frame rate of the human body action video image to improve the clarity, playing the human body action video image at a slow speed reduced by 50-80%, and capturing the joint points of the human body action based on the played image;

[0147] S33, performing human motion recognition according to the joint points of the human motion, the recognition of the human motion is completed by a human motion recognition method, and a human motion feature vector set is constructed;

[0148] S34, uploading the human motion feature vector set to the second server;

[0149] S35. Send an upload signal to the shooting device, which is used to confirm to the shooting device that the human motion video image transmitted for the first time has been processed and the human motion recognition has been completed, and the shooting device is caused to upload the video image of the next period again through the signal.

[0150] A landscape space vitality monitoring device based on human motion recognition, comprising:

[0151] The receiving module 210 is used to receive a video image of a human body movement within a certain period of time uploaded by a shooting device;

[0152] The pre-processing module 220 is used to pre-process the human body motion video image and complete the capture of the joint points of the human body motion;

[0153] Human action recognition module 230, which is used to perform human action recognition based on the joint points of human actions. The recognition of human actions is completed by a human action recognition method, and a human action feature vector set is constructed;

[0154] The uploading module 240 is used to upload the human motion feature vector set to the second server;

[0155] The sending module 250 is used to send an upload signal of the video image of the next time period to the shooting device.

[0156] The device further includes a storage module 260, which is used to store the local video images captured by the shooting device.

[0157] A landscape space vitality monitoring system based on human motion recognition includes: a shooting device 100, a first server 200, a second server 300 and a terminal 400;

[0158] The shooting device 100 establishes a communication connection with the first server 200, and the shooting device 100 is used to shoot a video image of a human body movement in a landscape space;

[0159] The first server 200 is used to receive the human body action video image, process it and send the human body action feature vector set to the second server 300;

[0160] The second server 300 establishes a communication connection with the first server 200, and the second server 300 receives the human motion feature vector set and determines whether the landscape space vitality in the area is increased or decreased;

[0161] The second server 300 establishes a communication connection with the terminal 400. The terminal 400 is used to display the vitality status of each area in the landscape space and can retrieve the spatial vitality status of each area in different time periods in the morning, noon and afternoon according to needs.

[0162] The terminal 400 is any one of a PC, a notebook, a tablet or a mobile phone.

[0163] The shooting device 100 is a camera and an infrared thermal imager.

[0164] In this embodiment, the specific steps of the landscape space vitality monitoring system are as follows:

[0165] S1, shooting video images of human body movements in landscape space;

[0166] S2. Upload the recorded human action video image;

[0167] S3, receiving a video image of a human body movement within a certain period of time uploaded by a shooting device;

[0168] S4, pre-processing the human body action video image and completing the capture of the joint points of the human body action;

[0169] S5. Recognize human actions according to the joint points of human actions and construct a human action feature vector set;

[0170] S6. Uploading the human motion feature vector set to the second server;

[0171] S7, sending an upload signal of the video image of the next period to the shooting device;

[0172] S8, obtaining a set of human motion feature vectors in at least a certain area of the landscape space within a time period T1;

[0173] S9, analyzing the human motion feature vector set and making a preliminary judgment on the human motion type;

[0174] S10, obtaining a spatial vitality value V1 of the area during the period based on the preliminarily determined human motion type;

[0175] S11, according to the human body movement type in the area in the next time period T2, T3...Tn, the spatial vitality value V2, V3...Vn in the time period is obtained;

[0176] S12, comparing the spatial vitality values V1, V2, V3...Vn accumulated in different time periods within the area;

[0177] S13. Evaluate whether the vitality of the landscape space in the area has increased or decreased based on the comparison results;

[0178] S14. Display the vitality status of each area in the landscape space.

[0179] Through this embodiment, by taking online monitoring of the landscape space, it is possible to quickly identify the human movements of tourists and determine the popularity of the vegetation layout in the space based on the frequency of human movements. At the same time, it provides a reference basis for the individual popularity of each vegetation and the growth trend of each vegetation. For example, if there are fewer smelling actions, it may be because the fragrant flowers need to be maintained, or if there are fewer running behaviors, it may be because more weeds have grown. Therefore, the present invention can effectively monitor the vitality state of the landscape space, and at the same time avoid the phenomenon of inadequate supervision and the occurrence of idle space for waste gas.

[0180] Example 3

[0181] Please refer to Figure 11-12 , shows a landscape space vitality monitoring system based on human motion recognition according to embodiment 3 of the present invention, including: a shooting device 100, a first server 200, a second server 300 and a terminal 400;

[0182] The camera 100 establishes a communication connection with the first server 200, and the camera 100 is used to collect human motion data in the landscape space;

[0183] The first server 200 is used to receive human motion data, perform human motion recognition on the extracted motion amplitude features using an SVM classification algorithm, and send the human motion recognition results to the second server 300;

[0184] The second server 300 establishes a communication connection with the first server 200, and the second server 300 receives the human motion recognition result and determines whether the vitality of the landscape space in the area is increased or decreased through comparison;

[0185] The second server 300 establishes a communication connection with the terminal 400. The terminal 400 is used to display the vitality status of each area in the landscape space and can retrieve the spatial vitality status of each area in different time periods in the morning, noon and afternoon according to needs.

[0186] The terminal 400 is any one of a PC, a notebook, a tablet or a mobile phone.

[0187] Camera 100 is a non-contact motion sensing device called Kinect, recently launched by Microsoft. It primarily consists of an infrared device, an RGB camera, a depth camera, and a microphone, and features full-body skeletal tracking and behavioral trajectory capture. Kinect uses a series of sensors to collect data from the human body, employs machine learning methods to obtain skeletal information, extracts features from this skeletal information, and finally uses a Support Vector Machine (SVM) classification algorithm based on statistical learning theory to identify human motions from the extracted feature vectors.

[0188] The method of human action recognition includes:

[0189] S21, obtaining motion amplitude characteristics of each joint point of the subject's human body within at least a certain time period, the motion amplitude characteristics being determined by the specific motion of each joint point of the subject's human body, constructing a three-dimensional human motion model based on the motion of the human joint points, and inputting the motion amplitude characteristics into the three-dimensional human motion model to construct a dynamic process of each joint point;

[0190] S22, extracting human joint points with motion amplitude characteristics within the time period, and extracting the human joint point movements occurring within the time period one by one, such as feature 1, feature 2, feature 3 ... feature n corresponding to the human joint points;

[0191] S23, analyzing the correlation between the joint points of the human body with the motion amplitude characteristics, inputting the above-mentioned features 1, 2, 3, ..., n into the three-dimensional human motion model to construct a dynamic process, and obtaining the degree of correlation between the motions of the subject's limbs from the dynamic process, thereby determining the combination of the human joint point motions and the specific motion of each limb after the combination;

[0192] S24. Extract motion data features based on the motion amplitude features of the human joint points. The motion data features are the specific motions of the human limbs after fusing features 1, 2, 3…n. The specific steps of motion data feature extraction are: constructing human motion vectors. The angles between these vectors are used to represent the rotation of the limbs, and the relative modulus ratio is used to represent the relative displacement of the limbs. The two types of parameters generate fusion feature extraction to describe the motion, and generate a fusion feature space vector for SVM to perform motion classification and recognition. The feature extraction process of motion data is as follows: Figure 11 shown.

[0193] Assume that two points in three-dimensional space are known to be p i (xi ,y i ,z i ),p j (x j ,y j ,z j ), then the distance d between them x,y,z As shown in formula (1):

[0194]

[0195] If the angle of the joint needs to be solved, the coordinates of at least three joint points are required in three-dimensional space, assuming they are pi, pj and pk. The pairwise distances are calculated and assumed to be dij, dik and djk. Then, the cosine theorem can be used to obtain the angle between each joint point as shown in formula (2).

[0196]

[0197] This embodiment extracts the angle and modulus ratio of the human body motion vector as fusion feature values, and generates a fusion feature space vector for use in SVM classification.

[0198] The basic idea of SVM is: assuming that there are many sample points in the m-dimensional space, if there is a way to find the optimal hyperplane of m-1 dimensions, which can just divide the sample points in the space evenly on both sides of the optimal hyperplane, then this m-1 hyperplane can be used to classify random sample points.

[0199] Feature spaces can be linearly separable or linearly inseparable, both of which can be classified by SVM. In the linearly separable case, SVM can be used to directly find the optimal classification hyperplane for the features. However, in linearly inseparable feature spaces, an additional step is required before finding the optimal classification hyperplane. SVM uses kernel functions to transform the feature space into a higher-dimensional space. This is because in higher-dimensional spaces, inseparable problems become separable. Once separable, classification can then be performed using the linearly separable steps.

[0200] The vector machine in the linearly separable feature space is shown in formula (3):

[0201]

[0202] Where: i = 1, 2, 3, .... l; w, b refer to the classification plane coefficients; x i Refers to the i-th training sample, y i Refers to the category to which the i-th training sample belongs; a i It means that the i-th training sample corresponds to the Lagrange coefficient, and a i ≥0;xs represents a specific support vector, y s Represents x s Category.

[0203] The vector machine in a linearly inseparable feature space is shown in formula (4), where ζi is a slack variable introduced to control outliers. In this way, by making some samples less than 1 away from the classification plane, some classification accuracy is sacrificed, allowing the algorithm to recognize a wider range of classifications.

[0204]

[0205] Kinect can obtain depth images and thus obtain information about human skeleton points. SVM also has excellent classification capabilities, especially in linear and nonlinear high-dimensional feature spaces, effectively avoiding the curse of dimensionality. The specific steps are to first use the Kinect sensor to capture human motion to generate a depth image, then process it to create a 3D human model. The angle and modulus ratio of the motion vector are extracted as eigenvalues, and finally, the SVM is used to classify and recognize human motion based on these eigenvalues.

[0206] S25. Determine a human motion feature vector based on the correlation between the joint points and the data features, and determine the human motion feature vector based on the correlation between the movements of the subject's limbs and the specific movements. That is, determine the specific movements made by the subject's limbs through the variables obtained by fusing the feature space.

[0207] S26. Construct a human motion feature vector set based on the accumulation of human feature vectors. The human motion feature vector set obtained based on the accumulation of human feature vectors is the specific type of human motion, such as squatting to smell, walking, running, etc.

[0208] The method of human action recognition uses the SVM classification algorithm to perform human action recognition on the extracted feature vectors.

[0209] In this embodiment, the specific steps of the landscape space vitality monitoring system are as follows:

[0210] S1, non-contact somatosensory device Kinect is used to collect human motion data;

[0211] S2, upload human motion data;

[0212] S3, receiving human motion data;

[0213] S4, using the SVM classification algorithm to perform human motion recognition on the extracted motion amplitude features;

[0214] S5. Sending the human motion recognition result to the second server;

[0215] S6, receiving the human motion recognition result and making a preliminary judgment on the human motion type;

[0216] S7, obtaining the spatial vitality value V1 of the area during the period based on the preliminarily determined human motion type;

[0217] S8, according to the human body movement type in the area in the next time period T2, T3...Tn, the spatial vitality value V2, V3...Vn in the time period is obtained;

[0218] S9, comparing the spatial vitality values V1, V2, V3...Vn accumulated in different time periods within the area;

[0219] S10. Evaluate whether the vitality of the landscape space in the area has increased or decreased based on the comparison results;

[0220] S11. Display the vitality status of each area in the landscape space.

[0221] Through this embodiment, the recognition of human movements is used to provide a monitoring basis for the vitality of the landscape space. The human movement recognition method can be used to quickly determine the movement type of tourists. At the same time, the SVM classification algorithm has excellent classification capabilities and can have a higher classification ability when facing linear and nonlinear high-dimensional feature spaces, effectively avoiding the dimensionality disaster, thereby improving the accuracy of human movement recognition, and being able to quickly make a basis for judging the movement type, thereby improving the accuracy of monitoring the space vitality status.

[0222] The landscape space vitality monitoring system of this embodiment is used to implement the corresponding landscape space vitality monitoring methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0223] The present invention can quickly identify tourists' body movements by online monitoring of the landscape space, and determine the popularity of the vegetation layout in the space based on the frequency of the body movements. At the same time, it provides a reference basis for the individual popularity of each vegetation and the growth trend of each vegetation. For example, if there are fewer smelling actions, it may be because the fragrant flowers need to be maintained, or if there are fewer running behaviors, it may be because the weeds have grown more. Therefore, the present invention can effectively monitor the vitality state of the landscape space, and at the same time avoid the phenomenon of inadequate supervision and the occurrence of idle space for waste gas.

[0224] The present invention monitors the vitality of the landscape space through human motion recognition, can monitor the spatial vitality of the entire landscape space in real time online during the day, and can make timely and effective evaluations based on the increase and decrease of spatial vitality, providing a basis for managers' scene arrangement and other measures. At the same time, it can make reminders when the spatial vitality continues to decrease, so that corresponding treatment measures can be made in a timely and rapid manner. After the landscape space is rearranged, it is verified through monitoring whether it is attractive enough to personnel and whether the update is effective. Therefore, the purpose of effectively monitoring the vitality of the landscape space and providing managers with a basis for judging the rearrangement of the scene is achieved, which can improve the spatial vitality of the landscape space, increase tourists' popularity of the landscape space, and improve tourists' experience and comfort in the landscape space.

[0225] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to in detail. For the above embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiments.

[0226] The above embodiments provide a detailed introduction to the present invention. In this embodiment, specific examples are used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for monitoring landscape space vitality based on human motion recognition, characterized in that: This monitoring method uses the recognition of human motion in the landscape space as a basis to monitor the vitality of the landscape space. The following processes are included: Obtaining a set of human motion feature vectors in at least a certain area of a landscape space within a time period T1; Analyze the human motion feature vector set and make a preliminary judgment on the human motion type; The spatial vitality value V1 of the area in the time period is obtained based on the preliminary judgment of the human body movement type; According to the human body movement type in the area in the next time period T2, T3...Tn, the spatial vitality value V2, V3...Vn in the time period is obtained; the spatial vitality is measured by the vitality value; Compare the spatial vitality values V1, V2, V3…Vn accumulated in different time periods within the area; Based on the comparison results, it is determined whether the vitality of the landscape space in the area has been improved; If the vitality of the landscape space increases, the specific landscape where the human body moves most frequently in the area is analyzed, and the specific orientation of the specific landscape and the specific characteristic vector of the human body movement are listed; If the vitality of the landscape space decreases, the area is marked as a key supervision area, and the reasons for the reduction of the specific characteristic vectors of the human body in this area are analyzed; After obtaining the human feature vector set, the area within the landscape space can be divided according to the actual situation on site. The total area is divided into multiple independent sub-areas, where a certain area corresponds to a sub-area. Camera equipment and infrared thermal imagers are deployed in the area to capture human joint movements, and a corresponding three-dimensional human movement model is established through the server. When analyzing the human body motion feature vector set, the specific movements made by the human body are synchronously analyzed based on the accumulation and frequency of occurrence of individual feature vectors in the human body feature vector set, combined with the human body joint points, to make corresponding judgments on the human body motion type.

2. The method for monitoring landscape space activity based on human motion recognition according to claim 1, characterized in that: The method of human action recognition includes the following processes: Obtaining motion amplitude characteristics of each joint point of the subject's body within at least a certain time period; Extracting human joint points with motion amplitude characteristics within the time period; Analyze the correlation between various joints of the human body with motion amplitude characteristics; Extract motion data features based on the motion amplitude features of human joint points; Determine the human motion feature vector based on the correlation between each joint point and data characteristics; A human motion feature vector set is constructed based on the accumulation of human feature vectors.

3. The method for monitoring landscape space activity based on human motion recognition according to claim 2, characterized in that: include: The human joint point movements that occur within this time period are extracted one by one, and the features 1, 2, 3, ..., n corresponding to the human joint points are obtained; The above-mentioned features 1, 2, 3…n are input into the three-dimensional model of human body motion to construct a dynamic process. The dynamic process is used to derive the degree of correlation between the motions of the various limbs of the target human body, thereby determining the combination of the motions of the human body joints and the specific motions of the limbs after the combination.

4. The method for monitoring landscape space activity based on human motion recognition according to claim 2, characterized in that: The motion data feature is the specific motion of each limb of the human body obtained by fusing feature 1, feature 2, feature 3...feature n.

5. The method for monitoring landscape space activity based on human motion recognition according to claim 1, characterized in that: In obtaining the human feature vector set, the human action feature vector set is a set of actions performed by human joints in the time period T1, which includes each single feature vector performed by the human body and is accumulated in the time period T1.

6. The method for monitoring landscape space activity based on human motion recognition according to claim 1, characterized in that: In the comparison of spatial vitality values, the comparison of spatial vitality values needs to be carried out with the values generated in different time periods within a specific area. In the three cumulative time periods of T1, T2 and T3, the corresponding spatial vitality values are V1, V2 and V3 respectively. At this time, V1 and V2 are compared to determine whether the spatial vitality has increased or decreased, and V2 and V3 are compared to determine whether the spatial vitality has increased or decreased.

7. A landscape space activity monitoring device for implementing the landscape space activity monitoring method based on human motion recognition as described in any one of claims 1 to 6, characterized in that: include: An acquisition module (110), the acquisition module (110) being used to acquire a human motion feature vector set in at least a certain area of a landscape space within a time period T1; An analysis and judgment module (120), the analysis and judgment module (120) is used to analyze the human body motion feature vector set and make a preliminary judgment on the human body motion type; A first spatial vitality value module (130), the first spatial vitality value module (130) is used to obtain a spatial vitality value V1 of the area within the time period according to the preliminarily determined human motion type; A second space vitality value module (140), the second space vitality value module (140) is used to obtain the space vitality values V2, V3...Vn in the next time period T2, T3...Tn according to the human body movement type in the area; A comparison module (150), the comparison module (150) is used to compare the spatial vitality values V1, V2, V3...Vn accumulated in different time periods within the area; An evaluation module (160) is used to evaluate whether the vitality of the landscape space in the area is improved based on the comparison results.

Images