An indoor intelligent monitoring method for pressure-sensitive floor based on edge machine learning

Abstract

The application discloses an indoor intelligent monitoring method for pressure-sensitive floor based on edge machine learning, and builds a monitoring system comprising a pressure-sensitive module, a data processing module, a communication module and an external interface. The pressure distribution in the room can be acquired in real time through the pressure-sensitive module. For the analysis process of the pressure, the edge machine learning model in the data processing module is used to directly perform data processing and instruction generation, and the communication module is used to transmit to the external interface, so that the control of external devices is realized. The method combines the pressure-sensitive floor and the edge machine learning, can improve the system response speed, does not need to interact with the server, can save the consumption of maintaining the server, the whole information processing process is completed by a low-power controller, and the basic requirements of real time, safety, privacy protection and the like are met.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of smart home, and relates to edge machine learning and pressure sensing, in particular to an indoor intelligent monitoring method for pressure-sensitive floor based on edge machine learning. BACKGROUND

[0002] With the rapid development of electronic technology, the continuous improvement of people's living standards and the wide application of intelligent electronic technology in life, the market size of the Internet of Things is also getting larger and larger. Edge machine learning is one of the technologies that people pay most attention to since the Internet of Things. Through this technology, intelligent devices can use machines and deep learning algorithms to process data locally, thereby reducing the dependence on cloud networks. Smart home has become the latest direction of the development of future home decoration trends. A variety of sensors are the basis of smart home systems, and pressure sensors are also a common and commonly used sensor in smart home systems. However, there is still a lack of research on large-area laying of pressure sensors as smart floor.

[0003] Prior art 1 is a distributed floor based on high-resolution pressure sensors designed by Tokyo University in Japan. A floor is composed of a 64x64 array of sensing units. When the pressure measured by the sensing unit exceeds a certain threshold, the output voltage will change from low to high. The size of each sensing unit is 500 square millimeters. Sixteen identical tiles are used to lay a 2m x 2m test environment, which includes a total of 4096 pressure switches. The team successfully determined the position of a person and a four-wheeled vehicle on a 2m x 2m test environment. In prior art 1, the sensor floor is designed as a modular structure, which makes it convenient to apply to real rooms with regular shapes, allowing unrestricted measurement of the positions of people and objects in the room.

[0004] Prior art 2 is the Gravity Space floor developed by the Hasso Plattner Institute in Potsdam, Germany. This floor includes a 6.4cm thick glass plate, as well as infrared cameras and rubber-wrapped pressure-sensitive films below, which can sense the contact and pressure of objects with the floor, and then determine the user's position and posture through processing algorithms, while analyzing whether a user has fallen or an object has fallen. The specific working principle is that when an object comes into contact with the floor, it will trigger the deflection of light between the infrared light and the pressure-sensitive film. Through the infrared camera, a footprint image is captured and created in real time, and through the analysis of the processing algorithm, the state of the human body or object can be obtained.

[0005] The prior art 1 has good robustness and positioning accuracy, but cannot accurately measure the pressure value due to the use of a pressure switch, and has weak functionality; and the prior art 2 is more functional, but has limited weighing capacity because an infrared camera is arranged below the glass plate, is complex to manufacture and has high cost, is easily damaged during use, and has poor practicality, and is not conducive to large-area paving.

[0006] In summary, the research and application of intelligent floor in the prior art are not only few in number, but also not combined with edge machine learning technology, and most of them are still in the laboratory stage, and no solution has appeared that can be popularized in life on a large scale. SUMMARY

[0007] In view of the deficiencies of the prior art, the present application proposes an indoor intelligent monitoring method for pressure-sensitive floor based on edge machine learning, which collects the surface pressure change condition through the laid pressure-sensitive floor, uses a local microcontroller to perform edge computing to complete data analysis and instruction transmission, controls various smart home devices through an external interface, has strong compatibility, and can speed up the response speed and reduce the burden on the cloud.

[0008] An indoor intelligent monitoring method for pressure-sensitive floor based on edge machine learning, comprising the following steps:

[0009] Step 1, deploy an indoor intelligent monitoring system for pressure-sensitive floor, comprising a pressure sensing module, a data processing module, a communication module and an external interface. The pressure sensing module uses a plurality of identical flexible film pressure sensors and is laid on the ground to be monitored. The data processing module is used to collect and analyze the pressure data output by the pressure sensing module and output control information. The communication module sends the control information output by the data processing module to the corresponding smart home device through the external interface.

[0010] Step 2, after the indoor intelligent monitoring system for pressure-sensitive floor is deployed, the pressure sensing module is used to collect the daily activity pressure value, and the data processing module is used to sample the pressure data p and the action generation time t to construct an action vector

[0011]

[0012] wherein p ij is the pressure data output by the flexible film pressure sensor in the ith row and jth column of the pressure sensing module. A plurality of action vectors corresponding to actions are collected to construct a training data set.

[0013] Preferably, the daily activities include standing still, falling down, walking at different speeds, sitting or lying on different furniture, etc.

[0014] Step 3, constructing a machine learning model on the computer side, inputting the training data set obtained in step 2 into the machine learning model, and outputting a vector as a control instruction.

[0015]

[0016] wherein i rm is the control instruction of the mth smart furniture device.

[0017] As preferred, the machine learning model is a convolutional neural network (CNN) or an artificial neural network (ANN).

[0018] Step 4, performing an exclusive or calculation between the output vector of the model and the expected output vector to calculate the accuracy A of the machine learning model:

[0019]

[0020] T i = i ri ⊙i ei

[0021] wherein b is the maximum value of the binary bits of T i , and m is the number of instructions.

[0022] When the accuracy of the machine learning model is greater than the required accuracy θ1, the machine learning model is transplanted into the data processing module; otherwise, the machine learning model is retrained until the accuracy requirement is met.

[0023] Step 5, inputting the training data obtained in step 2 into the machine learning model transplanted on the data processing module, calculating the accuracy thereof, and when the accuracy of the transplanted machine learning model is greater than the required accuracy θ2, completing the training; otherwise, returning to step 4 to retrain the machine learning model on the computer side until the accuracy requirement is met.

[0024] Step 6, sampling the output data of the pressure sensing module using the data processing module to obtain an action vector which is then processed by the machine learning model trained in step 5, and the output vector of the machine learning model is transmitted to the external interface through the communication circuit, used for controlling other smart home devices in the environment, and realizing edge machine learning-based indoor monitoring of pressure-sensitive floors.

[0025] Compared with the background art, the present application has the following beneficial effects:

[0026] 1、Through the pressure sensing module and the data processing module, the indoor pressure distribution can be sensed and displayed in real time, and various home intelligent controls are performed according to the collected pressure distribution, the edge machine learning is combined with the pressure floor intelligent system, the data processing process does not need to pass through the cloud, the processing efficiency is greatly improved, the load of the cloud is reduced, a faster response speed can be provided, and the production and maintenance cost is low and the robustness is high.

[0027] 2、By increasing the number of sensor units in the pressure sensing module, the monitoring range can be expanded, and the spatial resolution can remain unchanged, and the connection with existing smart home devices is realized through the external interface, and the coverage area and the type of control peripherals are both highly expandable. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 It is a flow chart of an indoor intelligent monitoring method of a pressure sensing floor based on edge machine learning.

[0029] Figure 2 It is a block diagram of a monitoring system attached in an embodiment.

[0030] Figure 3 It is a circuit block diagram of a data processing module and a communication module in an embodiment.

[0031] Figure 4 It is a multi-channel analog switch gating schematic diagram in an embodiment.

[0032] Figure 5 It is a machine learning model transplantation process.

[0033] Figure 6 It is an application program architecture in an embodiment. DETAILED DESCRIPTION

[0034] The present application will be further explained and described below with reference to the accompanying drawings; it should be understood that the embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the protection scope of the present application.

[0035] As shown in Figure 1 , an indoor intelligent monitoring method of a pressure sensing floor based on edge machine learning, comprising the following steps:

[0036] Step 1, deploy the monitoring system as shown in Figure 2 , including a pressure sensing module, a data processing module, a communication module and an external interface.

[0037] The pressure sensing module includes 16 identical flexible film pressure sensors, which are laid in a 4x4 order on the ground to be monitored, and each flexible film pressure sensor has a measurement range of 0-100kg. As shown in Figure 3As shown, the pressure sensing module converts the change of pressure into the change of resistance.

[0038] The data processing module scans the output of the pressure sensing module at a fixed sampling frequency, converts the change of resistance into a voltage signal through the resistance-voltage conversion circuit, and outputs control information after analysis using a machine learning model. In this embodiment, an STM32F746G processor chip is used to implement the data processing module. In order to simplify the circuit, a multi-channel analog switch is also provided between the pressure sensing module and the data processing module, as shown in Figure 4 As shown, the STM32F746G processor chip is used to control the opening or closing of the multi-channel analog switch through the GPIO, ensuring that only one flexible thin film pressure sensor is connected to the circuit during each sampling process. High-speed scanning switching of the pressure sensing module can be achieved through I / O control of the GPIO.

[0039] The communication module uses a WF-EPS8266 WiFi module to send the control information output by the data processing module to the corresponding smart home device through the external interface.

[0040] In this embodiment, a server end is also included. The server end can receive control information sent by the communication module when the system is online, record and store data, and update and iterate the machine learning model. The server end does not participate in the instruction transmission process, so it can still work normally when the system is offline.

[0041] Step 2, carry out daily activities in the space where the monitoring system is deployed, use the pressure sensing module to collect pressure values, and obtain pressure data P through the data processing module sampling:

[0042]

[0043] where p ij represents the pressure data output by the flexible thin film pressure sensor at the ith row and jth column.

[0044] The richness of the sample data and the framework of the model itself will affect the recognition effect. In order to ensure the usability and performance of the model, various situations need to be considered when collecting sample data. The diversity of data can effectively improve the stability of the recognition model performance effect. The more abundant the data is, the more situations the model can represent and use, the stronger the performance of the model, and the higher the accuracy of the model. Therefore, it is necessary to collect pressure data P and action generation time t in different states of different personnel standing still, falling, walking at different speeds, sitting and lying on different furniture, etc. Then arrange the elements in matrix P in row order to construct the action vector

[0045]

[0046] Step 3, build a machine learning model based on convolutional neural network or artificial neural network on the computer side, input the training data set obtained in step 2 into the machine learning model, and output a vector as a control instruction.

[0047]

[0048] Wherein, i1-i4 are all 2-bit binary numbers, which are control instructions of multi-stage light adjustment LED lamp, alarm, LCD display screen and intelligent automatic curtain respectively. The corresponding peripheral behaviors of the control instructions are shown in Table 1:

[0049]

[0050] When no pressure is detected or during the day, i1 outputs 00, and the LED does not light up. When pressure is detected at night, i1 outputs 01, and the LED emits weak light so that the user can see the environment. When pressure is detected at night, i1 outputs 10, and the LED normally emits light. When i1 outputs 11, it is a self-defined function, which can provide adjustment space for subsequent model iteration update.

[0051] When i2 outputs 00, it means no abnormality. When i2 outputs 01, it means that an abnormal state occurs in the room, such as an old person falling down or a long-term pressure in a certain place, which may indicate that someone is in a coma state, and an alarm is issued. When i2 outputs 10 or 11, it is a self-defined function.

[0052] When i3 outputs 00, it is detected as user A. When i3 outputs 01, it is detected as user B. When i3 outputs 10, it is detected as user C. When i3 outputs 11, it is detected as user D, and the LCD display screen displays the specific information specified by the user.

[0053] When i4 outputs 00, the curtain is automatically rolled up. When i4 outputs 01, the curtain is automatically unfolded. When i4 outputs 10 or 11, it is a user-defined state.

[0054] Step 4, the output vector of the model is calculated with the expected output vector to calculate the accuracy A of the machine learning model on the computer side c :

[0055]

[0056] T1=i1⊙i e1

[0057] T2=i2⊙ie2

[0058] T3 = i3 o i e3

[0059] T4 = i4 o i e4

[0060] When A c > 95%, the machine learning model is transplanted into the MCU. Otherwise, the machine learning model is retrained.

[0061] As Figure 5 shown, the TensorFlow Lite converter is used to convert the machine learning model trained on the computer side, and the model is optimized-quantized to generate a TensorFlow Lite file with a file format of.flite. The weights and biases in the TensorFlow File are generally 32-bit floating-point numbers, and quantization allows the precision of these weights and biases to be reduced so that they can be stored as 8-bit integers. Finally, it is deployed on the MCU. The calculation formula of the quantization process is as follows:

[0062]

[0063] where R represents the true floating-point value, Q represents the quantized fixed-point value, Z represents the quantized fixed-point value corresponding to the floating-point value, and S is the smallest scale that can be represented after fixed-point quantization. The evaluation formulas of S and Z are as follows:

[0064]

[0065] Z = Q max -R min ÷ S

[0066] R max represents the maximum floating-point value, R max = 30; R min represents the minimum floating-point value, R min = -10; Q max represents the maximum fixed-point value, Q max = 256; Q min represents the minimum fixed-point value, Q min = 0.

[0067] Step 5, use the training data set obtained in step 2 to train the machine learning model transplanted to the data processing module, and calculate its accuracy. When the accuracy of the transplanted machine learning model is greater than the required accuracy of 90%, the training is completed. Otherwise, return to step 4 and retrain the machine learning model on the computer side.

[0068] Step 6, as Figure 6As shown, the output data of the pressure sensing module is sampled using the data processing module to obtain a motion vector The output vector of the machine learning model is processed by the machine learning model trained in step 5 The output vector of the machine learning model is processed by the machine learning model trained in step 5

[0069] The method can obtain the pressure distribution in the room in real time through the pressure sensing module. For the analysis process of the pressure, an edge machine learning model is used to improve the response speed. The method can send instructions to the service, and the server delivers the instructions to each device. It can also work offline without information interaction with the server, saving the consumption of maintaining the server. The entire information processing process of the method is completed by a low-power controller, meeting the basic requirements of real-time, safety, privacy protection, etc.

Claims

1. A method for indoor intelligent monitoring of pressure-sensitive floors based on edge machine learning, characterized by: The method comprises the following steps: Step 1, deploying a monitoring system, including a pressure sensing module, a data processing module, a communication module and an external interface; wherein the pressure sensing module is laid on the ground to be monitored to output pressure data; the data processing module collects and analyzes the pressure data output by the pressure sensing module and outputs control instructions; The communication module sends the control instructions output by the data processing module to the smart home equipment connected to the external interface; Step 2, the user carries out daily activities in the space where the monitoring system is deployed, the pressure sensing module collects pressure values, and the data processing module samples the pressure data p and the action generation time t to construct an action vector wherein p cd is the pressure data output by the flexible film pressure sensor in the cth row and the dth column in the pressure sensing module; action vectors corresponding to different actions are collected to construct a training data set; Step 3, constructing a machine learning model on the computer side, inputting the training data set obtained in step 2 into the machine learning model, and outputting a vector as control instructions; Wherein, i rm is the control instruction of the mth smart home device; Step 4, the output vector of the model with the desired output vector performing an XOR calculation, accuracy A of the computer machine learning model: T i = i ri o i ei where b is the number of binary bits of T i the maximum value of the number of binary bits of T When the accuracy of the machine learning model is greater than the required accuracy θ1, the machine learning model is transplanted to the data processing module; otherwise, the machine learning model is retrained; Step 5, training the machine learning model transplanted to the data processing module using the training data set obtained in step 2, and calculating its accuracy; when the accuracy of the transplanted machine learning model is greater than the required accuracy θ2, the training is completed; Otherwise, return to step 4 and retrain the machine learning model on the computer side; Step 6, sampling the output data of the pressure module using the data processing module to obtain a motion vector After the machine learning model trained by step 5 is processed, the output vector of the machine learning model is obtained Transmitted to the external interface through the communication circuit, used for controlling other smart home devices in the environment, realizing edge machine learning-based indoor intelligent monitoring of pressure-sensitive floor.

2. The edge-based machine learning method for intelligent monitoring of a pressure-sensitive floor according to claim 1, wherein: The pressure sensing module comprises a plurality of identical flexible film pressure sensors, and the measurement range of each flexible film pressure sensor is 0-100 kg.

3. The edge-based machine learning method for intelligent monitoring of a pressure-sensitive floor according to claim 1, wherein: The daily activities include standing still, falling down, walking at different speeds, and sitting and lying on different furniture.

4. The edge-based machine learning method for intelligent monitoring of a pressure-sensitive floor according to claim 1, wherein: The machine learning model is a convolutional neural network or an artificial neural network.

5. The edge-based machine learning-based intelligent monitoring method for pressure-sensitive floor in a room according to claim 2, wherein: The monitoring system further comprises a multi-channel analog switch, and the data processing module selects the output of one flexible film pressure sensor in the pressure sensing module for sampling at one time through the multi-channel analog switch.

6. The edge-based machine learning-based intelligent indoor monitoring method for pressure-sensitive floor according to claim 1, wherein: The monitoring system further comprises a server end, which receives the control information sent by the communication module under the networked state of the system, records and stores the data, and updates and iterates the machine learning model.

7. The edge-based machine learning method for intelligent monitoring of a pressure-sensitive floor according to claim 1, wherein: The process of transplanting the machine learning model from the computer side to the data processing module is as follows: The TensorFlow Lite converter is used to convert the machine learning model trained on the computer side, and the model is optimized-quantized to generate a TensorFlow Lite File with a file format of.flite. The weights and biases in the TensorFlow File are 32-bit floating-point numbers. The quantization process reduces the precision of the weights and biases, and stores them as 8-bit integers. Finally, the quantized TensorFlow File is transplanted to the data processing module. The calculation formula of the quantization process is as follows: Wherein, R represents the real floating-point value, Q represents the quantized fixed-point value, Z represents the quantized fixed-point value corresponding to the floating-point value, and S represents the minimum scale that can be represented after fixed-point quantization: Z = Q max - R min ÷ S R max = 30, representing the largest floating point value; R min = -10, representing the smallest floating point value; Q max = 256, representing the largest fixed point value; Q min = 0, representing the smallest fixed point value.

8. The edge-based machine learning-based intelligent indoor monitoring method for pressure-sensitive floor according to claim 1, wherein: θ1=95%, θ2=90%.

Images