Wearable man-machine gesture recognition interaction system for intelligent construction site

Abstract

The invention discloses a wearable man-machine gesture recognition interaction system for an intelligent construction site, belongs to the technical field of intelligent buildings, and is used for solving the problem that existing wearable man-machine gesture recognition equipment is prone to being affected by factors such as construction environment vibration, electromagnetic interference and wearing deviation in a construction site environment, and is not prone to being damaged. The technical problems that gesture recognition is not stable or the misjudgment rate is high, and fatigue monitoring support for workers is insufficient are solved. The method comprises the steps that a mechanical-pressure sensing module is used for collecting a pressure sensing signal of the skin surface; the micro-electric signal electrode module is installed on the surface of the flexible air chamber assembly and synchronously collects micro-electric signals of muscles in the skin surface; the microcontroller module is used for carrying out signal synchronous sampling processing on the pressure sensing signal and the micro-electric signal and carrying out preprocessing and filtering on the collected multi-mode signal; and the signal processing and gesture recognition module is used for mapping the multi-modal signal into corresponding gesture information and corresponding muscle state information.

Description

Technical Field

[0001] This application relates to the field of smart buildings, and particularly to a wearable human-machine gesture recognition and interaction system for smart construction sites. Background Art

[0002] With the rise of the concept of smart construction sites, more and more construction site management and construction processes begin to apply automated and information-based means to improve operation efficiency and ensure construction safety. Existing human-machine interaction methods mostly rely on voice control or touch-based operation interfaces. However, in construction environments with high noise, high dust, or inconvenient worker operations, the accuracy of voice recognition decreases, touch screen operations are inflexible or easily contaminated, making it difficult to meet actual needs.

[0003] On the other hand, traditional wearable devices based on single-modal sensing (such as only using a certain type of muscle signal or mechanical signal) can capture human motion intentions, but are easily affected by factors such as construction environment vibration, electromagnetic interference, and wearing offset, resulting in unstable gesture recognition or a high misjudgment rate, and insufficient support for worker fatigue monitoring. Summary of the Invention

[0004] Embodiments of this application provide a wearable human-machine gesture recognition and interaction system for smart construction sites to solve the following technical problems: Existing wearable human-machine gesture recognition devices are easily affected by factors such as construction environment vibration, electromagnetic interference, and wearing offset in the construction site environment, resulting in unstable gesture recognition or a high misjudgment rate, and insufficient support for worker fatigue monitoring.

[0005] Embodiments of this application adopt the following technical solutions:

[0006] On the one hand, embodiments of this application provide a wearable human-machine gesture recognition and interaction system for smart construction sites, including: a flexible air chamber component, which is used to fit the body surface of the user and collect pressure perception signals on the skin surface through the mechanical-pressure perception module in the flexible air chamber component; a microelectrical signal electrode module, which is installed on the surface of the flexible air chamber component and synchronously collects the microelectrical signals of the muscles in the skin surface; a microcontroller module, which is used to perform synchronous sampling processing on the pressure perception signals and the microelectrical signals, and preprocess and filter the collected multi-modal signals; a signal processing and gesture recognition module, which is used to map the multi-modal signals into corresponding gesture information and corresponding muscle state information; a communication module, which is used to send the gesture information and the muscle state information to the smart construction site management platform.

[0007] In the embodiments of the present application, multi-modal fusion significantly improves the accuracy of gesture recognition and the anti-interference ability in the construction site environment. The air chamber is made of a flexible and stretchable material. Compared with traditional arm rings or fixed electrodes, the flexible air chamber can evenly distribute pressure, thus maintaining a high level of comfort during long-term wearing. Through algorithms, real-time interpretation of multi-modal signals is carried out. When predefined gestures (such as making a fist, raising the wrist, rotating the arm, etc.) are recognized, the system immediately sends control instructions to construction machinery or robots in the construction site, reducing the dependence on physical buttons and manual operations; by analyzing the fusion characteristics of microelectrical signals and mechanical-pressure sensing signals, the muscle fatigue degree of workers can be monitored and early warnings can be given. For example, when the fatigue index exceeds the threshold, a reminder is sent to the construction site management platform to help managers with personnel allocation and risk prevention and control. At the same time, the system supports wireless communication protocols to transmit gesture instructions, fatigue status, or abnormal action recognition information to the intelligent construction site management platform in real time. It can be integrated with construction machinery and equipment (such as tower cranes, excavators, robotic arms, etc.) or other sensors at the construction site to form multi-source data interaction, further improving construction safety and efficiency.

[0008] In a feasible implementation manner, the signal acquisition positions of the microelectrical signal electrode module are made to coincide with the signal acquisition positions of the mechanical-pressure sensing module in the flexible air chamber assembly, and based on the same timing characteristics, the microelectrical signals and the pressure sensing signals at the same signal acquisition position are synchronously acquired.

[0009] In a feasible implementation manner, the signal processing and gesture recognition module further includes: performing fusion generation processing on the corresponding multi-modal signals of the acquired pressure sensing signals and microelectrical signals; inputting the multi-modal signals into a preset gesture recognition algorithm, and through the gesture recognition algorithm, extracting and classifying the gesture information in the multi-modal signals for gesture features and gesture types to obtain gesture instruction information; calculating the current muscle fatigue degree of the muscle state information in the multi-modal signals through a preset muscle fatigue degree model to obtain a muscle fatigue degree index; recognizing the gesture instruction information and performing threshold judgment on the muscle fatigue degree index to respectively obtain a predetermined gesture instruction and a muscle fatigue degree early warning level.

[0010] In a feasible implementation manner, if the intelligent construction site management platform recognizes the predetermined gesture instruction, then through the intelligent construction site management platform, the control instruction corresponding to the predetermined gesture instruction is sent to the corresponding operating equipment in the intelligent construction site, so that the operating equipment completes contactless control; wherein, the predetermined gesture instruction at least includes: making a fist, raising the wrist, and rotating the arm; if the muscle fatigue degree early warning level is a high-risk early warning level, then through the intelligent construction site management platform, rest information is sent to and reminded to the user, and personnel allocation management is re-performed.

[0011] In a feasible implementation manner, it is characterized in that the micro - electrical signal electrode module is deployed on the surface of the air chamber in the flexible air chamber assembly for collecting the micro - electrical signals under muscle activities; each air chamber surface has three such micro - electrical signal electrode modules, and the micro - electrical signal electrode modules are fixed in a pre - fabricated circular housing.

[0012] In a feasible implementation manner, one such mechanical - pressure sensing module is deployed on the side of each air chamber; the numerical values of the signal channels, sampling frequencies, and resolutions of the mechanical - pressure sensing module and the micro - electrical signal electrode module are all the same; the air chamber, the micro - electrical signal electrode module, and the mechanical - pressure sensing module are combined into a combined sensing unit; several such combined sensing units are placed side by side to form a limb detection band.

[0013] In a feasible implementation manner, the micro - controller module further includes: removing the 50Hz power line noise in the micro - electrical signal through a notch filter; dividing the filtering range of the micro - electrical signal to obtain a physiological signal range; wherein, the physiological signal range includes: a first cut - off frequency, a second cut - off frequency, and a third cut - off frequency; the frequency range of the first cut - off frequency is 20 - 50Hz, the frequency range of the second cut - off frequency is 20 - 150Hz, and the frequency range of the third cut - off frequency is 20 - 500Hz; if the physiological signal range is the first cut - off frequency, the micro - electrical signal is filtered through a third - order Butterworth filter; if the physiological signal range is the second cut - off frequency, the micro - electrical signal is filtered through a high - pass and a low - pass filter; if the physiological signal range is the third cut - off frequency, the micro - electrical signal is filtered through a fourth - order Butterworth filter.

[0014] In a feasible implementation manner, based on the variation relationship between muscle force and pressure signal, a muscle force - pressure relationship model is established, specifically including: according to to obtain the total muscle force in the muscle force - pressure relationship model; wherein, F a is the active force generated by the contractile element of muscle fibers, F p is the passive force generated by the elastic element of muscle, f a (Δl) and f p (Δl) are respectively the normalized active function and passive force function related to the change in muscle length, is the maximum isometric muscle force, Δl is the change in muscle length, and a is the muscle activation level.

[0015] In a feasible implementation manner, according to Obtain the actual output muscle force F in the muscle force-pressure relationship model mv ; where, ΔA c is the change in the cross-sectional area of the muscle; the screw-in angle θ represents the angle formed between the tendon and the muscle fiber; is the maximum isometric muscle force; f a is the normalized active function related to the muscle length change; f p is the passive force function related to the muscle length change; a is the muscle activation level; calculate and output multimodal signals through the actual output muscle force in the muscle force-pressure relationship model.

[0016] In a feasible implementation manner, according to the correlation function, perform correlation calculation on the sensing methods between the mechanical-pressure sensing module and the microelectrical signal electrode module to obtain a correlation coefficient matrix; through the correlation coefficient matrix, and based on the pressure sensing signal, perform complementary collaborative processing on the microelectrical signal to obtain an adjusted and supplemented microelectrical signal.

[0017] The present application provides a wearable human-machine gesture recognition and interaction system for a smart construction site. Compared with the prior art, the embodiments of the present application have the following beneficial technical effects:

[0018] 1. Multimodal co-location sensing design

[0019] Integrate a pressure sensor inside the flexible air chamber to detect the change in the skin surface pressure caused by the contraction of the forearm muscle group when the worker performs various operation actions. Set microelectrical signal electrodes on the surface of the air chamber to collect the electrical activity signals of the muscle, so that the system can simultaneously obtain electrical signals and mechanical-pressure sensing signals at the same position. Significantly improve the gesture recognition accuracy and anti-interference ability in the construction site environment through multimodal fusion.

[0020] 2. High reliability and comfort

[0021] Use flexible and stretchable materials to manufacture the air chamber, and utilize 3D printing or elastomeric-like processes to ensure that the device can closely fit the forearm, reducing the interference caused by construction vibrations and wearing slippage; compared with traditional arm rings or fixed electrodes, the flexible air chamber can evenly distribute pressure, so as to maintain a high level of comfort during long-term wearing.

[0022] 3. Construction site operation control and fatigue monitoring

[0023] The multi-modal signals are interpreted in real time through algorithms. When predefined gestures (such as making a fist, raising the wrist, rotating the arm, etc.) are recognized, the system immediately sends control instructions to construction machinery or robots, reducing the dependence on physical buttons and manual operations; by analyzing the fusion characteristics of microelectrical signals and mechanical-pressure sensing signals, the muscle fatigue degree of workers can be monitored and early warnings can be given. For example, when the fatigue index exceeds the threshold, a reminder is sent to the construction site management platform to help managers with personnel allocation and risk prevention and control.

[0024] 4. Data Transmission and System Integration

[0025] The system supports wireless communication protocols and transmits gesture instructions, fatigue status, or abnormal action recognition information to the intelligent construction site management platform in real time; it can be integrated with construction machinery and equipment (such as tower cranes, excavators, robotic arms, etc.) or other sensors on the construction site to form multi-source data interaction, further improving construction safety and efficiency. Description of the Drawings

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. In the drawings:

[0027] Figure 1 It is a schematic structural diagram of a wearable human-machine gesture recognition and interaction system for an intelligent construction site provided by an embodiment of the present application;

[0028] Figure 2 It is a schematic structural diagram of a combined sensing unit provided by an embodiment of the present application;

[0029] Figure 3 It is a schematic structural diagram of a limb detection belt provided by an embodiment of the present application. Detailed Embodiments

[0030] In order to enable those skilled in the art to better understand the technical solutions in the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0031] It should be noted that based on the multi-modal sensing technology of the microelectrical signal electrode module (used to detect the electrical activities generated by human muscles) and the "mechanics-pressure sensing module", the gestures and muscle states of workers are accurately recognized and monitored in real time. By simultaneously collecting the electrical activity signals and mechanical deformation signals of muscles at the same position, the accuracy and anti-interference ability of gesture recognition are significantly improved. The system consists of a flexible air chamber, microelectrical signal electrodes installed on the air chamber, and a supporting data processing algorithm. It can be worn on the worker's forearm or other parts of the body, used to recognize various gestures for remotely / closely controlling construction site machinery, equipment, or robots, and analyze and warn the worker's fatigue during long-term high-intensity operations. Compared with traditional single-modal sensors, the present invention has higher stability in harsh environments such as high noise and multi-dust in construction sites, and is easy to integrate with the intelligent construction site management platform, with broad application prospects.

[0032] An embodiment of the present application provides a wearable human-machine gesture recognition and interaction system for an intelligent construction site. Figure 1 As a schematic structural diagram of a wearable human-machine gesture recognition and interaction system provided by an embodiment of the present application, as Figure 1 shown, the wearable human-machine gesture recognition and interaction system 100 includes: a flexible air chamber assembly, which is used to fit the body surface of the user and collect the pressure sensing signals on the skin surface through the mechanics-pressure sensing module 110 in the flexible air chamber assembly. A microelectrical signal electrode module 120, which is installed on the surface of the flexible air chamber assembly and synchronously collects the microelectrical signals of the muscles in the skin surface. A microcontroller module 130, which is used to synchronously sample and process the pressure sensing signals and microelectrical signals, and preprocess and filter the collected multi-modal signals. A signal processing and gesture recognition module 140, which is used to map the multi-modal signals into corresponding gesture information and corresponding muscle state information. A communication module 150, which is used to send the gesture information and muscle state information to the intelligent construction site management platform.

[0033] As a feasible implementation manner, the microelectrical signal electrode module 120 is deployed on the surface of the air chamber in the flexible air chamber assembly to collect the microelectrical signals under muscle activities. Each air chamber surface has three microelectrical signal electrode modules 120, and the microelectrical signal electrode modules 120 are fixed in a prefabricated circular housing.

[0034] As a feasible implementation, a mechanical-pressure sensing module 110 is deployed on the side of each air chamber. The numerical values of the signal channels, sampling frequencies, and resolutions of the mechanical-pressure sensing module 110 and the microelectrical signal electrode module 120 are all the same. The air chamber, the microelectrical signal electrode module 120, and the mechanical-pressure sensing module 110 are combined into a combined sensing unit. A plurality of combined sensing units are placed side by side to form a limb detection band.

[0035] As a feasible implementation, the signal acquisition positions of the microelectrical signal electrode module 120 are made to coincide with the signal acquisition positions of the mechanical-pressure sensing module 110 in the flexible air chamber assembly, and based on the same timing characteristics, the microelectrical signals and pressure sensing signals at the same signal acquisition position are synchronously acquired.

[0036] In one embodiment, Figure 2 FIG. is a schematic structural diagram of a combined sensing unit provided by an embodiment of the present application. Figure 3 FIG. is a schematic structural diagram of a limb detection band provided by an embodiment of the present application. As Figure 2 and Figure 3 shown, the entire limb detection band is composed of 8 pairs of complete microelectrical signal electrode modules 120 and mechanical-pressure sensing module 110 sensors. On the surface of each air chamber, three electrodes of the microelectrical signal electrode module 120 are placed and fixed in a prefabricated circular housing. In order to measure the internal pressure of the air chamber in response to various gestures, a pressure sensor (mechanical-pressure sensing module 110) is inserted on the side of each air chamber. An RISC-V processor is used as the data acquisition device. At the same time, 16 channels of the signals of the microelectrical signal electrode module 120 and the mechanical-pressure sensing module 110 have a sampling rate of 1000 hz and a resolution of 12 bits. The signals are transmitted between the development board and the processing computer through a micro usb-serial cable. Self-adhesive tape is selected as the installation medium for the limb detection band. Small pieces of Velcro tape are cut to fit the size of the air chamber and adhered to the back of the air chamber so that each multimodal sensor can be connected to the Velcro tape limb detection band. The size of the limb detection band can be easily changed to fit different arm sizes by changing the position of the air chamber.

[0037] As a feasible implementation, based on the variation relationship between muscle force and pressure signal, a muscle force-pressure relationship model is established, specifically including: according to to obtain the total muscle force in the muscle force-pressure relationship model. Wherein, F a is the active force generated by the contractile element of the muscle fiber, F p is the passive force generated by the elastic element of the muscle, f a (Δl) and f p$(Δl)$ are the normalized active function and the passive force function related to the change in muscle length, is the maximum isometric muscle force, Δl is the change in muscle length, and a is the muscle activation level.

[0038] In one embodiment, the relationship between the signal of the mechanics-pressure sensing module 110 and the change in muscle length or cross-sectional area. The mechanics-pressure sensing module 110 measures the change in pressure, which is closely related to muscle deformation (especially the change in cross-sectional area). Therefore, the output of the mechanics-pressure sensing module 110 can be used to adjust and supplement the information in the signal of the microelectrical signal electrode module 120. For example, in the case where the muscle electrical activity is weak or unstable, the mechanics-pressure sensing module 110 can provide more stable mechanical data. Introducing the output value of the mechanics-pressure sensing module 110 into the equation, by describing the pressure change during muscle contraction and combining with the microelectrical signal of the microelectrical signal electrode module 120, the muscle force is calculated jointly. This makes the estimation of muscle force more accurate and comprehensive.

[0039] As a feasible implementation, according to the actual output muscle force F in the muscle force-pressure relationship model is obtained mv . Among them, ΔA c is the change in the cross-sectional area of the muscle; the screw-in angle θ represents the angle formed between the tendon and the muscle fiber. is the maximum isometric muscle force. f a is the normalized active function related to the change in muscle length. f p is the passive force function related to the change in muscle length. a is the muscle activation level; through the actual output muscle force in the muscle force-pressure relationship model, multimodal signals are calculated and output.

[0040] In one embodiment, the direct contact between the air chamber and the arm skin can detect the vertical component of the muscle force. Since the model assumes a constant muscle volume, it can be inferred that the output value of the mechanics-pressure sensing module is related to the change in the cross-sectional area of the muscle (ΔA c ), which reflects the pressure change caused by muscle contraction. In the muscle force equation combining the signals of the microelectrical signal electrode module 120 and the mechanics-pressure sensing module 110, the role of the mechanics-pressure sensing module is reflected in its direct perception of muscle deformation.

[0041] As a feasible implementation, according to the correlation function, the correlation calculation of the sensing methods between the mechanics-pressure sensing module 110 and the microelectrical signal electrode module 120 is performed to obtain the correlation coefficient matrix. Through the correlation coefficient matrix and based on the pressure sensing signal, the microelectrical signal is processed complementarily and synergistically to obtain the adjusted and supplemented microelectrical signal.

[0042] In one embodiment, to study the complementarity between the mechanical-pressure sensing module 110 and the microelectrical signal electrode module 120, the correlation between these two sensing methods was calculated:

[0043]

[0044] where R is the correlation coefficient matrix, C is the covariance matrix, n is the number of samples in the dataset, x and y are the variables of the two datasets, and are the dataset means.

[0045] As a feasible implementation, the signal processing and gesture recognition module 140 further includes: performing fusion generation processing on the collected pressure sensing signals and microelectrical signals to generate corresponding multimodal signals. Inputting the multimodal signals into a preset gesture recognition algorithm, and through the gesture recognition algorithm, extracting and classifying the gesture information in the multimodal signals regarding gesture features and gesture types to obtain gesture command information. Calculating the current muscle fatigue degree of the muscle state information in the multimodal signals through a preset muscle fatigue degree model to obtain a muscle fatigue degree index. Recognizing the gesture command information and performing threshold judgment on the muscle fatigue degree index to respectively obtain a predetermined gesture command and a muscle fatigue degree warning level.

[0046] As a feasible implementation, if the intelligent construction site management platform recognizes a predetermined gesture command, the intelligent construction site management platform sends the control command corresponding to the predetermined gesture command to the corresponding operating equipment of the intelligent construction site, so that the operating equipment completes contactless control. Among them, the predetermined gesture commands at least include: making a fist, raising the wrist, and rotating the arm. If the muscle fatigue degree warning level is a high-risk warning level, the intelligent construction site management platform sends and reminds the user of rest information and re-performs personnel allocation management.

[0047] As a feasible implementation, the microcontroller module 130 further includes: removing the 50Hz power line noise in the microelectrical signals through a notch filter. Dividing the filtering range of the microelectrical signals to obtain a physiological signal range. The physiological signal range includes: a first cut-off frequency, a second cut-off frequency, and a third cut-off frequency. The frequency range of the first cut-off frequency is 20 - 50Hz, the frequency range of the second cut-off frequency is 20 - 150Hz, and the frequency range of the third cut-off frequency is 20 - 500Hz. If the physiological signal range is the first cut-off frequency, the microelectrical signals are filtered through a third-order Butterworth filter. If the physiological signal range is the second cut-off frequency, the microelectrical signals are filtered through a high-pass and a low-pass filter. If the physiological signal range is the third cut-off frequency, the microelectrical signals are filtered through a fourth-order Butterworth filter.

[0048] In one embodiment, to optimize the quality of the signals of the microelectrical signal electrode module 120, a set of filters is selected. The range of the signals of the microelectrical signal electrode module 120 is 0 - 500 Hz, but the most important physiological information range is 20 - 150 Hz. Different filtering methods and cut-off frequencies are adopted, including a third-order Butterworth filter of 20 - 50 Hz, a high-pass and low-pass filter of 20 - 150 Hz, and a fourth-order Butterworth filter of 20 - 500 Hz. At the same time, a notch filter is also used to remove the 50 Hz power line noise.

[0049] As a feasible implementation, the communication module 150 supports wireless data transmission and is interconnected with construction site mechanical equipment or the intelligent construction site management system through Wi-Fi, Bluetooth, or other low-power wireless protocols to achieve real-time interaction of gesture control signals and fatigue monitoring data. It is characterized by the following steps: wearing the wearable device on the worker's forearm or a designated part and performing initial calibration.

[0050] As a feasible implementation, the microcontroller module 130 simultaneously collects microelectrical signals and pressure sensing signals at the same sampling rate, completes preprocessing and denoising. The multi-modal signals are input into a gesture recognition algorithm for feature extraction and classification to obtain gesture commands or muscle fatigue degree indicators. When a predetermined gesture command is recognized, a corresponding control signal is sent to the construction site mechanical equipment through the communication module 150. If it is detected that the muscle fatigue degree exceeds the threshold, a warning is triggered and reported to the construction site management platform. The recognition results are recorded, visualized, or further decision-making and execution are carried out through the intelligent construction site management system or the local interface.

[0051] In one embodiment, 1. A number of wearable human-machine gesture recognition and interaction devices are distributed to multiple workers and uniformly connected to the intelligent construction site management platform. When the system detects that a worker is operating continuously under high load, through data modeling of the microelectrical signals and pressure sensing signals, it judges whether the worker has action deformation or fatigue conditions. Once an abnormality occurs, such as a significant decrease in grip strength or an abnormal muscle activity pattern, the system automatically alarms the manager to reduce safety accidents caused by operation errors. At the same time, the data is stored in the cloud and can be further comprehensively analyzed in combination with other sensors (such as environmental sensors, video monitoring) to form a decision-making basis for improving construction site safety and efficiency.

[0052] The embodiments of this application significantly improve the accuracy of gesture recognition and the anti-interference ability in the construction site environment through multi-modal fusion. A flexible and stretchable material is used to manufacture the air chamber. Compared with traditional armbands or fixed electrodes, the flexible air chamber can evenly distribute pressure, thus maintaining a high level of comfort during long-term wear. The multi-modal signals are interpreted in real time through algorithms. When predefined gestures (such as making a fist, raising the wrist, rotating the arm, etc.) are recognized, the system immediately sends control instructions to construction machinery or robots on the construction site, reducing the dependence on physical buttons and manual operations. By analyzing the fusion characteristics of microelectrical signals and mechanical-pressure sensing signals, the muscle fatigue of workers can be monitored and early warnings can be given. For example, when the fatigue index exceeds the threshold, a reminder is sent to the construction site management platform to help managers with personnel allocation and risk prevention and control. At the same time, the system supports wireless communication protocols to transmit gesture instructions, fatigue status, or abnormal action recognition information to the intelligent construction site management platform in real time. It can be integrated with construction machinery and equipment on the construction site (such as tower cranes, excavators, robotic arms, etc.) or other sensors to form multi-source data interaction, further improving construction safety and efficiency.

[0053] Each embodiment in this application is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. The key point of each embodiment is to illustrate the differences from other embodiments. In particular, for the embodiments of the device and the non-volatile computer storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0054] The device and medium provided by the embodiments of this application correspond one-to-one with the method. Therefore, the device and medium also have beneficial technical effects similar to those of the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device and medium will not be elaborated here.

[0055] Those skilled in the art should understand that the embodiments of this application can be provided as a method, a system, or a computer program product. Therefore, this application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing device generate means for realizing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for realizing the functions specified in multiple blocks.

[0057] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0058] Computer-readable media includes both permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transitory medium that can store information accessible by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0059] The specific embodiments of the present application have been described above. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be executed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0060] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the embodiments of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present application shall be included within the scope of the claims of the present application.

Claims

1. A wearable human-machine gesture recognition and interaction system for an intelligent construction site, characterized in that The system includes: A flexible air chamber assembly, which is used to fit the body surface of the user, and collect the pressure perception signal on the skin surface through the mechanical-pressure perception module in the flexible air chamber assembly; A microelectrical signal electrode module, which is installed on the surface of the flexible air chamber assembly and synchronously collects the microelectrical signals of the muscles in the skin surface; A microcontroller module, which is used to perform synchronous sampling processing on the pressure perception signal and the microelectrical signal, and preprocess and filter the collected multimodal signals; A signal processing and gesture recognition module, which is used to map the multimodal signals to corresponding gesture information and corresponding muscle state information; A communication module, which is used to send the gesture information and the muscle state information to the intelligent construction site management platform.

2. The wearable human-machine gesture recognition and interaction system for an intelligent construction site according to claim 1, wherein The signal acquisition positions of the microelectrical signal electrode module and the mechanical-pressure perception module in the flexible air chamber assembly are overlapped, and based on the same timing characteristics, the microelectrical signal and the pressure perception signal at the same signal acquisition position are synchronously collected.

3. The wearable human-machine gesture recognition and interaction system for intelligent construction sites according to claim 1, characterized in that, The signal processing and gesture recognition module further includes: Performing fusion generation processing on the collected pressure perception signal and the microelectrical signal to generate corresponding multimodal signals; Inputting the multimodal signals into a preset gesture recognition algorithm, and through the gesture recognition algorithm, extracting and classifying the gesture information in the multimodal signals for gesture features and gesture types to obtain gesture command information; Calculating the current muscle fatigue degree of the muscle state information in the multimodal signals through a preset muscle fatigue degree model to obtain a muscle fatigue degree index; Identifying the gesture command information and performing threshold judgment on the muscle fatigue degree index to obtain a predetermined gesture command and a muscle fatigue degree warning level respectively.

4. The wearable human-machine gesture recognition and interaction system for an intelligent construction site according to claim 3, wherein If the intelligent construction site management platform recognizes the predetermined gesture command, the control command corresponding to the predetermined gesture command is sent to the corresponding operating equipment of the intelligent construction site through the intelligent construction site management platform, so that the operating equipment completes contactless control; wherein, the predetermined gesture command at least includes: making a fist, raising the wrist, and rotating the arm; If the muscle fatigue degree warning level is a high-risk warning level, the intelligent construction site management platform sends and reminds the user of the rest information and re-performs personnel deployment management.

5. The wearable human-machine gesture recognition and interaction system for a smart construction site according to claim 1, characterized in that, The microelectrical signal electrode module is deployed on the air chamber surface in the flexible air chamber assembly for collecting the microelectrical signals under muscle activity; Each air chamber surface has three of the microelectrical signal electrode modules, and the microelectrical signal electrode modules are fixed in a prefabricated circular housing.

6. The wearable human-machine gesture recognition and interaction system for an intelligent construction site according to claim 5, wherein, One of the mechanical-pressure perception modules is deployed on the side of each air chamber; The numerical values of the signal channels, sampling frequencies, and resolutions of the mechanical-pressure sensing module and the microelectrical signal electrode module are all the same numerical values; The air chamber, the microelectrical signal electrode module, and the mechanical-pressure sensing module are combined into a combined sensing unit; A plurality of the combined sensing units are placed side by side to form a limb detection belt.

7. A wearable human-machine gesture recognition and interaction system for a smart construction site according to claim 1, characterized in that, The microcontroller module further includes: Removing the 50 Hz power line noise in the microelectrical signal through a notch filter; Dividing the filtering range of the microelectrical signal to obtain a physiological signal range; wherein, the physiological signal range includes: a first cut-off frequency, a second cut-off frequency, and a third cut-off frequency; the frequency range of the first cut-off frequency is 20 - 50 Hz, the frequency range of the second cut-off frequency is 20 - 150 Hz, and the frequency range of the third cut-off frequency is 20 - 500 Hz; If the physiological signal range is the first cut-off frequency, the microelectrical signal is filtered through a third-order Butterworth filter; If the physiological signal range is the second cut-off frequency, the microelectrical signal is filtered through a high-pass and a low-pass filter; If the physiological signal range is the third cut-off frequency, the microelectrical signal is filtered through a fourth-order Butterworth filter.

8. The wearable human-machine gesture recognition and interaction system for intelligent construction sites according to claim 1, characterized in that, Based on the variation relationship between the muscle force and the pressure signal, a muscle force-pressure relationship model is established, specifically including: According to obtain the total muscle force in the muscle force-pressure relationship model; wherein, F a is the active force generated by the contractile element of the muscle fiber, F p is the passive force generated by the elastic element of the muscle, f a (Δl) and f p (Δl) are the normalized active function and passive force function related to the muscle length change respectively, is the maximum isometric muscle force, Δl is the muscle length change, and a is the muscle activation level.

9. The wearable human-machine gesture recognition and interaction system for a smart construction site according to claim 8, wherein According to obtain the actual output muscle force F in the muscle force-pressure relationship model mv ; where, ΔA c is the change in the cross-sectional area of the muscle; the screw-in angle θ represents the angle formed between the tendon and the muscle fiber; is the maximum isometric muscle force; f a is the normalized active function related to the change in muscle length; f p is the passive force function related to the change in muscle length; a is the muscle activation level; The actual output muscle force in the muscle force-pressure relationship model is used to calculate and output a multimodal signal.

10. The wearable human-machine gesture recognition and interaction system for a smart construction site according to claim 1, wherein According to the correlation function, the correlation calculation of the sensing methods between the mechanical-pressure sensing module and the microelectrical signal electrode module is performed to obtain a correlation coefficient matrix; Based on the correlation coefficient matrix and the pressure sensing signal, the microelectrical signal is subjected to complementary collaborative processing to obtain an adjusted and supplemented microelectrical signal.

Images