Man-machine interaction system based on wearable device

Abstract

The invention discloses a man-machine interaction system based on wearable equipment, and relates to the technical field of wearable equipment, and the man-machine interaction system comprises an environment data identification module, a visual and auditory analysis module and a tactile feedback starting module. Evaluating the stability index of each part of the intelligent helmet during the industrial production operation, screening each abnormal part, releasing test dynamic objects and test abnormal sound information by the intelligent helmet, collecting visual and auditory feedback data of each abnormal part, analyzing the visual and auditory feedback index of each abnormal part of the intelligent helmet during the industrial production operation, and obtaining the visual and auditory feedback data of each abnormal part. Monitoring failure or false alarm caused by defects of equipment is avoided, whether visual and auditory feedback of abnormal parts is abnormal or not is judged, if yes, tactile feedback of all the abnormal parts is started, feedback is conducted according to the abnormal parts, accuracy is improved, an operator can rapidly position the problem direction, and the working efficiency is improved. The panic or misoperation caused by the fact that the abnormal source cannot be determined is avoided, and the accuracy and safety of danger response are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wearable devices, in particular to a human-computer interaction system based on a wearable device. BACKGROUND

[0002] In the industrial field, wearable devices such as intelligent helmets can realize real-time monitoring of the physiological state and operation behavior of workers, as well as the collection of work environment data by integrating sensors. Some intelligent helmets are equipped with cameras, which can assist workers in equipment detection and maintenance, and present device information to workers through visual recognition technology. However, the existing system has deficiencies in multi-modal data fusion, and various sensory data such as vision, hearing, and touch cannot be efficiently processed in coordination, resulting in poor timeliness and accuracy of information feedback, which cannot meet the demand for rapid response to emergencies in industrial production. Therefore, it is necessary to analyze a human-computer interaction system based on a wearable device.

[0003] The prior art such as the invention application patent with publication number CN118778796A discloses a human-computer interaction system based on a wearable device. The human-computer interaction system uses a wearable device to replace a handle for positioning or control, greatly reducing the overall size, freeing the user's hands, facilitating control and carrying, and improving the user experience.

[0004] The existing technology of a human-computer interaction system based on a wearable device can meet the basic requirements, but also has some potential defects and challenges, which are embodied in the following aspects: first, the evaluation of the stability index of each part of the intelligent helmet during industrial production is not deep enough in the existing technology, which affects the screening of each abnormal part, reduces the accuracy of collecting audio-visual feedback data of each abnormal part, and affects the analysis of the audio-visual feedback index of each abnormal part of the intelligent helmet during industrial production, resulting in monitoring failure or false alarm due to device defects, reducing the reliability of safety protection function, and reducing the processing efficiency of complex information by workers.

[0005] Second, the existing technology does not pay enough attention to whether the audio-visual feedback of the abnormal part is abnormal, which affects the feedback based on the abnormal part and reduces the accuracy, affecting the workers to quickly locate the problem direction, leading to panic or misoperation due to the inability to determine the source of the abnormality, reducing the accuracy and safety of risk response, and reducing the response speed. SUMMARY

[0006] The present application aims to provide a human-computer interaction system based on a wearable device, which solves the problems in the background art.

[0007] To solve the above technical problems, the present application adopts the following technical solutions: the present application provides a human-computer interaction system based on wearable devices, comprising: an environmental data identification module, an audio-visual analysis module and a tactile feedback starting module.

[0008] The environmental data identification module: through the high-definition camera carried by the intelligent helmet, the surrounding environment data of the intelligent helmet during industrial production operation is collected, and the stability index of each part of the intelligent helmet during industrial production operation is evaluated.

[0009] The audio-visual analysis module: based on the stability index of each part of the intelligent helmet during industrial production operation, each abnormal part is screened, and test dynamic objects and test abnormal sound information are released by the intelligent helmet, audio-visual feedback data of each abnormal part is collected, and audio-visual feedback index of each abnormal part of the intelligent helmet during industrial production operation is analyzed.

[0010] The tactile feedback starting module: based on the audio-visual feedback index of each abnormal part of the intelligent helmet during industrial production operation, it is judged whether the audio-visual feedback of the abnormal part is abnormal, if abnormal, the tactile feedback of each abnormal part is started, and an abnormal prompt warning is made.

[0011] Further, the evaluation of the stability index of each part of the intelligent helmet during industrial production operation has a specific analysis method: based on the obtained surrounding environment data, wherein the surrounding environment data includes: heat dissipation value, electronic component stability value, dust coverage area, and picture freezing frequency of each part, and the heat dissipation value safety interval, electronic component stability value safety interval, dust coverage area safety interval, and picture freezing frequency safety interval of each part of the intelligent helmet during industrial production operation are extracted from the database, the stability index of each part of the intelligent helmet during industrial production operation is analyzed, and the specific calculation formula is: W di W represents the i-th surrounding environment data of the d-th part of the intelligent helmet during industrial production operation, W di ' represents the safety interval of the i-th surrounding environment data of the d-th part of the intelligent helmet during industrial production operation, i∈[1,4], d represents the number of each part, d=1,2,...,k, and k represents the number of parts.

[0012] Further, the abnormal parts of the smart helmet in industrial production operation are screened, and a specific analysis method is as follows: based on the stability indexes of the parts of the smart helmet in industrial production operation obtained, the stability indexes of the parts of the smart helmet in industrial production operation are compared with the stability index thresholds of the parts of the smart helmet in industrial production operation in the database; if the stability index of a part of the smart helmet in industrial production operation is less than the stability index of the part, the stability index of the part of the smart helmet in industrial production operation is recorded as an abnormal index, and then the abnormal parts of the smart helmet in industrial production operation are screened.

[0013] Further, the audio-visual feedback index of the abnormal parts of the smart helmet in industrial production operation, and a specific analysis method is as follows: based on the obtained audio-visual feedback data, the audio-visual feedback index of the abnormal parts of the smart helmet in industrial production operation is analyzed, wherein the abnormal audio-visual data includes visual abnormal data and auditory abnormal data. and the auditory feedback index γ t , and then the audio-visual feedback index of the abnormal parts of the smart helmet in industrial production operation is evaluated, and a specific calculation formula is as follows: t represents the number of the abnormal part, t = 1, 2,..., h, and h represents the number of the abnormal part.

[0014] Further, the visual feedback index of the abnormal parts of the smart helmet in industrial production operation, and a specific analysis method is as follows: based on the obtained visual abnormal data, the visual feedback index of the abnormal parts of the smart helmet in industrial production operation is analyzed, wherein the visual abnormal data includes the basic information of the dynamic object of the abnormal parts of the smart helmet in industrial production operation, and the basic information includes volume, weight, moving track of the dynamic object, and threat index; the basic information of the dynamic object of the abnormal parts of the smart helmet in industrial production operation is compared with the basic information of the dynamic object stored in the database respectively; if the basic information of the dynamic object of the abnormal parts of the smart helmet in industrial production operation is different from the basic information of the dynamic object stored in the database, it indicates that the visual recognition of the abnormal parts of the smart helmet in industrial production operation is abnormal, the visual object interference feedback type of the smart helmet is obtained, the visual feedback reference duration is matched, if there is feedback, and the visual feedback duration of a certain abnormal part is greater than the feedback reference duration or there is no feedback, the parameters of the camera of the smart helmet are automatically adjusted, and the visual abnormal data is reacquired.

[0015] Further, the dynamic object threat index, and a specific analysis method is as follows: the dynamic data is obtained through visual perception of the camera, wherein the dynamic data includes the motion speed, the minimum distance, and the track intersection frequency of the dynamic object in each time period, and the motion reference speed, the minimum reference distance, and the track intersection reference frequency of the dynamic object are extracted from the database; the dynamic object threat index is analyzed, and a specific calculation formula is as follows: wherein c' represents the reference speed of motion of the dynamic object, c s represents the speed of motion of the dynamic object for the s-th time period, f' represents the minimum reference distance of the dynamic object, f s represents the minimum distance of the dynamic object for the s-th time period, a' represents the reference frequency of trajectory crossing of the dynamic object, a s represents the frequency of trajectory crossing of the dynamic object for the s-th time period, s represents the number of each time period, s = 1, 2,..., v, and v represents the number of time periods.

[0016] Further, the hearing feedback index of each abnormal part of the smart helmet during industrial production operation, and the specific analysis method is: based on the obtained hearing abnormal data, wherein the hearing abnormal data includes: test abnormal sound information of each abnormal part of the smart helmet during industrial production operation, and the test abnormal sound information includes sound source direction, sound volume and frequency; the test abnormal sound information of each abnormal part of the smart helmet during industrial production operation is compared with the test abnormal sound information stored in the database respectively; if the test abnormal sound information of each abnormal part of the smart helmet during industrial production operation is different from the test abnormal sound information stored in the database, it indicates that the hearing recognition of each abnormal part of the smart helmet during industrial production operation is abnormal, the hearing object interference feedback type of the smart helmet is obtained, the hearing feedback reference duration is matched, if there is feedback, and the hearing feedback duration of a certain abnormal part is greater than the feedback reference duration or there is no feedback, the parameters of the recording of the smart helmet are automatically adjusted, and the hearing abnormal data is reacquired.

[0017] Further, the specific analysis method of judging whether the audio-visual feedback of each abnormal part is abnormal or not is: based on the obtained audio-visual feedback index of each abnormal part of the smart helmet during industrial production operation, the audio-visual feedback index of each abnormal part of the smart helmet during industrial production operation is compared with the audio-visual feedback index threshold of each abnormal part of the smart helmet during industrial production operation stored in the database; if the audio-visual feedback index of a certain abnormal part of the smart helmet during industrial production operation is less than the audio-visual feedback index, it indicates that the audio-visual feedback of the abnormal part is in an abnormal state, and a heat map is generated to judge the distribution type of the audio-visual feedback, wherein the distribution type includes the aggregation type and the dispersion type.

[0018] Further, the specific analysis method of starting the tactile feedback of each abnormal part is: when the distribution type of the audio-visual feedback is the aggregation type, the tactile feedback of each abnormal part is started, the synchronous strong vibration of each tactile feedback area is triggered, the smart helmet tightening mechanism is combined, the voice prompt is triggered at the same time, the user is forced to stop moving and evacuate, and an alarm is sent to the background at the same time; when the distribution type of the audio-visual feedback is the dispersion type, the tactile feedback of each abnormal part is not started, and the feedback is performed again when the current operation is completed.

[0019] Further, the database is further included, wherein the database is used to store: a heat dissipation value safety interval of each part of the smart helmet during industrial production operation, an electronic element stability value safety interval, a dust coverage area safety interval, a picture lag frequency safety interval, a stability index threshold of each part of the smart helmet during industrial production operation, basic information of a dynamic object, a motion reference speed of the dynamic object, a minimum reference distance and a trajectory intersection reference frequency, test abnormal sound information, and an audio-visual feedback index threshold of each abnormal part of the smart helmet during industrial production operation.

[0020] The present application has the beneficial effects that, in the environmental data identification module and the audio-visual analysis module: the high-definition camera carried by the smart helmet is used to collect the surrounding environmental data of the smart helmet during industrial production operation, to identify potential dangers in advance, to evaluate the stability index of each part of the smart helmet during industrial production operation, to screen each abnormal part, to release test dynamic objects and test abnormal sound information by the smart helmet, to collect audio-visual feedback data of each abnormal part, to analyze the audio-visual feedback index of each abnormal part of the smart helmet during industrial production operation, to avoid monitoring failure or false reporting caused by equipment defects, to ensure the reliability of the safety protection function, and to improve the processing efficiency of complex information by the operation personnel.

[0021] In the tactile feedback starting module: based on the obtained audio-visual feedback index of each abnormal part of the smart helmet during industrial production operation, it is judged whether the audio-visual feedback of the abnormal part is abnormal, if it is abnormal, the tactile feedback of each abnormal part is started, and an abnormal prompt warning is made, the feedback is made according to the abnormal part, the precision is improved, the operation personnel can quickly locate the problem direction, the panic or misoperation caused by the inability to determine the abnormal source is avoided, the accuracy and safety of the danger response are improved, and the response speed is improved. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0023] Figure 1 The present application is a system structure connection diagram. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0025] Referring to Figure 1 As shown in the figure, the present application provides a human-computer interaction system based on a wearable device, comprising: an environmental data identification module, an audio-visual analysis module and a tactile feedback starting module.

[0026] The environmental data identification module: through the high-definition camera carried by the intelligent helmet, the surrounding environment data of the intelligent helmet during industrial production operation is collected, and the stability index of each part of the intelligent helmet during industrial production operation is evaluated.

[0027] In the above embodiment, the stability index of each part of the intelligent helmet during industrial production operation is evaluated, and the specific analysis method is: based on the obtained surrounding environment data, wherein the surrounding environment data includes: heat dissipation value, electronic component stability value, dust coverage area, and frame freezing frequency of each part, and the heat dissipation value safety interval, electronic component stability value safety interval, dust coverage area safety interval, and frame freezing frequency safety interval of each part of the intelligent helmet during industrial production operation are extracted from the database, and the stability index of each part of the intelligent helmet during industrial production operation is analyzed, and the specific calculation formula is: W di W represents the i-th surrounding environment data of the d-th part of the intelligent helmet during industrial production operation, W di W represents the safety interval of the i-th surrounding environment data of the d-th part of the intelligent helmet during industrial production operation, i∈[1,4], d represents the number of each part, d=1,2,...,k, and k represents the number of parts.

[0028] It should be noted that the heat dissipation value, electronic component stability value, dust coverage area, and frame freezing frequency of each part are influenced by the environment.

[0029] It should be noted that the heat dissipation value of each part affects the precision hardware such as the built-in camera, sensor, and processor of the intelligent helmet, which generates heat during operation. If the heat dissipation of a part is poor, such as the processor part at the top of the helmet, it may cause the performance of the chip to decline, the battery life to be shortened, and even cause circuit failure.

[0030] It should be noted that the electronic component stability value affects the environmental perception ability of the intelligent helmet. If the electronic component stability value is abnormal, it may cause the helmet to misjudge the head movement of the operator.

[0031] It should be noted that the dust coverage area will cause the smart helmet to reduce the clarity of the captured picture, for example, the lens dust will cause the image resolution to lose more than 30%.

[0032] It should be noted that the number of frame freezing times is to ensure the real-time and reliability of visual information. By analyzing the parts where freezing occurs, such as the front-end camera encoding module and the wireless transmission part, the bottleneck link can be located.

[0033] Audiovisual analysis module: based on the stability index of each part of the smart helmet during industrial production operation, screening each abnormal part, and releasing test dynamic objects and test abnormal sound information by the smart helmet, collecting audiovisual feedback data of each abnormal part, and analyzing audiovisual feedback index of each abnormal part of the smart helmet during industrial production operation.

[0034] In the above embodiment, the specific analysis method of screening each abnormal part is: based on the stability index of each part of the smart helmet during industrial production operation, comparing the stability index of each part of the smart helmet during industrial production operation with the stability index threshold of each part of the smart helmet during industrial production operation in the database, if the stability index of a part of the smart helmet during industrial production operation is less than the stability index of the part, the stability index of the part of the smart helmet during industrial production operation is recorded as an abnormal index, and then each abnormal part of the smart helmet during industrial production operation is screened.

[0035] In the above embodiment, the specific analysis method of the audiovisual feedback index of each abnormal part of the smart helmet during industrial production operation is: based on the obtained audiovisual feedback data, wherein the abnormal audiovisual data includes: visual abnormal data and auditory abnormal data, and analyzing the visual feedback index of each abnormal part of the smart helmet during industrial production operation And the auditory feedback index γ t , and then evaluating the audiovisual feedback index of each abnormal part of the smart helmet during industrial production operation, and the specific calculation formula is: t represents the number of each abnormal part, t = 1, 2,..., h, and h represents the number of abnormal parts.

[0036] In the above embodiment, the visual feedback index of the smart helmet at each abnormal position during industrial production operation is analyzed in particular as follows: based on the obtained visual abnormal data, the visual abnormal data including basic information of dynamic objects at each abnormal position of the smart helmet during industrial production operation, the basic information including volume, weight, moving track of the dynamic objects and threat index, the basic information of the dynamic objects at each abnormal position of the smart helmet during industrial production operation is compared with the basic information of the dynamic objects stored in the database, if the basic information of the dynamic objects at each abnormal position of the smart helmet during industrial production operation is different from the basic information of the dynamic objects stored in the database, it indicates that the visual recognition of the smart helmet at each abnormal position during industrial production operation is abnormal, the visual object interference feedback type of the smart helmet is obtained, the visual feedback reference time is matched, if there is feedback and the visual feedback time of a certain abnormal position is greater than the feedback reference time or there is no feedback, the parameters of the camera of the smart helmet are automatically adjusted, and the visual abnormal data is reacquired.

[0037] It should be noted that the visual object interference feedback type of the smart helmet includes feedback and no feedback.

[0038] It should be noted that the moving track of the dynamic object is used to judge the helmet protection performance and the lens impact resistance.

[0039] In the above embodiment, the threat index of the dynamic object is analyzed in particular as follows: dynamic data is obtained through visual perception of the camera, wherein the dynamic data includes movement speed, minimum distance and track intersection frequency of each time period of the dynamic object, and movement reference speed, minimum reference distance and track intersection reference frequency of the dynamic object are extracted from the database, and the threat index of the dynamic object is analyzed, and the specific calculation formula is as follows: Wherein, c' represents the movement reference speed of the dynamic object, c s represents the movement speed of the s-th time period of the dynamic object, f' represents the minimum reference distance of the dynamic object, f s represents the minimum distance of the s-th time period of the dynamic object, a' represents the track intersection frequency reference of the dynamic object, a s represents the track intersection frequency of the s-th time period of the dynamic object, s represents the number of each time period, s = 1, 2,..., v, and v represents the number of time periods.

[0040] It should be noted that the movement speed is used to evaluate the tracking stability of the helmet in the dynamic environment.

[0041] It should be noted that the track intersection frequency refers to the average number of intersection points in space or the number of intersections with the personnel path within a time period.

[0042] It should be noted that the minimum distance refers to the minimum distance between the splash trajectory and the smart helmet.

[0043] In the above embodiment, the specific analysis method of the auditory feedback index of each abnormal part of the smart helmet during industrial production operation is as follows: based on the obtained auditory abnormal data, wherein the auditory abnormal data includes test abnormal sound information of each abnormal part of the smart helmet during industrial production operation, the test abnormal sound information includes sound source direction, sound volume and frequency, the test abnormal sound information of each abnormal part of the smart helmet during industrial production operation is compared with the test abnormal sound information stored in the database, if the test abnormal sound information of each abnormal part of the smart helmet during industrial production operation is different from the test abnormal sound information stored in the database, it indicates that the auditory recognition of each abnormal part of the smart helmet during industrial production operation is abnormal, the auditory object interference feedback type of the smart helmet is obtained, the auditory feedback reference time is matched, if there is feedback, and the auditory feedback time of a certain abnormal part is greater than the feedback reference time or there is no feedback, the parameters of the recording of the smart helmet are automatically adjusted, and the auditory abnormal data is reacquired.

[0044] It should be noted that the auditory object interference feedback type of the smart helmet includes feedback and no feedback.

[0045] In the environmental data recognition module and the audiovisual analysis module: through the high-definition camera carried by the smart helmet, the surrounding environmental data of the smart helmet during industrial production operation is collected, potential dangers are identified in advance, the stability index of each part of the smart helmet during industrial production operation is evaluated, each abnormal part is screened, test dynamic objects and test abnormal sound information are released by the smart helmet, audiovisual feedback data of each abnormal part is collected, audiovisual feedback index of each abnormal part of the smart helmet during industrial production operation is analyzed, monitoring failure or false alarm caused by equipment defects is avoided, the reliability of safety protection function is ensured, and the processing efficiency of complex information of the operator is improved.

[0046] The tactile feedback starting module: based on the obtained audiovisual feedback index of each abnormal part of the smart helmet during industrial production operation, it is judged whether the audiovisual feedback of the abnormal part is abnormal, if it is abnormal, the tactile feedback of each abnormal part is started, and an abnormal prompt warning is made.

[0047] In the above embodiment, the specific analysis method of judging whether the audio-visual feedback of each abnormal part is abnormal is: based on the obtained audio-visual feedback index of each abnormal part of the intelligent helmet during industrial production operation, the audio-visual feedback index of each abnormal part of the intelligent helmet during industrial production operation is compared with the audio-visual feedback index threshold of each abnormal part of the intelligent helmet during industrial production operation stored in the database. If the audio-visual feedback index of a certain abnormal part of the intelligent helmet during industrial production operation is less than the audio-visual feedback index, it means that the audio-visual feedback of the abnormal part is in an abnormal state, and a heat map is generated to judge the distribution type of the audio-visual feedback, wherein the distribution type includes the aggregation type and the dispersion type.

[0048] In the above embodiment, the specific analysis method of starting the tactile feedback of each abnormal part is: when the distribution type of the audio-visual feedback is the aggregation type, the tactile feedback of each abnormal part is started, the synchronous strong vibration of each tactile feedback area is triggered, and the intelligent helmet tightening mechanism is combined, and the voice prompt is triggered at the same time, the user is forced to stop moving and evacuate, and an alarm is sent to the background. When the distribution type of the audio-visual feedback is the dispersion type, the tactile feedback of each abnormal part is not started, and the feedback is performed again when the current operation is completed.

[0049] It should be noted that the strong vibration is, for example, 10HZ high-frequency vibration.

[0050] It should be noted that the intelligent helmet tightening mechanism is, for example, a gas bag rapid inflation.

[0051] In the above embodiment, the database is further included, wherein the database is used to store: the heat dissipation value safety interval of each part of the intelligent helmet during industrial production operation, the electronic element stable value safety interval, the dust coverage area safety interval, the picture lag frequency safety interval, the stable index threshold of each part of the intelligent helmet during industrial production operation, the basic information of the dynamic object, the motion reference speed of the dynamic object, the minimum reference distance and the trajectory intersection reference frequency, the test abnormal sound information, and the audio-visual feedback index threshold of each abnormal part of the intelligent helmet during industrial production operation.

[0052] It should be noted that the environmental data recognition module and the audio-visual analysis module are connected, the audio-visual analysis module and the tactile feedback starting module are linked, the environmental data recognition module and the database are connected, the audio-visual analysis module and the database are connected, and the tactile feedback starting module and the database are connected.

[0053] In the tactile feedback starting module: based on the obtained audio-visual feedback index of each abnormal position of the smart helmet during industrial production operation, it is judged whether the audio-visual feedback of the abnormal position is abnormal, if abnormal, the tactile feedback of each abnormal position is started, and an abnormal prompt warning is made, feedback is made according to the abnormal position, the precision is improved, the operator can quickly locate the problem direction, avoids the panic or misoperation caused by the inability to determine the abnormal source, improves the accuracy and safety of the danger response, and improves the response speed.

[0054] The above is only an example and description of the concept of the application, and those skilled in the art can make various modifications or supplements to the described specific embodiments or use similar ways to replace, as long as the concept of the application is not deviated or the scope defined by the application is not exceeded, which shall belong to the protection scope of the application.

Claims

1. A human-computer interaction system based on wearable devices, characterized in that: include: Environmental data recognition module: The high-definition camera on the smart helmet collects data about the surrounding environment of the wearer during industrial production operations, and evaluates the stability index of various parts of the smart helmet during industrial production operations; Visual and auditory analysis module: Based on the obtained stability index of each part of the smart helmet during industrial production operations, each abnormal part is screened, and the smart helmet releases test dynamic objects and test abnormal sound information, collects visual and auditory feedback data of each abnormal part, and analyzes the visual and auditory feedback index of each abnormal part of the smart helmet during industrial production operations; Tactile feedback activation module: Based on the visual and auditory feedback index of each abnormal part of the smart helmet during industrial production operations, it is determined whether the visual and auditory feedback of the abnormal part is abnormal. If it is abnormal, the tactile feedback of each abnormal part is activated and an abnormal prompt warning is issued.

2. A wearable device-based human-computer interaction system according to claim 1, characterized in that: The specific analysis method for evaluating the stability index of each part of the smart helmet during industrial production operations is as follows: Based on the obtained ambient environment data, which includes: heat dissipation value of each part, stability value of electronic components, dust coverage area, and number of screen freezes, the heat dissipation value safety interval of each part of the smart helmet during industrial production operation, the electronic component stability value safety interval, the dust coverage area safety interval, and the number of screen freezes safety interval are extracted from the database, and the stability index of each part of the smart helmet during industrial production operation is analyzed. The specific calculation formula is: W di It is represented as the i-th surrounding environment data of the d-th part of the smart helmet during industrial production operation, W di ' represents the safety interval of the ith surrounding environment data of the dth part of the smart helmet during industrial production operation, i∈[1,4], d represents the number of each part, d=1,2,...,k, k represents the number of parts.

3. A wearable device-based human-computer interaction system according to claim 1, characterized in that: The specific analysis method for screening each abnormal part is as follows: Based on the obtained stability index of each part of the smart helmet during industrial production operation, the stability index of each part of the smart helmet during industrial production operation is compared with the stability index threshold of each part of the smart helmet during industrial production operation in the database. If the stability index of a certain part of the smart helmet during industrial production operation is less than the stability index of a certain part, the stability index of the part of the smart helmet during industrial production operation is recorded as an abnormal index, and then the abnormal parts of the smart helmet during industrial production operation are screened out.

4. A wearable device-based human-computer interaction system according to claim 1, characterized in that: The specific analysis method of the visual and auditory feedback index of each abnormal part of the smart helmet during industrial production operation is as follows: Based on the obtained visual and auditory feedback data, the abnormal visual and auditory data includes: visual abnormality data and auditory abnormality data, and analyzes the visual feedback index of each abnormal part of the smart helmet during industrial production operations and auditory feedback index γ t , and then evaluate the visual and auditory feedback index of each abnormal part of the smart helmet during industrial production operations. The specific calculation formula is: t represents the number of each abnormal part, t=1, 2, ..., h, and h represents the number of the abnormal part.

5. A wearable device-based human-computer interaction system according to claim 4, characterized in that: The specific analysis method of the visual feedback index of each abnormal part of the smart helmet during industrial production operation is as follows: Based on the obtained visual abnormality data, the visual abnormality data includes: basic information of dynamic objects in various abnormal parts of the smart helmet during industrial production operations, the basic information includes volume, weight, movement trajectory and threat index of dynamic objects, and the basic information of dynamic objects in various abnormal parts of the smart helmet during industrial production operations are compared with the basic information of dynamic objects stored in the database. If the basic information of dynamic objects in various abnormal parts of the smart helmet during industrial production operations is different from the basic information of dynamic objects stored in the database, it means that the visual recognition of various abnormal parts of the smart helmet during industrial production operations is abnormal, and the visual object interference feedback type of the smart helmet is obtained, and the visual feedback reference duration is matched. If there is feedback, and the visual feedback duration of a certain abnormal part is greater than the feedback reference duration or there is no feedback, the parameters of the camera of the smart helmet are automatically adjusted, and the visual abnormality data is re-acquired.

6. A wearable device-based human-computer interaction system according to claim 5, characterized in that: The dynamic object threat index is specifically analyzed as follows: Dynamic data is obtained through camera visual perception, where the dynamic data includes the movement speed, minimum spacing, and track crossing frequency of dynamic objects in each time period. The reference movement speed, minimum reference spacing, and track crossing reference frequency of dynamic objects are extracted from the database to analyze the dynamic object threat index. The specific calculation formula is: Among them, c' represents the reference speed of the dynamic object, c s It is expressed as the speed of the dynamic object in the sth time period, f' is expressed as the minimum reference distance of the dynamic object, and f s It is represented as the minimum spacing of the dynamic object in the sth time period, a' is represented as the trajectory cross-frequency reference rate of the dynamic object, and a s It is expressed as the trajectory crossing frequency of the dynamic object in the sth time period, s represents the number of each time period, s = 1, 2, ..., v, and v represents the number of time periods.

7. A wearable device-based human-computer interaction system according to claim 4, characterized in that: The specific analysis method of the auditory feedback index of each abnormal part of the smart helmet during industrial production operation is as follows: Based on the obtained auditory abnormality data, wherein the auditory abnormality data includes: the test abnormal sound information of each abnormal part of the smart helmet during industrial production operation, the test abnormal sound information includes the sound source direction, sound volume and frequency, and the test abnormal sound information of each abnormal part of the smart helmet during industrial production operation is compared with the test abnormal sound information stored in the database. If the test abnormal sound information of each abnormal part of the smart helmet during industrial production operation is different from the test abnormal sound information stored in the database, it means that the auditory recognition of each abnormal part of the smart helmet during industrial production operation is abnormal, and the auditory object interference feedback type of the smart helmet is obtained, and the auditory feedback reference duration is matched. If there is feedback, and the auditory feedback duration of a certain abnormal part is greater than the feedback reference duration or there is no feedback, the recording parameters of the smart helmet are automatically adjusted, and the auditory abnormality data is re-acquired.

8. The human-computer interaction system based on a wearable device according to claim 1, characterized in that: The specific analysis method for determining whether the visual and auditory feedback of each abnormal part is abnormal is as follows: Based on the obtained audiovisual feedback index of each abnormal part of the smart helmet during industrial production operation, the audiovisual feedback index of each abnormal part of the smart helmet during industrial production operation is compared with the audiovisual feedback index threshold of each abnormal part of the smart helmet during industrial production operation stored in the database. If the audiovisual feedback index of an abnormal part of the smart helmet during industrial production operation is less than the audiovisual feedback index, it means that the audiovisual feedback of the abnormal part is in an abnormal state, and a heat map is generated to determine the distribution type of the audiovisual feedback, where the distribution type includes an aggregated type and a dispersed type.

9. The human-computer interaction system based on a wearable device according to claim 1, characterized in that: The specific analysis method for starting the tactile feedback of each abnormal part is as follows: When the distribution type of visual and auditory feedback is the aggregated type, tactile feedback is activated for each abnormal part, triggering synchronous strong vibrations in each tactile feedback area. Combined with the tightening mechanism of the smart helmet, a voice reminder is triggered at the same time, forcing the user to stop the action and evacuate, and an alarm is sent to the background. When the distribution type of visual and auditory feedback is the dispersed type, tactile feedback for each abnormal part is not activated, and feedback is provided when the current task is completed.

10. The human-computer interaction system based on a wearable device according to claim 1, characterized in that: It also includes a database, wherein the database is used to store: the heat dissipation value safety interval of each part of the smart helmet during industrial production operations, the electronic component stability value safety interval, the dust coverage area safety interval, the screen freeze number safety interval, the stability index threshold of each part of the smart helmet during industrial production operations, basic information of dynamic objects, the movement reference speed of dynamic objects, the minimum reference spacing and trajectory cross-reference frequency, test abnormal sound information, and the visual and auditory feedback index threshold of each abnormal part of the smart helmet during industrial production operations.

Images