Industrial automation equipment man-machine interaction system based on wireless intelligent terminal

Abstract

The invention discloses an industrial automation equipment man-machine interaction system based on a wireless intelligent terminal, and relates to the technical field of man-machine interaction, the industrial automation equipment man-machine interaction system comprises an attention detection module, a fatigue qualitative module and an interaction adjustment module, the attention detection module is used for detecting supervision attention in the process that an operator supervises a display screen, and the fatigue qualitative module is used for determining the fatigue qualitative module; the fatigue qualitative module is used for analyzing the attention concentration degree of an operator in the process of monitoring a display screen, the interactive adjustment module is used for intelligently controlling and adjusting the cutting state according to the attention of the operator in the process of monitoring the display screen, and the attention detection module is electrically connected with the fatigue qualitative module. The fatigue qualitative module is electrically connected with the interaction adjusting module, the attention detection module comprises a thermal radiation detection module, an imaging module, a photographing module and a pupil capturing module, the thermal radiation detection module is used for emitting thermal radiation and converting the thermal radiation into an electric signal for thermal imaging, and the system has the advantage of timely response.

Description

Technical Field

[0001] The present invention relates to the technical field of human-computer interaction, and in particular to a human-computer interaction system for industrial automation equipment based on wireless intelligent terminals. Background Art

[0002] High-precision laser cutting machines use high-energy laser beams to quickly heat the workpiece, causing it to melt or vaporize in a very short time, thereby achieving the cutting effect. The laser beam is precisely controlled by the control system and cuts according to the pre-set drawing requirements.

[0003] In the prior art, by taking high-definition video of the cutting station and imaging it on the screen, workers monitor the cutting process in real time to find problems caused by material or other unexpected situations, and perform manual intervention to ensure the safety of the workpiece during laser cutting. However, due to the lengthy cutting process, workers may lose their concentration due to eye fatigue or distraction. If not discovered in time, the cutting may deviate significantly, or even cause the entire piece of material to be scrapped, resulting in a large loss. Therefore, it is necessary to design a human-computer interaction system for industrial automation equipment based on wireless intelligent terminals that can respond in a timely manner. Summary of the invention

[0004] The purpose of the present invention is to provide an industrial automation equipment human-computer interaction system based on a wireless intelligent terminal to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: an industrial automation equipment human-computer interaction system based on a wireless intelligent terminal, comprising an attention detection module, a fatigue qualitative module and an interactive adjustment module, wherein the attention detection module is used to detect the monitoring attention of the operator during the process of monitoring the display screen, the fatigue qualitative module is used to analyze the concentration level of the operator during the process of monitoring the display screen, and the interactive adjustment module is used to intelligently control and adjust the cutting state according to the attention of the operator in monitoring the display screen, the attention detection module is electrically connected to the fatigue qualitative module, and the fatigue qualitative module is electrically connected to the interactive adjustment module.

[0006] According to the above technical solution, the attention detection module includes a thermal radiation detection module, an imaging module, a photographic module, a pupil focus judgment module and a pupil capture module. The thermal radiation detection module is used to emit thermal radiation and convert it into an electrical signal for thermal imaging. The imaging module is used to image the thermal radiation detection body image of the operator. The pupil capture module is electrically connected to the photographic module. The photographic module is used to perform high-definition photography of the operator's head. The pupil capture module is used to capture and lock the operator's pupil in real time according to the imaging image of the photographic module. The pupil focus judgment module is used to judge whether the operator's pupil sight is focused on the laser cutting image of the display screen;

[0007] The fatigue qualitative module includes a sitting posture interpretation module and a concentration calculation module. The sitting posture interpretation module is electrically connected to the thermal radiation detection module, and the sitting posture interpretation module is used to analyze the sitting posture of the operator when monitoring the display screen. The concentration calculation module is electrically connected to the pupil capture module, and the concentration calculation module is used to analyze and calculate the pupil focusing degree of the operator when monitoring the display screen.

[0008] The interactive adjustment module includes an efficiency change module, a voice reminder module, and a processing pause module. The efficiency change module is used to control and adjust the laser cutting speed, the voice reminder module is used to control and adjust the voice broadcast on the display screen to remind the operator to concentrate, and the processing pause module is used to control the laser cutting machine to pause work.

[0009] According to the above technical solution, the working method of the system is:

[0010] S1: The operator sits in front of the monitoring display screen, turns on the display screen and starts to monitor the laser cutting status on the display screen. After the laser cutting machine is started, the attention detection module located at the end of the display screen is started;

[0011] S2: The attention detection module starts scanning and detecting the attention of the operator in front of the display screen;

[0012] S3: The fatigue qualitative module analyzes the operator's attention to the display screen in real time, and determines the current operator's concentration level on the display screen in real time;

[0013] S4: The interactive adjustment module adjusts the operation of the laser cutting machine according to the results of the attention analysis and provides voice reminders.

[0014] According to the above technical solution, S2 further includes the following steps:

[0015] S2-1: After the operator is in place, the thermal radiation detection module installed behind the display screen in the monitoring room detects the thermal radiation energy and converts it into electrical signals for thermal imaging, thereby obtaining the thermal imaging image of the operator;

[0016] S2-2: At the same time, the imaging module starts to shoot the picture in the direction of the workstation in front of the display screen, and transmits the obtained picture information electrical signal to the pupil capture module;

[0017] S2-3: The pupil capture module locks and captures pupil image information in the captured image according to the set pupil features.

[0018] According to the above technical solution, S3 further includes the following steps:

[0019] S3-1: The sitting posture interpretation module establishes a set monitoring sitting posture range based on the workstation position of the operator;

[0020] S3-2: Fit the operator's sitting posture contour range through the thermal imaging screen, and compare it with the established standard monitoring sitting posture range;

[0021] S3-3: Calculate the standard sitting posture of the current operator when monitoring the display screen by the ratio of the overlap range to the established standard monitoring sitting posture range;

[0022] S3-4: The concentration calculation module works when the display screen is turned on. When the pupil capture module just locks the pupil image information, the current pupil black range area is calibrated and calculated to obtain the standard pupil exposure area w;

[0023] S3-5: In the subsequent process of the operator monitoring the display screen, the concentration calculation module continuously calculates the pupil exposure area according to the image information of the pupil capture module to obtain the current pupil exposure area wt, and takes the wt measured multiple times within a certain period of time as the average value for calculation;

[0024] S3-6: The concentration calculation module uses the ratio of the current pupil exposure area wt to the quasi-pupil exposure area w, combined with whether the pupil's line of sight is in the laser cutting screen of the display screen, to calculate the real-time concentration of the operator during the process of monitoring the display screen.

[0025] According to the above technical solution, the calculation formula of the sitting posture standard of the current operator when monitoring the display screen in S3-3 is:

[0026]

[0027] Among them, p x is the overlapping area value of the operator's sitting posture contour and the standard monitoring sitting posture range, p is the established standard monitoring sitting posture range area value, R is the sitting posture standard value, and the overlapping area ratio is proportional to the sitting posture standard value. When the overlapping area ratio is larger, it means that the sitting posture is more correct and the attention is more focused when monitoring the display screen.

[0028] According to the above technical solution, the real-time concentration calculation formula of the operator monitoring the display screen in S3-6 is:

[0029]

[0030] Among them, H is the real-time concentration value of the operator during the process of monitoring the display screen. The larger the current pupil exposure area, the greater the degree of eye openness, and the more focused the attention when monitoring the display screen. Conversely, the less focused the attention, including drowsiness. When the current pupil exposure area is 0, the pupil cannot be detected, which indicates that the current operator is asleep and the degree of concentration when monitoring the display screen is 0.

[0031] According to the above technical solution, S4 further includes the following steps:

[0032] S41: When the imaging module measures that the sitting posture standard value R is less than 50%, the sitting posture interpretation module analyzes that the operator may be sitting fatigued and may leave the workstation, triggering an electrical signal to the processing pause module to control the laser cutting machine to pause working;

[0033] S42: When the imaging module measures that the sitting posture standard value R is 50% ≤ R < 80%, the sitting posture interpretation module determines that the operator may be working fatigued at this time. The pupil focus judgment module finds that if the worker's pupil sight is not focused on the laser cutting screen on the display screen, the voice reminder module is used to remind the worker to correct the posture and suspend work. If the worker's pupil sight is found to be focused, the efficiency change module is used to control the laser cutting machine to work at a slow speed v0;

[0034] S43: When the imaging module measures that the sitting posture standard value R is 80% ≤ R, the sitting posture interpretation module determines that the operator is now in a state of serious work, and the pupil focus judgment module also finds that the worker's pupil sight is also focused on the laser cutting area of ​​the display screen. At this time, the concentration calculation module is used to analyze the operator's pupil focus and adjust the laser cutting speed. The laser cutting speed v is proportional to the pupil focus, specifically v = v0 + μH, where μ is the laser cutting speed coefficient. When w t =w, the laser cutting speed v reaches its maximum.

[0035] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the present invention detects the sitting posture of the operator and compares it with the standard sitting posture, and at the same time detects the pupil line of sight and the pupil display area to determine whether the worker is concentrating on the processing at this time, and adjusts the cutting speed of the laser cutting machine according to the degree of concentration of the worker, so that the worker can perform efficient processing in the best working state, and appropriately reduces the cutting speed when the worker is tired to prevent major mistakes. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0037] Figure 1 It is a schematic diagram of the overall module structure of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] See also Figure 1 The present invention provides a technical solution: an industrial automation equipment human-computer interaction system based on a wireless intelligent terminal, comprising an attention detection module, a fatigue qualitative module and an interactive adjustment module, wherein the attention detection module is used to detect the monitoring attention of the operator during the process of monitoring the display screen, the fatigue qualitative module is used to analyze the concentration of the operator during the process of monitoring the display screen, and the interactive adjustment module is used to intelligently control and adjust the cutting state according to the operator's attention to monitoring the display screen, the attention detection module is electrically connected to the fatigue qualitative module, and the fatigue qualitative module is electrically connected to the interactive adjustment module;

[0040] The attention detection module includes a thermal radiation detection module, an imaging module, a photographic module, a pupil focus judgment module and a pupil capture module. The thermal radiation detection module is used to emit thermal radiation and convert it into an electrical signal for thermal imaging. The imaging module is used to image the thermal radiation detection body of the operator. The pupil capture module is electrically connected to the photographic module. The photographic module is used to take a high-definition photo of the operator's head. The pupil capture module is used to capture and lock the operator's pupil in real time according to the imaging picture of the photographic module. The pupil focus judgment module is used to judge whether the operator's pupil sight is focused on the laser cutting picture of the display screen.

[0041] The fatigue qualitative module includes a sitting posture interpretation module and a concentration calculation module. The sitting posture interpretation module is electrically connected to the thermal radiation detection module. The sitting posture interpretation module is used to analyze the sitting posture of the operator when monitoring the display screen. The concentration calculation module is electrically connected to the pupil capture module. The concentration calculation module is used to analyze and calculate the pupil focusing degree of the operator when monitoring the display screen.

[0042] The interactive adjustment module includes an efficiency change module, a voice reminder module, and a processing pause module. The efficiency change module is used to control and adjust the laser cutting speed. The voice reminder module is used to control and adjust the voice broadcast on the display screen to remind the operator to concentrate. The processing pause module is used to control the laser cutting machine to pause work.

[0043] The system works as follows:

[0044] S1: The operator sits in front of the monitoring display screen, turns on the display screen and starts to monitor the laser cutting status on the display screen. After the laser cutting machine is started, the attention detection module located at the end of the display screen is started;

[0045] S2: The attention detection module starts scanning and detecting the attention of the operator in front of the display screen;

[0046] S3: The fatigue qualitative module analyzes the operator's attention to the display screen in real time, and determines the current operator's concentration level on the display screen in real time;

[0047] S4: The interactive adjustment module adjusts the operation of the laser cutting machine according to the results of the attention analysis and gives voice reminders;

[0048] S2 further comprises the following steps:

[0049] S2-1: After the operator is in place, the thermal radiation detection module installed behind the display screen in the monitoring room detects the thermal radiation energy and converts it into electrical signals for thermal imaging, thereby obtaining the thermal imaging image of the operator;

[0050] S2-2: At the same time, the imaging module starts to shoot the picture in the direction of the workstation in front of the display screen, and transmits the obtained picture information electrical signal to the pupil capture module;

[0051] S2-3: The pupil capture module locks and captures pupil image information in the captured image according to the set pupil features;

[0052] S3 further comprises the following steps:

[0053] S3-1: The sitting posture interpretation module establishes a set monitoring sitting posture range based on the workstation position of the operator;

[0054] S3-2: Fit the operator's sitting posture contour range through the thermal imaging screen, and compare it with the established standard monitoring sitting posture range;

[0055] S3-3: Calculate the standard sitting posture of the current operator when monitoring the display screen by the ratio of the overlap range to the established standard monitoring sitting posture range;

[0056] S3-4: The concentration calculation module works when the display screen is turned on. When the pupil capture module just locks the pupil image information, the current pupil black range area is calibrated and calculated to obtain the standard pupil exposure area w;

[0057] S3-5: In the subsequent process of the operator monitoring the display screen, the concentration calculation module continuously calculates the pupil exposure area according to the image information of the pupil capture module to obtain the current pupil exposure area wt, and takes the wt measured multiple times within a certain period of time as the average value for calculation;

[0058] S3-6: The concentration calculation module uses the ratio of the current pupil exposure area wt to the quasi-pupil exposure area w, combined with whether the pupil's sight direction is in the laser cutting screen of the display screen, to calculate the real-time concentration of the operator during the process of monitoring the display screen;

[0059] The calculation formula for the current sitting posture standard of the operator when monitoring the display screen in S3-3 is:

[0060]

[0061] Among them, p x is the overlapped area between the operator's sitting posture contour and the standard monitoring sitting posture range, p is the established standard monitoring sitting posture range area, and R is the sitting posture standard value. The larger the overlapped area ratio is, the more correct the sitting posture is, and the more focused the attention is when monitoring the display screen.

[0062] The real-time concentration calculation formula of the operator monitoring the display screen in S3-6 is:

[0063]

[0064] Among them, H is the real-time concentration value of the operator during the process of monitoring the display screen. The larger the current pupil exposure area, the greater the degree of eye opening, and the more focused the attention when monitoring the display screen. On the contrary, the less focused the attention, including drowsiness. When the current pupil exposure area is 0, the pupil cannot be detected, indicating that the current operator is asleep, and the concentration level when monitoring the display screen is 0;

[0065] S4 further comprises the following steps:

[0066] S41: When the imaging module measures that the sitting posture standard value R is less than 50%, the sitting posture interpretation module analyzes that the operator may be sitting fatigued and may leave the workstation, triggering an electrical signal to the processing pause module to control the laser cutting machine to pause working;

[0067] S42: When the imaging module measures that the sitting posture standard value R is 50% ≤ R < 80%, the sitting posture interpretation module determines that the operator may be working fatigued at this time. The pupil focus judgment module finds that if the worker's pupil sight is not focused on the laser cutting screen on the display screen, the voice reminder module is used to remind the worker to correct the posture and suspend work. If the worker's pupil sight is found to be focused, the efficiency change module is used to control the laser cutting machine to work at a slow speed v0;

[0068] S43: When the imaging module measures that the sitting posture standard value R is 80% ≤ R, the sitting posture interpretation module determines that the operator is now in a state of serious work, and the pupil focus judgment module also finds that the worker's pupil sight is also focused on the laser cutting area of ​​the display screen. At this time, the concentration calculation module is used to analyze the operator's pupil focus and adjust the laser cutting speed. The laser cutting speed v is proportional to the pupil focus, specifically v = v0 + μH, where μ is the laser cutting speed coefficient. When w t =w, the laser cutting speed v reaches its maximum.

[0069] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0070] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the attention and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. The human-computer interaction system for industrial automation equipment based on wireless intelligent terminals is characterized by: It includes an attention detection module, a fatigue qualitative module and an interactive adjustment module. The attention detection module is used to detect the monitoring attention of the operator during the process of monitoring the display screen. The fatigue qualitative module is used to analyze the concentration level of the operator during the process of monitoring the display screen. The interactive adjustment module is used to intelligently control and adjust the cutting state according to the operator's attention to monitoring the display screen. The attention detection module is electrically connected to the fatigue qualitative module, and the fatigue qualitative module is electrically connected to the interactive adjustment module.

2. The industrial automation equipment human-computer interaction system based on wireless intelligent terminal according to claim 1 is characterized in that: The attention detection module includes a thermal radiation detection module, an imaging module, a photographic module, a pupil focus judgment module and a pupil capture module. The thermal radiation detection module is used to emit thermal radiation and convert it into an electrical signal for thermal imaging. The imaging module is used to image the thermal radiation detection body of the operator. The pupil capture module is electrically connected to the photographic module. The photographic module is used to take a high-definition photo of the operator's head. The pupil capture module is used to capture and lock the operator's pupil in real time according to the imaging picture of the photographic module. The pupil focus judgment module is used to judge whether the operator's pupil sight is focused on the laser cutting picture of the display screen; The fatigue qualitative module includes a sitting posture interpretation module and a concentration calculation module. The sitting posture interpretation module is electrically connected to the thermal radiation detection module, and the sitting posture interpretation module is used to analyze the sitting posture of the operator when monitoring the display screen. The concentration calculation module is electrically connected to the pupil capture module, and the concentration calculation module is used to analyze and calculate the pupil focusing degree of the operator when monitoring the display screen. The interactive adjustment module includes an efficiency change module, a voice reminder module, and a processing pause module. The efficiency change module is used to control and adjust the laser cutting speed, the voice reminder module is used to control and adjust the voice broadcast on the display screen to remind the operator to concentrate, and the processing pause module is used to control the laser cutting machine to pause work.

3. The industrial automation equipment human-computer interaction system based on wireless intelligent terminal according to claim 2 is characterized in that: The system works as follows: S1: The operator sits in front of the monitoring display screen, turns on the display screen and starts to monitor the laser cutting status on the display screen. After the laser cutting machine is started, the attention detection module located at the end of the display screen is started; S2: The attention detection module starts scanning and detecting the attention of the operator in front of the display screen; S3: The fatigue qualitative module analyzes the operator's attention to the display screen in real time, and determines the current operator's concentration level on the display screen in real time; S4: The interactive adjustment module adjusts the operation of the laser cutting machine according to the results of the attention analysis and provides voice reminders.

4. The industrial automation equipment human-computer interaction system based on wireless intelligent terminal according to claim 3 is characterized in that: The S2 further comprises the following steps: S2-1: After the operator is in place, the thermal radiation detection module installed behind the display screen in the monitoring room detects the thermal radiation energy and converts it into electrical signals for thermal imaging, thereby obtaining the thermal imaging image of the operator; S2-2: At the same time, the imaging module starts to shoot the picture in the direction of the workstation in front of the display screen, and transmits the obtained picture information electrical signal to the pupil capture module; S2-3: The pupil capture module locks and captures pupil image information in the captured image according to the set pupil features.

5. The industrial automation equipment human-computer interaction system based on wireless intelligent terminal according to claim 4 is characterized in that: The S3 further comprises the following steps: S3-1: The sitting posture interpretation module establishes a set monitoring sitting posture range based on the workstation position of the operator; S3-2: Fit the operator's sitting posture contour range through the thermal imaging screen, and compare it with the established standard monitoring sitting posture range; S3-3: Calculate the standard sitting posture of the current operator when monitoring the display screen by the ratio of the overlap range to the established standard monitoring sitting posture range; S3-4: The concentration calculation module works when the display screen is turned on. When the pupil capture module just locks the pupil image information, the current pupil black range area is calibrated and calculated to obtain the standard pupil exposure area w; S3-5: During the subsequent monitoring of the display screen by the operator, the concentration calculation module continuously calculates the pupil exposure area according to the image information of the pupil capture module to obtain the current pupil exposure area w t , take the w measured multiple times within a certain period of time t Perform average processing for calculation; S3-6: The concentration calculation module uses the current pupil exposure area w t The real-time concentration of the operator during monitoring the display screen is calculated by combining the ratio of the pupil exposure area w to the quasi-pupil exposure area w and whether the pupil's sight direction is on the laser cutting screen of the display screen.

6. The industrial automation equipment human-computer interaction system based on wireless intelligent terminal according to claim 5 is characterized in that: The calculation formula for the sitting posture standard of the current operator when monitoring the display screen in S3-3 is: Among them, p x is the overlapping area value of the operator's sitting posture contour and the standard monitoring sitting posture range, p is the established standard monitoring sitting posture range area value, R is the sitting posture standard value, and the overlapping area ratio is proportional to the sitting posture standard value. When the overlapping area ratio is larger, it means that the sitting posture is more correct and the attention is more focused when monitoring the display screen.

7. The industrial automation equipment human-computer interaction system based on wireless intelligent terminal according to claim 6 is characterized in that: The real-time concentration calculation formula during the operator monitoring the display screen in S3-6 is: Among them, H is the real-time concentration value of the operator during the process of monitoring the display screen. The larger the current pupil exposure area, the greater the degree of eye openness, and the more focused the attention when monitoring the display screen. Conversely, the less focused the attention, including drowsiness. When the current pupil exposure area is 0, the pupil cannot be detected, which indicates that the current operator is asleep and the degree of concentration when monitoring the display screen is 0.

8. The industrial automation equipment human-computer interaction system based on wireless intelligent terminal according to claim 7 is characterized in that: The S4 further comprises the following steps: S41: When the imaging module measures that the sitting posture standard value R is less than 50%, the sitting posture interpretation module analyzes that the operator may be sitting fatigued and may leave the workstation, triggering an electrical signal to the processing pause module to control the laser cutting machine to pause working; S42: When the imaging module measures that the sitting posture standard value R is 50% ≤ R < 80%, the sitting posture interpretation module determines that the operator may be working fatigued at this time. The pupil focus judgment module finds that if the worker's pupil sight is not focused on the laser cutting screen on the display screen, the voice reminder module is used to remind the worker to correct the posture and suspend work. If the worker's pupil sight is found to be focused, the efficiency change module is used to control the laser cutting machine to work at a slow speed v0; S43: When the imaging module measures that the sitting posture standard value R is 80% ≤ R, the sitting posture interpretation module determines that the operator is now in a state of serious work, and the pupil focus judgment module also finds that the worker's pupil sight is also focused on the laser cutting area of ​​the display screen. At this time, the concentration calculation module is used to analyze the operator's pupil focus and adjust the laser cutting speed. The laser cutting speed v is proportional to the pupil focus, specifically v = v0 + μH, where μ is the laser cutting speed coefficient. When w t =w, the laser cutting speed v reaches its maximum.

Images