Multi-scene adaptive sign language translator and control method thereof

Abstract

The invention discloses a multi-scene adaptive sign language translator and a control method thereof. The multi-scene adaptive sign language translator comprises a base, a rotating mechanism is mounted at the top of the base, a lifting mechanism is mounted at the top of the rotating mechanism, and a translation mechanism capable of capturing in multiple directions is mounted at the top of the lifting mechanism. While the monitoring probe is used for shooting, the position of a sign language person can be monitored in real time through the infrared sensor, a monitored sensing signal is transmitted to the PLC of the control module, the servo motor is controlled to operate through the PLC, and therefore the rotating seat can be driven to rotate in the circumferential direction, the shooting angle of the monitoring probe can be adjusted, and the shooting accuracy of the sign language person is improved. Meanwhile, a first forward and reverse rotation motor can be controlled to operate through a PLC, the first forward and reverse rotation motor drives a threaded column to rotate, and a lifting cylinder can be driven to perform lifting adjustment under the action of threads of a threaded sleeve, so that the camera shooting height of the monitoring probe can be adjusted, and then the camera shooting device can be suitable for multi-scene camera shooting.

Description

Technical Field

[0001] The present invention relates to the technical field of sign language translators, and in particular to a multi-scenario adaptable sign language translator and a control method thereof. Background Art

[0002] A sign language translator generally refers to UNI's sign language-audio-text conversion device, which can recognize sign language and translate it into voice, and can also convert the other party's voice into text and display it on the screen. The sign language translator will help people with speech and hearing impairments to communicate with ordinary people in a two-way manner. The display screen and camera of the sign language recognition interactive translator are easily contaminated by dust during use, which will affect the user's normal use. However, the existing sign language translator is not equipped with a dust removal device, which is not conducive to improving the dust removal performance of the sign language translator.

[0003] For example, a Chinese patent with a publication (announcement) number of CN220600881U discloses a sign language translator for the deaf-mute, including a base, a translator body and an adjustment mechanism, the adjustment mechanism including a drive assembly, two moving cylinders, two inclined rods, a transmission plate and a mounting frame, sign language translation can be performed through the translator body, when the height of the translator body needs to be adjusted, the drive assembly is started, the drive assembly drives the two moving cylinders to approach each other, the two moving cylinders drive the two inclined rods to rotate, when the angle between the two moving cylinders and the two inclined rods becomes larger, the two inclined rods drive the transmission plate to move to a side away from the base, the transmission plate drives the mounting frame to slide to a side away from the base, when the translator body on the mounting frame moves to a preset position, the height adjustment of the translator body can be completed, thereby solving the problem that the height of the sign language translator is fixed and cannot be adjusted according to the actual situation of the user.

[0004] Although the above technical solution solves the corresponding technical problem, the sign language translator in the above technology is not convenient for multi-directional photography, thereby reducing the applicability of the sign language translator. Summary of the invention

[0005] The purpose of the present invention is to provide a sign language translator with multi-scene adaptability and a control method thereof. First, the sign language is photographed in real time by a monitoring probe, and then processed by a control module built into the body, and displayed on a display screen and broadcast by a loudspeaker, so that convenient translation of the sign language can be achieved. At the same time, while the monitoring probe is photographed, the position of the signer can be monitored in real time by an infrared sensor, and the monitored sensing signal is transmitted to a PLC controller of the control module. The servo motor is controlled by the PLC controller to operate, so that the rotating seat can be driven to rotate circumferentially, so that the camera angle of the monitoring probe can be adjusted. At the same time, the first forward and reverse motor can be controlled by the PLC controller to operate, and the first forward and reverse motor drives the threaded column to rotate, and under the action of the thread of the threaded sleeve, the lifting cylinder can be driven to lift and lower, so that the camera height of the monitoring probe can be adjusted, and thus it can be suitable for multi-scene photography. At the same time, when the monitoring probe is stained with dust, it only needs to drive the arc cleaning plate to rotate back and forth continuously by the second forward and reverse motor, so that the dust on the surface of the monitoring probe can be cleaned by the arc cleaning brush, so as to ensure the clarity of the monitoring probe photography, thereby greatly improving the practicality of the sign language translator of the present invention.

[0006] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0007] A multi-scenario adaptive sign language translator and a control method thereof, comprising:

[0008] A base is provided with a rotating mechanism on the top of the base, a lifting mechanism is provided on the top of the rotating mechanism, and a translation mechanism capable of capturing in multiple directions is provided on the top of the lifting mechanism.

[0009] The above-mentioned sign language translator with multi-scenario adaptability, wherein the rotating mechanism includes a servo motor fixedly connected to the top of the base, and a rotating seat is fixedly connected to the output shaft of the servo motor.

[0010] The above-mentioned multi-scenario adaptable sign language translator, wherein the lifting mechanism includes a mounting shell fixedly connected to the top of the rotating seat, a first forward and reverse motor is fixedly connected to the center of the bottom of the inner wall of the mounting shell, a threaded column is fixedly connected to the output shaft of the first forward and reverse motor, a threaded sleeve is threadedly connected to the threaded column, a lifting cylinder is fixedly connected to the top of the threaded sleeve, the top of the lifting cylinder movably passes through the mounting shell and extends to the outside of the mounting shell, a display screen is installed on the front of the mounting shell, and speakers are installed on the front of the mounting shell and on both sides of the display screen.

[0011] The above-mentioned multi-scenario adaptable sign language translator, wherein the translation mechanism includes a base fixedly connected to the top of the lifting cylinder, a body is installed on the top of the base, a monitoring probe is installed on the front of the body, a cleaning component for cleaning the monitoring probe is installed on the body and at a position corresponding to the monitoring probe, and an infrared sensor is installed on the front of the base.

[0012] The above-mentioned multi-scenario adaptable sign language translator, wherein the cleaning component includes a mounting bracket fixedly connected to the body, a second forward and reverse motor is fixedly connected to the side of the mounting bracket, an arc-shaped cleaning plate is fixedly connected to the output shaft of the second forward and reverse motor, and an arc-shaped cleaning brush for cleaning dust on the surface of the monitoring probe is fixedly connected to the inner wall of the arc-shaped cleaning plate.

[0013] The above-mentioned sign language translator with adaptability to multiple scenarios, wherein the body is also fixedly connected to a limit seat, a side of the limit seat is rotatably mounted with a limit shaft via a bearing, and a side of the limit shaft close to the arc-shaped cleaning plate is fixedly connected to the side of the arc-shaped cleaning plate.

[0014] The above-mentioned sign language translator with multi-scenario adaptability, wherein both sides of the threaded sleeve are fixedly connected to limit sliders, and both sides of the inner wall of the mounting shell and corresponding to the position of the limit slider are provided with limit sliding grooves matching the limit sliding groove, and the side of the limit sliding groove close to the limit sliding groove moves through the limit sliding groove and extends to the interior of the limit sliding groove, and is slidably connected to the inner wall of the limit sliding groove.

[0015] The above-mentioned sign language translator with adaptability to multiple scenarios, wherein a plurality of equally spaced limiting pillars are fixedly connected to the edge of the bottom of the rotating seat, support balls are installed at the bottom of the limiting pillars, and the bottom of the limiting pillars is rollingly connected to the top of the base through the support balls.

[0016] The control method of the above-mentioned multi-scenario adaptive sign language translator includes a control module built into the body, and the control module includes a PLC controller, an image preprocessing module, a feature extraction module, a motion recognition module and a text display module.

[0017] In the control method of the above-mentioned multi-scenario adaptive sign language translator, the image preprocessing module is used to first convert the acquired color image into a grayscale image to reduce the amount of calculation, and then detect the approximate position of the hand to crop the image, retain only the hand area, and crop other irrelevant areas;

[0018] The feature extraction module is used to extract the contour features of the hand. The feature extraction module can use the cv2.findContours function to find the contour of the hand. At the same time, it can also use the pre-trained hand key point detection model to extract the key point coordinates of the fingers;

[0019] The action recognition module is used to compare the extracted features with a pre-established sign language action feature library, which contains feature data corresponding to various sign language actions. For example, a simple fist-clenching action may correspond to a specific hand contour shape and finger key point position relationship. The similarity between the features can be calculated to determine whether the current action matches a certain sign language action.

[0020] The text display module is used to convert the meaning corresponding to the recognized sign language action into text content, display it on the display screen through the graphics library, perform speech synthesis by calling SAPI, and broadcast it through the speaker.

[0021] Compared with the prior art, the advantages and positive effects of the present invention are:

[0022] 1. The present invention firstly records the sign language in real time through the monitoring probe, and then processes it through the built-in control module of the body, and displays it through the display screen and broadcasts it through the loudspeaker, so as to realize the convenient translation of the sign language. At the same time, while recording through the monitoring probe, the position of the signer can be monitored in real time through the infrared sensor, and the monitored sensing signal is transmitted to the PLC controller of the control module. The servo motor is controlled by the PLC controller to drive the rotating seat to rotate circumferentially, so that the camera angle of the monitoring probe can be adjusted. At the same time, the first forward and reverse motor can be controlled by the PLC controller to drive the threaded column to rotate, and under the action of the thread of the threaded sleeve, the lifting cylinder can be driven to be raised and lowered, so the camera height of the monitoring probe can be adjusted, and it can be suitable for multi-scene camera.

[0023] 2. In the present invention, when the monitoring probe is stained with dust, it is only necessary to drive the arc-shaped cleaning plate to rotate back and forth continuously through the second forward and reverse motor, so that the dust on the surface of the monitoring probe can be swept by the arc-shaped cleaning brush, thereby ensuring the clarity of the monitoring probe's camera, thereby greatly improving the practicality of the sign language translator of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0025] Figure 1A schematic diagram of the structure of a multi-scenario adaptable sign language translator and a control method thereof of the present invention;

[0026] Figure 2 It is a cross-sectional structural schematic diagram of a multi-scenario adaptable sign language translator and a control method thereof of the present invention;

[0027] Figure 3 It is a structural schematic diagram of a rotating mechanism in a sign language translator with multi-scenario adaptability and a control method thereof of the present invention;

[0028] Figure 4 It is a structural schematic diagram of the lifting mechanism in the multi-scenario adaptable sign language translator and the control method thereof of the present invention;

[0029] Figure 5 It is a cross-sectional structural schematic diagram of the lifting mechanism in the multi-scenario adaptable sign language translator and the control method thereof of the present invention;

[0030] Figure 6 It is a structural schematic diagram of a translation mechanism in a sign language translator with multi-scenario adaptability and a control method thereof of the present invention;

[0031] Figure 7 This is a schematic structural diagram of the cleaning component in the multi-scenario adaptable sign language translator and the control method thereof of the present invention.

[0032] Description of Figure Numbers:

[0033] 1. Base; 2. Rotating mechanism; 3. Lifting mechanism; 4. Translation mechanism;

[0034] 201, servo motor; 202, rotating seat; 203, limiting pillar; 2031, supporting ball;

[0035] 301, mounting shell; 302, first forward and reverse motor; 303, threaded column; 304, threaded sleeve; 305, lifting cylinder; 306, display screen; 307, speaker; 308, limit slider; 3081, limit slide groove;

[0036] 401, machine base; 402, machine body; 403, monitoring probe; 404, cleaning component; 405, infrared sensor;

[0037] 4041, mounting frame; 4042, second forward and reverse motor; 4043, arc cleaning plate; 4044, arc cleaning brush; 4045, limit seat; 40451, limit rotating shaft. DETAILED DESCRIPTION

[0038] The following will describe the implementation methods of the present application in detail with the help of accompanying drawings and examples, so that the implementation process of how the present application uses technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0039] Example 1

[0040] Please refer to Figures 1 to 7 As shown, an embodiment of the present invention provides a multi-scenario adaptive sign language translator and a control method thereof, comprising: a base 1, a rotating mechanism 2 is installed on the top of the base 1, a lifting mechanism 3 is installed on the top of the rotating mechanism 2, and a translation mechanism 4 capable of multi-directional capture is installed on the top of the lifting mechanism 3.

[0041] Please refer to Figures 1 to 7 As shown, the rotating mechanism 2 includes a servo motor 201 fixedly connected to the top of the base 1 , and a rotating base 202 is fixedly connected to the output shaft of the servo motor 201 .

[0042] Please refer to Figures 1 to 7 As shown, the lifting mechanism 3 includes a mounting shell 301 fixedly connected to the top of the rotating seat 202, a first forward and reverse motor 302 is fixedly connected to the center of the bottom of the inner wall of the mounting shell 301, a threaded column 303 is fixedly connected to the output shaft of the first forward and reverse motor 302, a threaded sleeve 304 is threadedly connected to the threaded column 303, a lifting cylinder 305 is fixedly connected to the top of the threaded sleeve 304, the top of the lifting cylinder 305 movably passes through the mounting shell 301 and extends to the outside of the mounting shell 301, a display screen 306 is installed on the front of the mounting shell 301, and speakers 307 are installed on the front of the mounting shell 301 and on both sides of the display screen 306.

[0043] Please refer to Figures 1 to 7 As shown, the translation mechanism 4 includes a base 401 fixedly connected to the top of the lifting cylinder 305, a body 402 is installed on the top of the base 401, a monitoring probe 403 is installed on the front of the body 402, a cleaning component 404 for cleaning the monitoring probe 403 is installed on the body 402 and at a position corresponding to the monitoring probe 403, and an infrared sensor 405 is installed on the front of the base 401.

[0044] Please refer to Figures 1 to 7 As shown, the cleaning component 404 includes a mounting frame 4041 fixedly connected to the body 402, a second forward and reverse motor 4042 is fixedly connected to the side of the mounting frame 4041, an arc-shaped cleaning plate 4043 is fixedly connected to the output shaft of the second forward and reverse motor 4042, and an arc-shaped cleaning brush 4044 for cleaning dust on the surface of the monitoring probe 403 is fixedly connected to the inner wall of the arc-shaped cleaning plate 4043.

[0045] Please refer to Figures 1 to 7 As shown, a limit seat 4045 is also fixedly connected to the body 402 , and a limit shaft 40451 is rotatably installed on the side of the limit seat 4045 through a bearing. The side of the limit shaft 40451 close to the arc cleaning plate 4043 is fixedly connected to the side of the arc cleaning plate 4043 .

[0046] Please refer to Figures 1 to 7 As shown, both sides of the threaded sleeve 304 are fixedly connected to the limiting slider 308, and the limiting grooves 3081 matching the limiting slider 308 are opened on both sides of the inner wall of the mounting shell 301 and corresponding to the position of the limiting slider 308, and the side of the limiting slider 308 close to the limiting groove 3081 moves through the limiting groove 3081 and extends to the interior of the limiting groove 3081, and is slidably connected to the inner wall of the limiting groove 3081.

[0047] Please refer to Figures 1 to 7 As shown, a plurality of equally spaced limiting pillars 203 are fixedly connected to the edge of the bottom of the rotating seat 202 , a supporting ball 2031 is installed at the bottom of the limiting pillar 203 , and the bottom of the limiting pillar 203 is rollingly connected to the top of the base 1 through the supporting ball 2031 .

[0048] Example 2

[0049] Please refer to Figures 1 to 7 As shown, a control method of a sign language translator with multi-scenario adaptability includes a control module built into a body 402, and the control module includes a PLC controller, an image preprocessing module, a feature extraction module, a motion recognition module, and a text display module.

[0050] Please refer to Figures 1 to 7 As shown, the image preprocessing module is used to obtain an image from the monitoring probe 403, first convert the obtained color image into a grayscale image to reduce the amount of calculation, and then crop the image by detecting the approximate position of the hand, retaining only the hand area and cropping other irrelevant areas;

[0051] The feature extraction module is used to extract the contour features of the hand. The feature extraction module can use the cv2.findContours function to find the contour of the hand. At the same time, it can also use the pre-trained hand key point detection model to extract the key point coordinates of the fingers;

[0052] The action recognition module is used to compare the extracted features with the pre-established sign language action feature library. This feature library contains feature data corresponding to various sign language actions. For example, a simple fist-clenching action may correspond to a specific hand contour shape and finger key point position relationship. The similarity between the features can be calculated to determine whether the current action matches a certain sign language action.

[0053] The text display module is used to convert the meaning corresponding to the recognized sign language action into text content, display it on the display screen 306 through the graphics library, perform speech synthesis by calling SAPI, and broadcast it through the speaker 307.

[0054] The working principle of the present invention is: first, the sign language is photographed in real time through the monitoring probe 403, and then processed through the control module built into the body 402, and displayed through the display screen 306 and broadcasted by the speaker 307, so as to realize the convenient translation of the sign language. At the same time, while the monitoring probe 403 is photographing, the position of the signer can be monitored in real time through the infrared sensor 405, and the monitored sensing signal is transmitted to the PLC controller of the control module. The servo motor 201 is controlled by the PLC controller to drive the rotating seat 202 to rotate in a circumferential direction, so that the camera angle of the monitoring probe 403 can be adjusted. At the same time, the PLC controller can be used to control the position of the signer in real time. The first forward and reverse motor 302 is controlled to operate, and the first forward and reverse motor 302 drives the threaded column 303 to rotate, and under the action of the thread of the threaded sleeve 304, it can drive the lifting cylinder 305 to be raised and lowered, so that the camera height of the monitoring probe 403 can be adjusted, and thus it can be suitable for multi-scene camera shooting. At the same time, when the monitoring probe 403 is stained with dust, it is only necessary to drive the arc cleaning plate 4043 to rotate back and forth continuously through the second forward and reverse motor 4042, so that the dust on the surface of the monitoring probe 403 can be cleaned by the arc cleaning brush 4044, thereby ensuring the clarity of the camera shooting of the monitoring probe 403, thereby greatly improving the practicality of the sign language translator of the present invention.

[0055] The above description shows and describes several preferred embodiments of the present invention, but as before, it should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the invention concept described herein through the above teachings or the technology or knowledge of the relevant field. And the changes and modifications made by those skilled in the art do not depart from the spirit and scope of the present invention, and should be within the scope of protection of the claims attached to the present invention.

Claims

1. A sign language translator with multi-scenario adaptability, comprising a base (1), characterized in that: A rotating mechanism (2) is installed on the top of the base (1), a lifting mechanism (3) is installed on the top of the rotating mechanism (2), and a translation mechanism (4) capable of multi-directional capture is installed on the top of the lifting mechanism (3).

2. The multi-scenario adaptive sign language translator according to claim 1, characterized in that: The rotating mechanism (2) comprises a servo motor (201) fixedly connected to the top of the base (1), and a rotating seat (202) is fixedly connected to the output shaft of the servo motor (201).

3. The multi-scenario adaptive sign language translator according to claim 2, characterized in that: The lifting mechanism (3) comprises a mounting shell (301) fixedly connected to the top of the rotating seat (202); a first forward and reverse motor (302) is fixedly connected to the center of the bottom of the inner wall of the mounting shell (301); a threaded column (303) is fixedly connected to the output shaft of the first forward and reverse motor (302); a threaded sleeve (304) is threadedly connected to the threaded column (303); a lifting cylinder (305) is fixedly connected to the top of the threaded sleeve (304); the top of the lifting cylinder (305) movably passes through the mounting shell (301) and extends to the outside of the mounting shell (301); a display screen (306) is installed on the front of the mounting shell (301); and speakers (307) are installed on the front of the mounting shell (301) and on both sides of the display screen (306).

4. The multi-scenario adaptive sign language translator according to claim 3, characterized in that: The translation mechanism (4) includes a base (401) fixedly connected to the top of the lifting cylinder (305), a body (402) is installed on the top of the base (401), a monitoring probe (403) is installed on the front of the body (402), a cleaning component (404) for cleaning the monitoring probe (403) is installed on the body (402) and at a position corresponding to the monitoring probe (403), and an infrared sensor (405) is installed on the front of the base (401).

5. The multi-scenario adaptive sign language translator according to claim 4, characterized in that: The cleaning component (404) comprises a mounting frame (4041) fixedly connected to the body (402); a second forward and reverse motor (4042) is fixedly connected to the side of the mounting frame (4041); an arc-shaped cleaning plate (4043) is fixedly connected to the output shaft of the second forward and reverse motor (4042); and an arc-shaped cleaning brush (4044) for cleaning dust on the surface of the monitoring probe (403) is fixedly connected to the inner wall of the arc-shaped cleaning plate (4043).

6. The multi-scenario adaptive sign language translator according to claim 5, characterized in that: The body (402) is also fixedly connected to a limit seat (4045), and a limit shaft (40451) is rotatably mounted on the side of the limit seat (4045) via a bearing, and the side of the limit shaft (40451) close to the arc-shaped cleaning plate (4043) is fixedly connected to the side of the arc-shaped cleaning plate (4043).

7. The multi-scenario adaptive sign language translator according to claim 6, characterized in that: The threaded sleeve (304) is fixedly connected to a limiting slider (308) on both sides, and limiting grooves (3081) matching the limiting slider (308) are provided on both sides of the inner wall of the mounting shell (301) and at positions corresponding to the limiting slider (308). The side of the limiting slider (308) close to the limiting groove (3081) moves through the limiting groove (3081) and extends to the interior of the limiting groove (3081) and is slidably connected to the inner wall of the limiting groove (3081).

8. The multi-scenario adaptive sign language translator according to claim 7, characterized in that: A plurality of equally spaced limiting pillars (203) are fixedly connected to the edge of the bottom of the rotating seat (202), a supporting ball (2031) is installed at the bottom of the limiting pillar (203), and the bottom of the limiting pillar (203) is rollingly connected to the top of the base (1) via the supporting ball (2031).

9. A control method for the multi-scenario adaptive sign language translator according to claim 8, characterized in that: It comprises a control module built into the body (402), wherein the control module comprises a PLC controller, an image preprocessing module, a feature extraction module, an action recognition module and a text display module.

10. The control method of a multi-scenario adaptive sign language translator according to claim 9, characterized in that: The image preprocessing module is used to obtain an image from the monitoring probe (403), first convert the obtained color image into a grayscale image to reduce the amount of calculation, and then cut the image by detecting the approximate position of the hand, retaining only the hand area and cutting out other irrelevant areas; The feature extraction module is used to extract the contour features of the hand. The feature extraction module can use the cv2.findContours function to find the contour of the hand, and can also use the pre-trained hand key point detection model to extract the key point coordinates of the fingers; The motion recognition module is used to compare the extracted features with a pre-established sign language motion feature library, which contains feature data corresponding to various sign language motions; The text display module is used to convert the meaning corresponding to the recognized sign language action into text content, display it on the display screen (306) through the graphics library, perform speech synthesis by calling SAPI, and broadcast it through the speaker (307).

Images