34 results about "Deaf mutes" patented technology

The invention discloses a neck-hung sign language interpretation device and a sign language semantic recognition interpretation method thereof.The neck-hung sign language interpretation device comprises a wearing support which defines a neck surrounding space and comprises a middle connecting arm and two shells, and the two shells are connected to the two sides of the middle connecting arm respectively; the image acquisition unit is arranged on the front side of the shell, comprises a three-dimensional depth camera and an adjusting mechanism for driving the three-dimensional depth camera to rotate, and is used for dynamically tracking and acquiring continuous sign language videos; the processing module is integrated in the shell, comprises a main control processor and a memory, and is used for operating a sign language recognition algorithm; the voice output unit is electrically connected with the main control processor and is used for playing the translation result; and the power supply unit supplies power to the image acquisition unit, the processing module and the voice output unit. Gesture languages used by the deaf-mute for communication are converted into natural languages which can be easily understood by normal people, the deaf-mute group is helped to speak, and the problem of information output when the deaf-mute communicates with the normal people is solved.

The invention relates to the technical field of sign language intelligent communication, in particular to a deaf-mute sign language translation pronunciation system which comprises a data acquisition and preprocessing module, a sign language model recognition module, a dynamic gesture model module and a voice conversion module. According to the sign language translation pronunciation system for the deaf-mute, wide-angle camera intelligent glasses are adopted for visual collection, wearable equipment is not needed, video streams and thermodynamic diagrams are combined, visual information and key point probability distribution are complementary, the influence of single-mode noise is reduced, video space-time features are extracted through a 3D CNN, long-range dependence is captured in combination with a self-attention mechanism of Transform, and therefore the sign language translation pronunciation system for the deaf-mute is obtained. The sign language recognition precision is improved, an internal reference matrix and a distortion coefficient are calculated through a calibration board, wide-angle lens distortion is corrected, it is ensured that hand key points are accurately positioned, YOLOv8 is adopted to segment an interference object, background noise is prevented from affecting recognition, 21 key point thermodynamic diagrams are generated through MediaPipe, fault tolerance of low-confidence-coefficient key points is enhanced through Gaussian kernel diffusion, and the recognition accuracy is improved. And the model identification precision is improved.

The invention relates to a sign language-to-multilingual text speech mutual conversion method based on deep learning and application thereof, belongs to the field of artificial intelligence, man-machine interaction and barrier-free communication, and particularly relates to a sign language recognition, semantic understanding and multilingual speech synthesis method based on deep learning and an application system thereof. The sign language to voice text conversion method comprises the following steps: 1, an image acquisition and preprocessing module, 2, a hand key point detection module, 3, time sequence modeling and sign language recognition, 4, natural language understanding and semantic optimization, 5, a multi-language translation module, and 6, a text and voice synthesis output module. According to the sign language-to-multilingual text voice mutual conversion method based on deep learning and the application thereof, sign languages can be automatically, flexibly and accurately converted into voices and characters, and deaf-mutes are helped to efficiently communicate with common people; the system can be widely applied to hospitals, schools, government affair windows, public transportation and other scenes.

The present invention discloses a data processing method and system for multimodal facial motion point data and vocal cord motion data. The method includes providing text, collecting continuous facial images or videos and laryngeal vibration data of a normal person speaking, preprocessing the data, extracting temporal and spatial features, establishing a facial and neck motion model for Chinese pronunciation, and having a deaf-mute person imitate the pronunciation according to the model and obtain feedback. The system includes a depth camera, a laryngeal vibration sensor, and a microphone. By comprehensively utilizing multimodal data, it provides instant feedback to the deaf-mute, lowers the learning threshold, and improves communication efficiency. It is applicable to deaf-mute groups worldwide. This invention promotes speech vocalization training and has broad application prospects and social significance.

The invention discloses an intelligent sign language auxiliary communication method. The method comprises the following steps: acquiring video data and audio data of an environment; aI intelligent analysis is carried out on the audio and video data, the video data and the audio data are preprocessed, the identity of each person in the environment is identified from the preprocessed data, independent voice of each person is separated from the multi-sound-source mixed audio, the spatial position of a speaker is positioned, and the timbre feature of each separated voice stream is extracted and marked; performing cross-modal matching on the character, the voice and the timbre, converting the voice into character information, generating a corresponding sign language animation according to the character information, and sending the character information and the sign language animation to the deaf-mute terminal; and sending the text information input by the deaf-mute to the speaker terminal. In a multi-person dialogue scene, the deaf-mute can clearly distinguish different speakers and corresponding speaking contents through the terminal, the real-time performance of communication is ensured, the deaf-mute can accurately understand the intention of the speaker, and the communication efficiency is improved.

The application discloses a kind of tactile perception system and its use method, tactile perception system includes: information acquisition module, analysis module and output module.The use method of tactile perception system includes: obtaining information for analysis;According to the information obtained, analysis is carried out, and the information that needs to be output to user is obtained;The information that needs to be output to user is converted into tactile perception signal, and the tactile perception signal is expressed information by contacting the position of user body, and at least one of contact type, shape, contact area, action, temperature and pressure;Output tactile perception signal.Through the tactile perception system and its use method of the application, people can better utilize tactile to perceive external information, enhance the effect of perception, and can help blind, deaf-mute or help normal people in the environment that is not conducive to vision and hearing to improve the efficiency of walking, searching, using computer and intelligent device / intelligent system etc..

The utility model discloses a legal sign language translation device and equipment, and relates to the technical field of electronic equipment, the legal sign language translation device comprises a voice module, a bending sensor, an attitude sensor and a signal processing module; the signal processing module is connected with the voice module, bending sensors arranged on all fingers on the palm and a posture sensor arranged on the palm of the user. The real-time posture of the palm of a user is collected through the posture sensor, the bending degree of each finger on the palm is collected through the bending sensor, and the electric signals are transmitted to the signal processing module, so that the signal processing module sends corresponding voice signals to the voice module; therefore, the voice module can play the voice corresponding to the legal sign language formed by the palm and the fingers at present. The legal sign language used by a user can be directly converted into voice, so that the user can quickly communicate with people who do not understand the legal sign language through played semantics, and the deaf-mute can be better helped to carry out legal consultation.

1. Name of the product in this design: Sign Language Translator. 2. Purpose of this design: A sign language interpretation device to assist deaf and mute people in daily life and meetings. 3. The key design feature of this product is its shape. 4. The image or photograph that best illustrates the design's key points: 3D view 1.

The invention relates to a deaf-mute barrier-free concert interaction system and method based on multi-modal dynamic mapping. The system comprises a wearable tactile execution unit, an AI processing unit, a multi-mode synchronous controller and a VR interaction module, and the tactile execution unit outputs multi-dimensional vibration modes to different body areas through differentially arranged linear resonance actuators. The AI processing unit maps real-time audio features into tactile parameters based on a contrast learning model and balances experience and energy consumption through the optimization module, the multi-mode synchronous controller achieves millisecond alignment of audio, vision and tactile, and the VR interaction module maps sign language input into a virtual scene and a tactile feedback adjustment instruction. Through the innovative tactile layout and the cross-modal mapping model, the immersion, rhythm perception and safety of the deaf-mute user are remarkably improved, and the method is suitable for various scenes such as concerts, education and rehabilitation.

The invention belongs to the technical field of interaction, and discloses a deaf-mute two-way interaction method, which comprises the steps of collecting voice of a dialogue person, converting the voice into first character information, visually displaying the first character information to a deaf-mute user, or converting the first character information into sign language actions or sign language schematic diagrams, displaying the sign language actions or sign language schematic diagrams to the deaf-mute user, capturing lip shape images of the dialogue person, and displaying the lip shape images to the deaf-mute user. Third character information is generated through matching based on the lip language database, the third character information is matched with the first character information to determine that the first character information is from the target dialogue person, a hand action image of the deaf-mute user is captured, the hand action image is converted into second character information, and the second character information is sent to the deaf-mute user. According to the method, the voice of the dialogue person is collected, the voice is converted into characters or sign language actions, the characters or sign language actions are visually displayed to the deaf-mute user, and the sign language actions are converted into the characters based on the sign language database by collecting the sign language actions of the deaf-mute user and are played in the form of voice.

1. The name of the design product: intelligent sign language translation machine. 2. The use of the design product: sign language translation equipment for assisting the deaf and mute to communicate and exchange in daily life and meetings. 3. The design points of the design product: in shape. 4. The picture or photo that best indicates the design points: perspective drawing.

The doorbell comprises a transmitting module, a receiving module, a function selection module and a master control module, the transmitting module is used for transmitting a radio frequency signal, the receiving module is used for outputting a wireless starting signal according to the radio frequency signal, the function selection module is used for selecting a deaf-mute function or a blind function, and the master control module is used for controlling the deaf-mute function or the blind function. And the main control module is used for executing lamp-on and horn-off operation according to the deaf-mute function and executing horn-off operation according to the blind function. According to the doorbell, different function modes are switched through the function selection module to meet the use requirements of the deaf-mute blind; for the deaf-mute, the main control module carries out the operation that the lamp is turned on and the loudspeaker does not sound, so that the bright light reminds the deaf-mute that someone comes to visit; and for the blind person, the main control module prompts the blind person to visit through loud sound by executing the operation that the loudspeaker is not turned on, so that not only is the use flexibility of the doorbell improved, but also the independent living ability and living quality of the blind person can be improved.

The invention relates to the technical field of medical first-aid equipment, and discloses a first-aid pager automatic alarm system based on multi-modal perception, the system is internally provided with eight modules, a multi-modal perception module collects user state data online, a special group data collection module pays attention to information of three types of crowds of deaf-mute, blind and cognitive impairment, and a first-aid pager alarm module is provided with a first-aid alarm module and a second-aid alarm module. The data analysis module calculates a session key, a permission compliance rate, a fault influence degree and a success rate of simplified operation of a cognitive impairment, the sensitive information encryption protection module performs end-to-end encryption on collected data, the fine-grained permission management and control module performs dynamic access control judgment, the distributed fault self-healing module performs self-healing strategy judgment, and the distributed fault self-healing module performs self-healing strategy judgment. The special crowd interaction optimization module performs interaction strategy evaluation, and the instruction execution module receives encryption, authority, self-healing and interaction optimization instructions issued by all the modules, uniformly controls underlying hardware and network resources to complete actions after priority arbitration, and returns an execution result log to form a closed loop.

The invention provides a doctor-seeing intelligent guiding system suitable for a deaf-mute, and belongs to the technical field of hospital doctor-seeing guiding, and the system comprises a doctor-seeing guiding module which obtains the appointment registration information and the real-time doctor-seeing condition of a deaf-mute patient, transmits doctor-seeing process guiding information to a user terminal device at the deaf-mute patient, and sends the doctor-seeing process guiding information to the user terminal device; marking the doctor-seeing guide information on a hospital three-dimensional map, and guiding the deaf-mute patient to see a doctor; the doctor seeing interaction module is used for collecting and recognizing sign language video images made by the deaf-mute patient, obtaining sign language recognition results, displaying the sign language recognition results to medical staff, collecting voice information or character information sent by the medical staff, and generating sign language conversion videos; and the doctor-seeing customer service module performs information interaction with the deaf-mute patient through a video interaction device, an information interaction device and a customer service robot which are arranged, and guides the deaf-mute patient to seeing a doctor smoothly. According to the invention, intelligent guide treatment for the deaf-mute patient is realized, information interaction between medical staff and the patient is optimized, the treatment effect is improved, and the treatment experience of the deaf-mute patient is improved.

According to the automatic program running device of the computer, an instruction is automatically output through voice recognition of a user and is input as a program control instruction, so that the user can automatically start a required program only by voice without operating through an icon on a display screen. Therefore, the program starting time is shortened, and an accurate program execution result can be obtained. Besides, a result obtained through a program device is integrated into a virtual environment platform linked with the 6G technology, and a user can experience a more real simulation test in a non-face-to-face virtual space, so that safer purchase compared with reality is realized, and automatic service of the whole process from shopping to distribution is obtained. Particularly, by introducing various input modes, the blind, the deaf-mute and the old can conveniently use the computer, and meanwhile, the working efficiency of an enterprise can be greatly improved by utilizing a program automation system.

The invention relates to the technical field of computer vision, in particular to a sign language recognition method and system. The method comprises the following steps: acquiring an initial sign language prediction text based on multiple continuous to-be-recognized sign language pictures of a user after preprocessing; and if the text is vacant, outputting the probability that the vacant part belongs to each candidate word through the linear projection layer, and taking the probability as the visual confidence. When the number of vacancies is smaller than a preset value, calculating semantic similarity between the candidate words and the initial sign language prediction text as text confidence; on the contrary, the initial sign language prediction text and the initial sign language prediction text corresponding to multiple frames before and after are spliced into a total prediction text, and then the semantic similarity is calculated to obtain the text confidence coefficient. And obtaining a dynamic adjustment factor in combination with the definition of the sign language picture, fusing the factor with the two confidence coefficients to obtain a target confidence coefficient, and complementing the vacancy with the highest confidence coefficient to obtain a final sign language recognition result. The sign language semantic recognition precision is greatly improved, and the immersion of interaction with the deaf-mute is enhanced.

This invention provides a sign language recognition and interaction system based on edge-cloud collaboration, belonging to the field of artificial intelligence technology. It includes an image acquisition module for acquiring images of gestures from deaf and mute individuals and extracting keyframe images from these images; a client processing module, wirelessly connected to the image acquisition module, for receiving keyframe images, extracting key hand points from the keyframe images, and transmitting the extracted key hand point sequence to the cloud; a cloud processing module for processing the transmitted key hand point sequence using a sign language recognition model, recognizing the gestures as text based on the key hand point sequence; the cloud processing module also transmits the text to the client processing module, which in turn performs voice playback of the text and simultaneously acquires the voice response from a hearing person, converting the voice response into text for display. This system improves the portability and accuracy of sign language translation, reduces production costs, and improves the quality of life for deaf and mute individuals.

The invention relates to the field of intelligent wearable equipment, and discloses a gesture recognition and remote translation bracelet for emergency help of a deaf-mute. Comprising a bracelet body, a gesture recognition module, a central processing module, a translation module, a communication module, an alarm module, a display module and a power module, the gesture recognition module comprises a plurality of miniature cameras and an inertial sensor, and a deep learning neural network model is placed in the central processing module; the hand ring body comprises a display screen assembly and two belts, the surface of one belt is provided with a plurality of buckles, one end of the other belt is provided with a hoop, and the other end of the other belt is provided with a translation module; the gesture image information acquired by the plurality of micro cameras is fused with the dynamic data acquired by the inertial sensor, so that the recognition effect on the complex emergency help-seeking gesture is better, the help-seeking intention of the deaf-mute can be quickly and accurately captured, the position information of the user is synchronously transmitted, and quick response and rescue are realized.

A luminous percussion instrument 100, such as a drum, comprises at least one light-emitting element arranged inside a hollow housing. The percussion instrument also includes at least one beating region 3 and at least one light-transmittable surface 11. An induction switch, which may be an infrared induction switch operating with a range of 5 to 50 millimetres, is arranged in each beating region. The induction switches are electrically connected to the light-emitting element(s) to control their activation in response to a user striking the beating regions. The percussion instrument may include three beating regions which each have an associated light-emitting element and where the light-emitting elements are different colours, for example red, green and blue. The percussion instrument may also include a snare with an adjusting rod. The instrument may provide multi-sensory stimulation to players, such as deaf people, learning to play instruments.

The invention relates to a vehicle-mounted interaction method and device, electronic equipment, a storage medium and a program product, and relates to the technical field of vehicle-mounted interaction.The method in one embodiment comprises the steps that a triggering instruction for triggering a vehicle-mounted interaction auxiliary mode is received, and the vehicle-mounted interaction auxiliary mode is entered in response to the triggering instruction; in the vehicle-mounted interaction auxiliary mode, a first interaction operation instruction of a driver is obtained, and first interaction content information of the first interaction operation instruction is obtained based on a preset first corresponding relation between the interaction operation instruction and the interaction content information; and outputting the first interaction content information. By adopting the scheme of the embodiment, the communication requirements of drivers with language barriers such as deaf-mute people can be met, and the vehicle-mounted interaction performance is improved.

The application provides an order reminding bracelet and a take-out order receiving assembly. The order reminding bracelet is wirelessly connected with an order receiving device. The order reminding bracelet comprises a main body and a wristband connected with the main body. The main body comprises a reminding lamp and / or a vibration assembly. One of the order reminding bracelet and the order receiving device is provided with a control device, which controls the reminding lamp to perform light reminding and / or the vibration assembly to perform vibration reminding according to take-out order information received by the order receiving device. The order reminding bracelet of the embodiment of the application can effectively improve the timely response to the take-out order through light reminding and / or vibration reminding, avoid the occurrence of the situation of missed order and delayed order receiving, and is especially suitable for the use of deaf-mute personnel to solve the problems of missed order and delayed order receiving caused by the inability to hear sound reminding due to hearing defects. Meanwhile, the order reminding bracelet is convenient to wear and carry, can ensure that the order reminding can be monitored at any time, and can avoid the occurrence of the situation of missed order and delayed order receiving.

The invention discloses a sign language translation device and method and a storage medium. The sign language translation device comprises a controller, a first display screen, a camera, a second display screen and a microphone, and the first display screen, the camera, the second display screen and the microphone are all electrically connected to the controller; wherein the first display screen faces the deaf-mute and is used for the deaf-mute to watch information; the second display screen is arranged facing the healthy hearing person and is used for the healthy hearing person to watch information; the camera is used for collecting sign language actions of the deaf-mute, the controller processes sign language information collected by the camera so that corresponding characters can be displayed on the first display screen and the second display screen at the same time, the microphone is used for collecting voice information of a healthy listener, and the controller further processes the voice information collected by the microphone so that the voice information can be displayed on the first display screen and the second display screen at the same time. And displaying the corresponding sign language video on the second display screen.

The present invention discloses a method, system, device and medium for generating a learning style portrait based on behavioral data, including: acquiring sample data, constructing a convolutional neural network model, processing first behavioral data, and obtaining cognitive emotional state recognition data; constructing a Transformer model that fuses spatiotemporal features, processing second behavioral data, and obtaining cognitive concentration recognition data; constructing a semantic recognition model, processing third behavioral data, and obtaining interactive activity recognition data; quantifying cognitive emotional state recognition data, cognitive concentration recognition data, and interactive activity recognition data to generate a behavioral data sequence, constructing a learning style coding framework, generating a learning style portrait based on the behavioral data sequence, and generating a learning style by tracking the informal learning behavior data of deaf-mute people in a venue visiting situation, thereby grasping the learning status of the deaf-mute learners and improving the informal learning interaction experience of the deaf-mute people.

The present application relates to the technical field of computer vision, and especially relates to a sign language recognition method and system. The present application obtains initial sign language prediction text based on the pre-processed continuous multiple frames of sign language pictures to be recognized of a user; if the text has a vacancy, the probability that the vacancy part belongs to each candidate word is output by a linear projection layer as visual confidence. When the number of vacancies is less than a preset value, the semantic similarity of the candidate word and the initial sign language prediction text is calculated as text confidence; otherwise, the initial sign language prediction text and the corresponding initial sign language prediction texts of the multiple frames before and after the initial sign language prediction text are spliced into a total prediction text, and the semantic similarity is calculated to obtain text confidence. A dynamic adjustment factor is obtained in combination with the clarity of the sign language picture, the target confidence is obtained by fusing the factor and the two kinds of confidence, the highest one is taken to complete the vacancy, and the final sign language recognition result is obtained. The present application greatly improves the sign language semantic recognition accuracy and enhances the immersion of interaction with the deaf-mute.

The invention relates to the technical field of intelligent equipment, in particular to a barrier-free reply system. According to the barrier-free reply system provided by the invention, the technical problem that the content output by the deaf-mute cannot be played in a sound mode when the deaf-mute has a conversation in the prior art is solved. A barrier-free reply system comprises a recognizer; a voice processor, wherein the voice processor is in communication with the recognizer; reply content data are stored in the reply content storage device; a reply content selector for selecting reply content data stored in the reply content memory; according to the application scene and the context, several replies are preset for the question of the user to be selected by the deaf-mute, after the deaf-mute selects one reply, the reply is converted into voice, and the voice is played to the other party in the form of sound through the reply content output device worn by the deaf-mute, so that the deaf-mute can effectively communicate with others.

1. Name of the product in this design: AI Translator for the Deaf and Mute. 2. Purpose of this design: This design is intended for use in interactive communication with deaf and mute individuals through AI technology. 3. The key design feature of this product is its shape. 4. The image or photograph that best illustrates the design's key points: a 3D model.

This invention discloses an accessible graphic and text-based emergency call system for deaf and mute individuals, comprising: a first emergency call signal generation module for receiving a first emergency call signal from a silent user; a first emergency call signal response module for responding to the first emergency call signal and triggering a plurality of distributed matching requests in parallel; the plurality of matching requests include: a first matching request, a second matching request, and a third matching request; a distress signal generation module for responding to the distributed matching requests to generate a plurality of distributed distress signals; a signal encapsulation module for encapsulating the plurality of distributed distress signals to generate a distress data packet; and a data packet transmission module for transmitting the distress data packet according to the current transmission frequency. The graphic and text-based emergency call system of this invention generates multi-signal encapsulated distress data packets, enabling rescuers to quickly obtain the specific situation, urgency level, and location signal of the silent user from the distress signals, thus making a more efficient rescue response.

The invention relates to the technical field of computers, in particular to a sign language recognition and display method and system in a double-screen all-in-one machine, and the method comprises the steps: collecting sign language postures made by a deaf-mute through a camera, and obtaining to-be-recognized sign language data; after preprocessing the sign language data to be recognized, inputting the sign language data to be recognized into the AI intelligent sign language recognition model to output first semantic information corresponding to the sign language data to be recognized; converting the first semantic information into voice content, and playing the voice content through a loudspeaker; collecting voice data of a user through a microphone to obtain voice data to be recognized; preprocessing the to-be-recognized voice data, and inputting the preprocessed to-be-recognized voice data into the voice automatic recognition model to output second semantic information corresponding to the to-be-recognized voice data; and generating a sign language expression model based on the second semantic information, driving the digital person to generate a sign language animation based on the sign language expression model, and displaying the sign language animation on a display screen. The method can solve the problems that in the communication process of the deaf-mute and an ordinary person, the communication efficiency is low and translation is inconvenient due to the fact that the ordinary person does not grasp sign languages generally.

The invention discloses a vehicle auxiliary driving method and device, electronic equipment, a storage medium and a vehicle, and relates to the technical field of auxiliary driving. According to the vehicle aided driving method provided by the invention, the semantic tag is obtained by fusing the acoustic feature data and the vehicle dynamic data for comprehensive semantic analysis, the semantic fuzziness problem caused by a single sound feature is avoided, and the semantic recognition accuracy in a complex scene is improved. According to the semantic label and the vehicle dynamic data, multi-sense collaborative feedback is performed through the photoelectric device and the vibration device, and the information receiving efficiency of the deaf-mute driver is improved. And the multi-sense collaborative feedback can shorten the understanding time of the deaf-mute driver on complex instructions.

The invention discloses a rescue communication method for a deaf-mute trapped elevator car, a deaf-mute rescue communication system is constructed, the system comprises a monitoring room part and an elevator part, the monitoring room part comprises a monitoring room computer, the monitoring room computer is provided with monitoring room software, and the elevator part comprises a car monitor and an elevator main control board. The lift car monitor is installed on an operation box of a lift car, the lift main control board is installed on a lift control cabinet, the lift car monitor is connected with the lift main control board through a CANBUS of a traveling cable, and lift car monitoring software is installed on the lift car monitor. The YES button, the NO button, the lift car help seeking button and the microphone are installed on the lift car operation box, and the loudspeaker and the second camera are installed on one side of the lift car operation box. When the trapped person is a deaf-mute person, rescue communication with the deaf-mute person can be carried out, so that the state of the deaf-mute trapped person can be known in time, and the deaf-mute trapped person can be pacified.