49 results about "Deaf mutes" patented technology

The invention discloses a neck-hung sign language interpretation device and a sign language semantic recognition interpretation method thereof.The neck-hung sign language interpretation device comprises a wearing support which defines a neck surrounding space and comprises a middle connecting arm and two shells, and the two shells are connected to the two sides of the middle connecting arm respectively; the image acquisition unit is arranged on the front side of the shell, comprises a three-dimensional depth camera and an adjusting mechanism for driving the three-dimensional depth camera to rotate, and is used for dynamically tracking and acquiring continuous sign language videos; the processing module is integrated in the shell, comprises a main control processor and a memory, and is used for operating a sign language recognition algorithm; the voice output unit is electrically connected with the main control processor and is used for playing the translation result; and the power supply unit supplies power to the image acquisition unit, the processing module and the voice output unit. Gesture languages used by the deaf-mute for communication are converted into natural languages which can be easily understood by normal people, the deaf-mute group is helped to speak, and the problem of information output when the deaf-mute communicates with the normal people is solved.

The invention relates to the technical field of sign language intelligent communication, in particular to a deaf-mute sign language translation pronunciation system which comprises a data acquisition and preprocessing module, a sign language model recognition module, a dynamic gesture model module and a voice conversion module. According to the sign language translation pronunciation system for the deaf-mute, wide-angle camera intelligent glasses are adopted for visual collection, wearable equipment is not needed, video streams and thermodynamic diagrams are combined, visual information and key point probability distribution are complementary, the influence of single-mode noise is reduced, video space-time features are extracted through a 3D CNN, long-range dependence is captured in combination with a self-attention mechanism of Transform, and therefore the sign language translation pronunciation system for the deaf-mute is obtained. The sign language recognition precision is improved, an internal reference matrix and a distortion coefficient are calculated through a calibration board, wide-angle lens distortion is corrected, it is ensured that hand key points are accurately positioned, YOLOv8 is adopted to segment an interference object, background noise is prevented from affecting recognition, 21 key point thermodynamic diagrams are generated through MediaPipe, fault tolerance of low-confidence-coefficient key points is enhanced through Gaussian kernel diffusion, and the recognition accuracy is improved. And the model identification precision is improved.

The invention relates to a sign language-to-multilingual text speech mutual conversion method based on deep learning and application thereof, belongs to the field of artificial intelligence, man-machine interaction and barrier-free communication, and particularly relates to a sign language recognition, semantic understanding and multilingual speech synthesis method based on deep learning and an application system thereof. The sign language to voice text conversion method comprises the following steps: 1, an image acquisition and preprocessing module, 2, a hand key point detection module, 3, time sequence modeling and sign language recognition, 4, natural language understanding and semantic optimization, 5, a multi-language translation module, and 6, a text and voice synthesis output module. According to the sign language-to-multilingual text voice mutual conversion method based on deep learning and the application thereof, sign languages can be automatically, flexibly and accurately converted into voices and characters, and deaf-mutes are helped to efficiently communicate with common people; the system can be widely applied to hospitals, schools, government affair windows, public transportation and other scenes.

The utility model relates to an intelligent auxiliary glasses system for a deaf-mute. The intelligent auxiliary glasses system comprises a central control module, a sign language detection module, a voice playing module, an environment detection module, a sound detection module, a display module and a vibration module, the central control module is connected with the sign language detection module, the voice playing module, the environment detection module, the sound detection module, the display module and the vibration module. The intelligent auxiliary glasses system provided by the utility model has comprehensive functions, and can fully consider the safety problem of deaf-mute people going out.

The invention discloses a deaf-mute sign language recognition method, device and equipment based on multiple modes and a medium. The method comprises the steps of obtaining a static image or a dynamic video of a deaf-mute sign language; for the static image, a static sign language recognition network based on a ViT model is adopted, and static sign languages of the deaf-mute are classified and recognized; for the dynamic video, a dynamic sign language recognition network is adopted for classification and recognition; the dynamic sign language recognition network firstly carries out key frame extraction on an original continuous sign language video; then, a background removing module is used for removing the background in the video frame sequence; an ACN encoder jointly constructed by convolution, space attention and LSTM is used to extract space-time fusion features; and finally, constructing an ACN decoder by combining a time attention mechanism and an LSTM, and carrying out decoding learning on the space-time fusion features. The sign language recognition method effectively solves the problems of complex background interference, redundant frame processing and the like in sign language recognition, and has high recognition precision and stability.

The invention discloses a sign language recognition translation model training method based on deep learning and a large model, and relates to the technical field of sign language recognition. Converting the sign language video into a sign language image; the sign language image comprises a static sign language image and a dynamic sign language image; extracting static frame images from the dynamic sign language image, and performing weighted stacking on the static frame images to obtain a weighted stacking image; inputting the static sign language image and the weighted superposition image into a pre-constructed convolutional neural network to obtain a discrete sign language; inputting the discrete sign language into the large language model for synthesis to obtain a complete sign language; and verifying the convolutional neural network according to the complete sign language, and obtaining a final sign language recognition translation model. The sign language recognition translation model trained by using the method provided by the invention can quickly and accurately recognize the sign language information of the deaf-mute, and the communication effect with the deaf-mute is better improved.

The present invention discloses a data processing method and system for multimodal facial motion point data and vocal cord motion data. The method includes providing text, collecting continuous facial images or videos and laryngeal vibration data of a normal person speaking, preprocessing the data, extracting temporal and spatial features, establishing a facial and neck motion model for Chinese pronunciation, and having a deaf-mute person imitate the pronunciation according to the model and obtain feedback. The system includes a depth camera, a laryngeal vibration sensor, and a microphone. By comprehensively utilizing multimodal data, it provides instant feedback to the deaf-mute, lowers the learning threshold, and improves communication efficiency. It is applicable to deaf-mute groups worldwide. This invention promotes speech vocalization training and has broad application prospects and social significance.

The invention provides a control method and device based on deaf-mute travel auxiliary vision glasses and a medium, and belongs to the technical field of auxiliary vision. The method comprises the following steps: acquiring current first image information, and detecting whether a gesture action exists or not according to a set gesture recognition model; if yes, the current first image information is recognized according to the set gesture recognition model, single isolated word information is extracted and obtained, and the isolated word information is added to the cache annular queue one by one according to the time sequence. And when the quantity of the isolated word information is greater than the set quantity threshold value, correspondingly reading the context information stored in the current storage text. And processing the isolated word information and the context information according to the set semantic integration large model, capturing the context information, carrying out semantic integration on the isolated word information according to the context information and the context information, and displaying an obtained output statement. The data acquisition accuracy, the sign language recognition translation accuracy and the user experience can be improved.

The invention discloses an intelligent sign language auxiliary communication method. The method comprises the following steps: acquiring video data and audio data of an environment; aI intelligent analysis is carried out on the audio and video data, the video data and the audio data are preprocessed, the identity of each person in the environment is identified from the preprocessed data, independent voice of each person is separated from the multi-sound-source mixed audio, the spatial position of a speaker is positioned, and the timbre feature of each separated voice stream is extracted and marked; performing cross-modal matching on the character, the voice and the timbre, converting the voice into character information, generating a corresponding sign language animation according to the character information, and sending the character information and the sign language animation to the deaf-mute terminal; and sending the text information input by the deaf-mute to the speaker terminal. In a multi-person dialogue scene, the deaf-mute can clearly distinguish different speakers and corresponding speaking contents through the terminal, the real-time performance of communication is ensured, the deaf-mute can accurately understand the intention of the speaker, and the communication efficiency is improved.

The present invention describes a method for processing three-dimensional gesture information based on a Leap motion somatosensory sensor in combination with a BPN neural network to finally recognize gestures. The whole process includes three stages, namely, the stage of collecting special gesture images specified by the user, the stage of performing dimensionality reduction operations on the three-dimensional gesture information as the input of the BPN neural network, and the stage of realizing gesture recognition by the BPN neural network. First, the user defines a set of dynamic multi-frame gesture recognition sets and binds name tags. The information obtained from saving at the driver layer needs to be cut, selected, and subjected to dimensionality reduction operations for long samples. Then, it is used as the input of the BPN neural network for network training to achieve convergence, so as to realize the ultimate goal of online recognition of various gestures. The present invention can perform high-precision and high-speed recognition and detection on the three-dimensional gesture actions defined by the user, and output and print the recognized action names in real time, and can also ensure the synchronous progress of network training and recognition. It is of great significance for the communication between deaf-mute people and ordinary people.

1. The name of the product of this design: intelligent sign language translation gloves and their intelligent control and translation terminal. 2. The purpose of this design product: It is used for the deaf-mute people. Through sign language translation, the audio of normal people's speech is translated into sign language movements of digital people, and the sign language movements of the deaf-mute people are translated and broadcasted through the speakers. 3. The design highlights of this appearance design product are: the overall appearance and connection method of the wearable gloves and smart terminal products, the layout of the wearable gloves sensors and the line planning. 4. The picture or photo that best illustrates the design points: Stereoscopic diagram 1.

The application discloses a kind of tactile perception system and its use method, tactile perception system includes: information acquisition module, analysis module and output module.The use method of tactile perception system includes: obtaining information for analysis;According to the information obtained, analysis is carried out, and the information that needs to be output to user is obtained;The information that needs to be output to user is converted into tactile perception signal, and the tactile perception signal is expressed information by contacting the position of user body, and at least one of contact type, shape, contact area, action, temperature and pressure;Output tactile perception signal.Through the tactile perception system and its use method of the application, people can better utilize tactile to perceive external information, enhance the effect of perception, and can help blind, deaf-mute or help normal people in the environment that is not conducive to vision and hearing to improve the efficiency of walking, searching, using computer and intelligent device / intelligent system etc..

The present invention discloses a method and system for testing speech signals based on laryngeal vibration and electroencephalogram (EEG) electrical stimulation, which relates to the field of speech information classification and includes: determining whether to apply EEG electrical stimulation to a user during the speech preparation stage according to the basic EEG signal; acquiring the laryngeal vibration signal and EEG signal of the user synchronously collected during the speech execution stage and respectively performing data preprocessing on them to obtain the preprocessed laryngeal vibration signal and EEG signal; using the PageRank algorithm to perform channel selection on the preprocessed EEG signal to obtain the EEG signal after channel selection; respectively performing feature extraction and dimensionality reduction processing on the preprocessed laryngeal vibration signal and the EEG signal after channel selection to obtain the laryngeal vibration features after dimensionality reduction and the EEG features after dimensionality reduction, and inputting them into the dual-modal SPCA with L1 regularization to obtain sparse features and inputting them into the trained speech signal classification model to obtain the speech classification and recognition result. The present invention solves the problem of homophony but different meanings in the pronunciation of deaf-mute people.

The invention relates to the technical field of artificial intelligence, computer vision and voice processing, in particular to a lip language recognition system and method based on an improved TSM-ResNet50 space-time network model for the voice recognition requirements of deaf-mute people and in a noisy environment. According to the system, efficient and accurate recognition of lip language video data is achieved by fusing deep learning technologies such as a YOLOv7 target detection algorithm, a TSM-ResNet50 space-time network model, a GRU network and an attention mechanism, visual display of lip language recognition is achieved by adopting a B / S framework and a lightweight Flask framework, and user experience is greatly improved. The invention aims to provide a more convenient and efficient communication mode for the deaf-mute, and promote the development and application of the lip language recognition technology.

The utility model discloses a legal sign language translation device and equipment, and relates to the technical field of electronic equipment, the legal sign language translation device comprises a voice module, a bending sensor, an attitude sensor and a signal processing module; the signal processing module is connected with the voice module, bending sensors arranged on all fingers on the palm and a posture sensor arranged on the palm of the user. The real-time posture of the palm of a user is collected through the posture sensor, the bending degree of each finger on the palm is collected through the bending sensor, and the electric signals are transmitted to the signal processing module, so that the signal processing module sends corresponding voice signals to the voice module; therefore, the voice module can play the voice corresponding to the legal sign language formed by the palm and the fingers at present. The legal sign language used by a user can be directly converted into voice, so that the user can quickly communicate with people who do not understand the legal sign language through played semantics, and the deaf-mute can be better helped to carry out legal consultation.

1. Name of the product in this design: Sign Language Translator. 2. Purpose of this design: A sign language interpretation device to assist deaf and mute people in daily life and meetings. 3. The key design feature of this product is its shape. 4. The image or photograph that best illustrates the design's key points: 3D view 1.

The invention relates to a deaf-mute barrier-free concert interaction system and method based on multi-modal dynamic mapping. The system comprises a wearable tactile execution unit, an AI processing unit, a multi-mode synchronous controller and a VR interaction module, and the tactile execution unit outputs multi-dimensional vibration modes to different body areas through differentially arranged linear resonance actuators. The AI processing unit maps real-time audio features into tactile parameters based on a contrast learning model and balances experience and energy consumption through the optimization module, the multi-mode synchronous controller achieves millisecond alignment of audio, vision and tactile, and the VR interaction module maps sign language input into a virtual scene and a tactile feedback adjustment instruction. Through the innovative tactile layout and the cross-modal mapping model, the immersion, rhythm perception and safety of the deaf-mute user are remarkably improved, and the method is suitable for various scenes such as concerts, education and rehabilitation.

The invention belongs to the technical field of interaction, and discloses a deaf-mute two-way interaction method, which comprises the steps of collecting voice of a dialogue person, converting the voice into first character information, visually displaying the first character information to a deaf-mute user, or converting the first character information into sign language actions or sign language schematic diagrams, displaying the sign language actions or sign language schematic diagrams to the deaf-mute user, capturing lip shape images of the dialogue person, and displaying the lip shape images to the deaf-mute user. Third character information is generated through matching based on the lip language database, the third character information is matched with the first character information to determine that the first character information is from the target dialogue person, a hand action image of the deaf-mute user is captured, the hand action image is converted into second character information, and the second character information is sent to the deaf-mute user. According to the method, the voice of the dialogue person is collected, the voice is converted into characters or sign language actions, the characters or sign language actions are visually displayed to the deaf-mute user, and the sign language actions are converted into the characters based on the sign language database by collecting the sign language actions of the deaf-mute user and are played in the form of voice.

1. The name of the design product: intelligent sign language translation machine. 2. The use of the design product: sign language translation equipment for assisting the deaf and mute to communicate and exchange in daily life and meetings. 3. The design points of the design product: in shape. 4. The picture or photo that best indicates the design points: perspective drawing.

The doorbell comprises a transmitting module, a receiving module, a function selection module and a master control module, the transmitting module is used for transmitting a radio frequency signal, the receiving module is used for outputting a wireless starting signal according to the radio frequency signal, the function selection module is used for selecting a deaf-mute function or a blind function, and the master control module is used for controlling the deaf-mute function or the blind function. And the main control module is used for executing lamp-on and horn-off operation according to the deaf-mute function and executing horn-off operation according to the blind function. According to the doorbell, different function modes are switched through the function selection module to meet the use requirements of the deaf-mute blind; for the deaf-mute, the main control module carries out the operation that the lamp is turned on and the loudspeaker does not sound, so that the bright light reminds the deaf-mute that someone comes to visit; and for the blind person, the main control module prompts the blind person to visit through loud sound by executing the operation that the loudspeaker is not turned on, so that not only is the use flexibility of the doorbell improved, but also the independent living ability and living quality of the blind person can be improved.

The invention relates to the technical field of medical first-aid equipment, and discloses a first-aid pager automatic alarm system based on multi-modal perception, the system is internally provided with eight modules, a multi-modal perception module collects user state data online, a special group data collection module pays attention to information of three types of crowds of deaf-mute, blind and cognitive impairment, and a first-aid pager alarm module is provided with a first-aid alarm module and a second-aid alarm module. The data analysis module calculates a session key, a permission compliance rate, a fault influence degree and a success rate of simplified operation of a cognitive impairment, the sensitive information encryption protection module performs end-to-end encryption on collected data, the fine-grained permission management and control module performs dynamic access control judgment, the distributed fault self-healing module performs self-healing strategy judgment, and the distributed fault self-healing module performs self-healing strategy judgment. The special crowd interaction optimization module performs interaction strategy evaluation, and the instruction execution module receives encryption, authority, self-healing and interaction optimization instructions issued by all the modules, uniformly controls underlying hardware and network resources to complete actions after priority arbitration, and returns an execution result log to form a closed loop.

The invention provides a doctor-seeing intelligent guiding system suitable for a deaf-mute, and belongs to the technical field of hospital doctor-seeing guiding, and the system comprises a doctor-seeing guiding module which obtains the appointment registration information and the real-time doctor-seeing condition of a deaf-mute patient, transmits doctor-seeing process guiding information to a user terminal device at the deaf-mute patient, and sends the doctor-seeing process guiding information to the user terminal device; marking the doctor-seeing guide information on a hospital three-dimensional map, and guiding the deaf-mute patient to see a doctor; the doctor seeing interaction module is used for collecting and recognizing sign language video images made by the deaf-mute patient, obtaining sign language recognition results, displaying the sign language recognition results to medical staff, collecting voice information or character information sent by the medical staff, and generating sign language conversion videos; and the doctor-seeing customer service module performs information interaction with the deaf-mute patient through a video interaction device, an information interaction device and a customer service robot which are arranged, and guides the deaf-mute patient to seeing a doctor smoothly. According to the invention, intelligent guide treatment for the deaf-mute patient is realized, information interaction between medical staff and the patient is optimized, the treatment effect is improved, and the treatment experience of the deaf-mute patient is improved.

The present invention relates to a voice-assisted mobile phone system for deaf-mutes, comprising: a display module configured to display text information; a voice input module configured to input voice information of the user's daily communication language; an intelligent voice processing module configured to extract the voice information of the user's daily communication language and establish a database with a one-to-one correspondence with the text information; a voice recognition module configured to, through the input of the user's external voice information, recognize the content of the voice information, obtain the corresponding text information from the database, and output the text information to the display module; a voice playback module configured to be able to convert the text information displayed by the display module into voice information for playback. The present invention integrates a deaf-mute auxiliary communication system into a mobile phone. In daily communication, users can carry the mobile phone to assist deaf-mutes in completing daily communication, and this system helps deaf-mutes build confidence in communication.

According to the automatic program running device of the computer, an instruction is automatically output through voice recognition of a user and is input as a program control instruction, so that the user can automatically start a required program only by voice without operating through an icon on a display screen. Therefore, the program starting time is shortened, and an accurate program execution result can be obtained. Besides, a result obtained through a program device is integrated into a virtual environment platform linked with the 6G technology, and a user can experience a more real simulation test in a non-face-to-face virtual space, so that safer purchase compared with reality is realized, and automatic service of the whole process from shopping to distribution is obtained. Particularly, by introducing various input modes, the blind, the deaf-mute and the old can conveniently use the computer, and meanwhile, the working efficiency of an enterprise can be greatly improved by utilizing a program automation system.

The invention relates to the technical field of computer vision, in particular to a sign language recognition method and system. The method comprises the following steps: acquiring an initial sign language prediction text based on multiple continuous to-be-recognized sign language pictures of a user after preprocessing; and if the text is vacant, outputting the probability that the vacant part belongs to each candidate word through the linear projection layer, and taking the probability as the visual confidence. When the number of vacancies is smaller than a preset value, calculating semantic similarity between the candidate words and the initial sign language prediction text as text confidence; on the contrary, the initial sign language prediction text and the initial sign language prediction text corresponding to multiple frames before and after are spliced into a total prediction text, and then the semantic similarity is calculated to obtain the text confidence coefficient. And obtaining a dynamic adjustment factor in combination with the definition of the sign language picture, fusing the factor with the two confidence coefficients to obtain a target confidence coefficient, and complementing the vacancy with the highest confidence coefficient to obtain a final sign language recognition result. The sign language semantic recognition precision is greatly improved, and the immersion of interaction with the deaf-mute is enhanced.

This invention provides a sign language recognition and interaction system based on edge-cloud collaboration, belonging to the field of artificial intelligence technology. It includes an image acquisition module for acquiring images of gestures from deaf and mute individuals and extracting keyframe images from these images; a client processing module, wirelessly connected to the image acquisition module, for receiving keyframe images, extracting key hand points from the keyframe images, and transmitting the extracted key hand point sequence to the cloud; a cloud processing module for processing the transmitted key hand point sequence using a sign language recognition model, recognizing the gestures as text based on the key hand point sequence; the cloud processing module also transmits the text to the client processing module, which in turn performs voice playback of the text and simultaneously acquires the voice response from a hearing person, converting the voice response into text for display. This system improves the portability and accuracy of sign language translation, reduces production costs, and improves the quality of life for deaf and mute individuals.

The invention relates to the field of intelligent wearable equipment, and discloses a gesture recognition and remote translation bracelet for emergency help of a deaf-mute. Comprising a bracelet body, a gesture recognition module, a central processing module, a translation module, a communication module, an alarm module, a display module and a power module, the gesture recognition module comprises a plurality of miniature cameras and an inertial sensor, and a deep learning neural network model is placed in the central processing module; the hand ring body comprises a display screen assembly and two belts, the surface of one belt is provided with a plurality of buckles, one end of the other belt is provided with a hoop, and the other end of the other belt is provided with a translation module; the gesture image information acquired by the plurality of micro cameras is fused with the dynamic data acquired by the inertial sensor, so that the recognition effect on the complex emergency help-seeking gesture is better, the help-seeking intention of the deaf-mute can be quickly and accurately captured, the position information of the user is synchronously transmitted, and quick response and rescue are realized.

A luminous percussion instrument 100, such as a drum, comprises at least one light-emitting element arranged inside a hollow housing. The percussion instrument also includes at least one beating region 3 and at least one light-transmittable surface 11. An induction switch, which may be an infrared induction switch operating with a range of 5 to 50 millimetres, is arranged in each beating region. The induction switches are electrically connected to the light-emitting element(s) to control their activation in response to a user striking the beating regions. The percussion instrument may include three beating regions which each have an associated light-emitting element and where the light-emitting elements are different colours, for example red, green and blue. The percussion instrument may also include a snare with an adjusting rod. The instrument may provide multi-sensory stimulation to players, such as deaf people, learning to play instruments.

The invention relates to the technical field of intelligent equipment, in particular to a method for displaying call content of a smart phone through AR glasses. According to the method for displaying the call content of the smart phone through the AR glasses provided by the invention, the technical problem of low dialogue efficiency during dialogue with the deaf-mute in the prior art is solved. The invention discloses a method for displaying call content of a smart phone through AR glasses. The method comprises the following steps: establishing communication between the AR glasses and the smart phone; establishing communication between the smart phone and the cloud server; the voice information of the communication party is picked up to form the voice data, one or more response data is recommended according to the context of the voice data and the communication scene, a user only needs to select one of the response data, and then the response data is played in the form of voice through the loudspeaker. Therefore, the communication party can quickly obtain the response of the user in the dialogue, so that the communication efficiency is effectively improved. And moreover, the user does not need to handwriting the response information, so that the use experience of the user is optimized.

The invention relates to a vehicle-mounted interaction method and device, electronic equipment, a storage medium and a program product, and relates to the technical field of vehicle-mounted interaction.The method in one embodiment comprises the steps that a triggering instruction for triggering a vehicle-mounted interaction auxiliary mode is received, and the vehicle-mounted interaction auxiliary mode is entered in response to the triggering instruction; in the vehicle-mounted interaction auxiliary mode, a first interaction operation instruction of a driver is obtained, and first interaction content information of the first interaction operation instruction is obtained based on a preset first corresponding relation between the interaction operation instruction and the interaction content information; and outputting the first interaction content information. By adopting the scheme of the embodiment, the communication requirements of drivers with language barriers such as deaf-mute people can be met, and the vehicle-mounted interaction performance is improved.