61 results about "Deaf mutes" patented technology

The invention discloses a neck-hung sign language interpretation device and a sign language semantic recognition interpretation method thereof.The neck-hung sign language interpretation device comprises a wearing support which defines a neck surrounding space and comprises a middle connecting arm and two shells, and the two shells are connected to the two sides of the middle connecting arm respectively; the image acquisition unit is arranged on the front side of the shell, comprises a three-dimensional depth camera and an adjusting mechanism for driving the three-dimensional depth camera to rotate, and is used for dynamically tracking and acquiring continuous sign language videos; the processing module is integrated in the shell, comprises a main control processor and a memory, and is used for operating a sign language recognition algorithm; the voice output unit is electrically connected with the main control processor and is used for playing the translation result; and the power supply unit supplies power to the image acquisition unit, the processing module and the voice output unit. Gesture languages used by the deaf-mute for communication are converted into natural languages which can be easily understood by normal people, the deaf-mute group is helped to speak, and the problem of information output when the deaf-mute communicates with the normal people is solved.

The invention relates to the technical field of sign language intelligent communication, in particular to a deaf-mute sign language translation pronunciation system which comprises a data acquisition and preprocessing module, a sign language model recognition module, a dynamic gesture model module and a voice conversion module. According to the sign language translation pronunciation system for the deaf-mute, wide-angle camera intelligent glasses are adopted for visual collection, wearable equipment is not needed, video streams and thermodynamic diagrams are combined, visual information and key point probability distribution are complementary, the influence of single-mode noise is reduced, video space-time features are extracted through a 3D CNN, long-range dependence is captured in combination with a self-attention mechanism of Transform, and therefore the sign language translation pronunciation system for the deaf-mute is obtained. The sign language recognition precision is improved, an internal reference matrix and a distortion coefficient are calculated through a calibration board, wide-angle lens distortion is corrected, it is ensured that hand key points are accurately positioned, YOLOv8 is adopted to segment an interference object, background noise is prevented from affecting recognition, 21 key point thermodynamic diagrams are generated through MediaPipe, fault tolerance of low-confidence-coefficient key points is enhanced through Gaussian kernel diffusion, and the recognition accuracy is improved. And the model identification precision is improved.

The invention relates to a sign language-to-multilingual text speech mutual conversion method based on deep learning and application thereof, belongs to the field of artificial intelligence, man-machine interaction and barrier-free communication, and particularly relates to a sign language recognition, semantic understanding and multilingual speech synthesis method based on deep learning and an application system thereof. The sign language to voice text conversion method comprises the following steps: 1, an image acquisition and preprocessing module, 2, a hand key point detection module, 3, time sequence modeling and sign language recognition, 4, natural language understanding and semantic optimization, 5, a multi-language translation module, and 6, a text and voice synthesis output module. According to the sign language-to-multilingual text voice mutual conversion method based on deep learning and the application thereof, sign languages can be automatically, flexibly and accurately converted into voices and characters, and deaf-mutes are helped to efficiently communicate with common people; the system can be widely applied to hospitals, schools, government affair windows, public transportation and other scenes.

The invention belongs to the technical field of pronunciation correction methods, and particularly discloses a deaf-mute pronunciation correction method, which comprises the following steps of: marking pronunciation units in standard audio data, extracting Mel-frequency cepstral coefficients and filter bank characteristics in the standard audio data, and establishing an acoustic model of a deep neural network hidden Markov model; acoustic features in standard audio data can be effectively extracted, user audio is converted into a text through an acoustic model, and time axis alignment with the standard audio data is carried out through a dynamic time warping algorithm, so that accurate scores are provided for the user audio, a user is helped to know own pronunciation ability more clearly, and the user experience is improved. And more efficient correction and learning are realized.

The utility model relates to an intelligent auxiliary glasses system for a deaf-mute. The intelligent auxiliary glasses system comprises a central control module, a sign language detection module, a voice playing module, an environment detection module, a sound detection module, a display module and a vibration module, the central control module is connected with the sign language detection module, the voice playing module, the environment detection module, the sound detection module, the display module and the vibration module. The intelligent auxiliary glasses system provided by the utility model has comprehensive functions, and can fully consider the safety problem of deaf-mute people going out.

The invention discloses a deaf-mute sign language recognition method, device and equipment based on multiple modes and a medium. The method comprises the steps of obtaining a static image or a dynamic video of a deaf-mute sign language; for the static image, a static sign language recognition network based on a ViT model is adopted, and static sign languages of the deaf-mute are classified and recognized; for the dynamic video, a dynamic sign language recognition network is adopted for classification and recognition; the dynamic sign language recognition network firstly carries out key frame extraction on an original continuous sign language video; then, a background removing module is used for removing the background in the video frame sequence; an ACN encoder jointly constructed by convolution, space attention and LSTM is used to extract space-time fusion features; and finally, constructing an ACN decoder by combining a time attention mechanism and an LSTM, and carrying out decoding learning on the space-time fusion features. The sign language recognition method effectively solves the problems of complex background interference, redundant frame processing and the like in sign language recognition, and has high recognition precision and stability.

The invention belongs to the technical field of computer vision and artificial intelligence, and particularly relates to a deaf-mute gesture recognition method based on a DCGAN-Yolov5 algorithm, and the method comprises the following steps: 1, firstly, making a deaf-mute gesture original data set which comprises different gesture sample pictures of a deaf-mute; 2, expanding the different gesture sample pictures in the step 1 by using DCGAN to obtain a mixed data set; and step 3, performing label labeling operation on the mixed data set obtained in the step 2 to obtain a label corresponding to the mixed data set, and putting the label into a PostgreSQL database. According to the method, an original data set is expanded by adopting a DCGAN (deep convolutional generative adversarial network) model, so that the problem that an identification model is easy to over-fit under the condition of insufficient data volume is effectively solved. By means of the data enhancement technology, the recognition model can learn feature representation with richer samples, and therefore better generalization ability is shown in practical application.

The invention discloses a sign language recognition translation model training method based on deep learning and a large model, and relates to the technical field of sign language recognition. Converting the sign language video into a sign language image; the sign language image comprises a static sign language image and a dynamic sign language image; extracting static frame images from the dynamic sign language image, and performing weighted stacking on the static frame images to obtain a weighted stacking image; inputting the static sign language image and the weighted superposition image into a pre-constructed convolutional neural network to obtain a discrete sign language; inputting the discrete sign language into the large language model for synthesis to obtain a complete sign language; and verifying the convolutional neural network according to the complete sign language, and obtaining a final sign language recognition translation model. The sign language recognition translation model trained by using the method provided by the invention can quickly and accurately recognize the sign language information of the deaf-mute, and the communication effect with the deaf-mute is better improved.

The present invention discloses a data processing method and system for multimodal facial motion point data and vocal cord motion data. The method includes providing text, collecting continuous facial images or videos and laryngeal vibration data of a normal person speaking, preprocessing the data, extracting temporal and spatial features, establishing a facial and neck motion model for Chinese pronunciation, and having a deaf-mute person imitate the pronunciation according to the model and obtain feedback. The system includes a depth camera, a laryngeal vibration sensor, and a microphone. By comprehensively utilizing multimodal data, it provides instant feedback to the deaf-mute, lowers the learning threshold, and improves communication efficiency. It is applicable to deaf-mute groups worldwide. This invention promotes speech vocalization training and has broad application prospects and social significance.

The invention provides a control method and device based on deaf-mute travel auxiliary vision glasses and a medium, and belongs to the technical field of auxiliary vision. The method comprises the following steps: acquiring current first image information, and detecting whether a gesture action exists or not according to a set gesture recognition model; if yes, the current first image information is recognized according to the set gesture recognition model, single isolated word information is extracted and obtained, and the isolated word information is added to the cache annular queue one by one according to the time sequence. And when the quantity of the isolated word information is greater than the set quantity threshold value, correspondingly reading the context information stored in the current storage text. And processing the isolated word information and the context information according to the set semantic integration large model, capturing the context information, carrying out semantic integration on the isolated word information according to the context information and the context information, and displaying an obtained output statement. The data acquisition accuracy, the sign language recognition translation accuracy and the user experience can be improved.

The invention discloses an intelligent sign language auxiliary communication method. The method comprises the following steps: acquiring video data and audio data of an environment; aI intelligent analysis is carried out on the audio and video data, the video data and the audio data are preprocessed, the identity of each person in the environment is identified from the preprocessed data, independent voice of each person is separated from the multi-sound-source mixed audio, the spatial position of a speaker is positioned, and the timbre feature of each separated voice stream is extracted and marked; performing cross-modal matching on the character, the voice and the timbre, converting the voice into character information, generating a corresponding sign language animation according to the character information, and sending the character information and the sign language animation to the deaf-mute terminal; and sending the text information input by the deaf-mute to the speaker terminal. In a multi-person dialogue scene, the deaf-mute can clearly distinguish different speakers and corresponding speaking contents through the terminal, the real-time performance of communication is ensured, the deaf-mute can accurately understand the intention of the speaker, and the communication efficiency is improved.

The present invention describes a method for processing three-dimensional gesture information based on a Leap motion somatosensory sensor in combination with a BPN neural network to finally recognize gestures. The whole process includes three stages, namely, the stage of collecting special gesture images specified by the user, the stage of performing dimensionality reduction operations on the three-dimensional gesture information as the input of the BPN neural network, and the stage of realizing gesture recognition by the BPN neural network. First, the user defines a set of dynamic multi-frame gesture recognition sets and binds name tags. The information obtained from saving at the driver layer needs to be cut, selected, and subjected to dimensionality reduction operations for long samples. Then, it is used as the input of the BPN neural network for network training to achieve convergence, so as to realize the ultimate goal of online recognition of various gestures. The present invention can perform high-precision and high-speed recognition and detection on the three-dimensional gesture actions defined by the user, and output and print the recognized action names in real time, and can also ensure the synchronous progress of network training and recognition. It is of great significance for the communication between deaf-mute people and ordinary people.

1. The name of the product of this design: intelligent sign language translation gloves and their intelligent control and translation terminal. 2. The purpose of this design product: It is used for the deaf-mute people. Through sign language translation, the audio of normal people's speech is translated into sign language movements of digital people, and the sign language movements of the deaf-mute people are translated and broadcasted through the speakers. 3. The design highlights of this appearance design product are: the overall appearance and connection method of the wearable gloves and smart terminal products, the layout of the wearable gloves sensors and the line planning. 4. The picture or photo that best illustrates the design points: Stereoscopic diagram 1.

System and methods for performing routing determinations for communications requests for both assisted communications and non-assisted communications with devices operated by deaf or hard-of-hearing (DHH) users using a single number are described. A network component may determine whether to use routing that directs such requests to an assistance system based on the type of communications requested and one or more identifiers in a request. If a communications request includes a DHH user device's identifier and communications type that are represented in single number communications data, the network component may forward that request to an assistance system. Otherwise, the network component may route such a request normally.

The application discloses a kind of tactile perception system and its use method, tactile perception system includes: information acquisition module, analysis module and output module.The use method of tactile perception system includes: obtaining information for analysis;According to the information obtained, analysis is carried out, and the information that needs to be output to user is obtained;The information that needs to be output to user is converted into tactile perception signal, and the tactile perception signal is expressed information by contacting the position of user body, and at least one of contact type, shape, contact area, action, temperature and pressure;Output tactile perception signal.Through the tactile perception system and its use method of the application, people can better utilize tactile to perceive external information, enhance the effect of perception, and can help blind, deaf-mute or help normal people in the environment that is not conducive to vision and hearing to improve the efficiency of walking, searching, using computer and intelligent device / intelligent system etc..

The present invention discloses a method and system for testing speech signals based on laryngeal vibration and electroencephalogram (EEG) electrical stimulation, which relates to the field of speech information classification and includes: determining whether to apply EEG electrical stimulation to a user during the speech preparation stage according to the basic EEG signal; acquiring the laryngeal vibration signal and EEG signal of the user synchronously collected during the speech execution stage and respectively performing data preprocessing on them to obtain the preprocessed laryngeal vibration signal and EEG signal; using the PageRank algorithm to perform channel selection on the preprocessed EEG signal to obtain the EEG signal after channel selection; respectively performing feature extraction and dimensionality reduction processing on the preprocessed laryngeal vibration signal and the EEG signal after channel selection to obtain the laryngeal vibration features after dimensionality reduction and the EEG features after dimensionality reduction, and inputting them into the dual-modal SPCA with L1 regularization to obtain sparse features and inputting them into the trained speech signal classification model to obtain the speech classification and recognition result. The present invention solves the problem of homophony but different meanings in the pronunciation of deaf-mute people.

The invention relates to the technical field of artificial intelligence, computer vision and voice processing, in particular to a lip language recognition system and method based on an improved TSM-ResNet50 space-time network model for the voice recognition requirements of deaf-mute people and in a noisy environment. According to the system, efficient and accurate recognition of lip language video data is achieved by fusing deep learning technologies such as a YOLOv7 target detection algorithm, a TSM-ResNet50 space-time network model, a GRU network and an attention mechanism, visual display of lip language recognition is achieved by adopting a B / S framework and a lightweight Flask framework, and user experience is greatly improved. The invention aims to provide a more convenient and efficient communication mode for the deaf-mute, and promote the development and application of the lip language recognition technology.

The utility model discloses a legal sign language translation device and equipment, and relates to the technical field of electronic equipment, the legal sign language translation device comprises a voice module, a bending sensor, an attitude sensor and a signal processing module; the signal processing module is connected with the voice module, bending sensors arranged on all fingers on the palm and a posture sensor arranged on the palm of the user. The real-time posture of the palm of a user is collected through the posture sensor, the bending degree of each finger on the palm is collected through the bending sensor, and the electric signals are transmitted to the signal processing module, so that the signal processing module sends corresponding voice signals to the voice module; therefore, the voice module can play the voice corresponding to the legal sign language formed by the palm and the fingers at present. The legal sign language used by a user can be directly converted into voice, so that the user can quickly communicate with people who do not understand the legal sign language through played semantics, and the deaf-mute can be better helped to carry out legal consultation.

1. Name of the product in this design: Sign Language Translator. 2. Purpose of this design: A sign language interpretation device to assist deaf and mute people in daily life and meetings. 3. The key design feature of this product is its shape. 4. The image or photograph that best illustrates the design's key points: 3D view 1.

The invention relates to a deaf-mute barrier-free concert interaction system and method based on multi-modal dynamic mapping. The system comprises a wearable tactile execution unit, an AI processing unit, a multi-mode synchronous controller and a VR interaction module, and the tactile execution unit outputs multi-dimensional vibration modes to different body areas through differentially arranged linear resonance actuators. The AI processing unit maps real-time audio features into tactile parameters based on a contrast learning model and balances experience and energy consumption through the optimization module, the multi-mode synchronous controller achieves millisecond alignment of audio, vision and tactile, and the VR interaction module maps sign language input into a virtual scene and a tactile feedback adjustment instruction. Through the innovative tactile layout and the cross-modal mapping model, the immersion, rhythm perception and safety of the deaf-mute user are remarkably improved, and the method is suitable for various scenes such as concerts, education and rehabilitation.

The invention belongs to the technical field of interaction, and discloses a deaf-mute two-way interaction method, which comprises the steps of collecting voice of a dialogue person, converting the voice into first character information, visually displaying the first character information to a deaf-mute user, or converting the first character information into sign language actions or sign language schematic diagrams, displaying the sign language actions or sign language schematic diagrams to the deaf-mute user, capturing lip shape images of the dialogue person, and displaying the lip shape images to the deaf-mute user. Third character information is generated through matching based on the lip language database, the third character information is matched with the first character information to determine that the first character information is from the target dialogue person, a hand action image of the deaf-mute user is captured, the hand action image is converted into second character information, and the second character information is sent to the deaf-mute user. According to the method, the voice of the dialogue person is collected, the voice is converted into characters or sign language actions, the characters or sign language actions are visually displayed to the deaf-mute user, and the sign language actions are converted into the characters based on the sign language database by collecting the sign language actions of the deaf-mute user and are played in the form of voice.

1. The name of the design product: intelligent sign language translation machine. 2. The use of the design product: sign language translation equipment for assisting the deaf and mute to communicate and exchange in daily life and meetings. 3. The design points of the design product: in shape. 4. The picture or photo that best indicates the design points: perspective drawing.

The utility model relates to sign language translation auxiliary equipment, and discloses a multi-language barrier-free audio-visual translation device which comprises a main body assembly. The main body assembly comprises a base, and the top of the base is fixedly connected with a display screen; the camera is arranged on the base; the base is provided with a height adjusting assembly, and the height adjusting assembly comprises an adjusting plate which is inserted into the base and is in sliding connection with the adjusting plate; the device has the advantages that the position of the camera can be adjusted, and the device is convenient to use, when a tall or short deaf-mute uses the device, a user pulls a pull plate to drive a limiting block to extrude a first spring and then pulls an adjusting plate to drive the camera to move upwards or downwards, and when the device is adjusted to a proper position, the user loosens the pull plate to drive the camera to move upwards or downwards. And a first spring drives a limiting block to bounce into a limiting groove, so that the camera is adjusted and fixed, people of different heights can use the camera conveniently, and the problem that the camera is inconvenient to use by people of different heights due to fixed installation is solved.

The utility model discloses a deaf-mute rear warning device in the technical field of warning devices, which comprises a main body, the main body is arc-shaped, two ends of the main body are connected with elastic rods, one end of each elastic rod is connected with a mounting block, and a vibration groove is arranged in the mounting block. The outer side of the main body is connected with the radio microphone, the radio microphone collects rear sound, the main body is connected with the radio microphone at a high position to avoid being blocked by clothes, and when the sound collected by the radio microphone exceeds the sound emitted by a common loudspeaker by 80 decibels, the control equipment enables the vibration motors at the two ends to rotate, so that the sound is collected. The output end of the vibration motor drives the semicircular vibration block to rotate so as to generate vibration, meanwhile, the eccentric hole in the vibration block drives the touch panel to eccentrically rotate, the touch panel makes contact with a user, vibration generated by the vibration motor and the touch panel which eccentrically rotates in a reciprocating mode conduct vibration and touch warning, somatosensory contact warning is conducted, and no warning blind area exists. And the warning effect is good.

The doorbell comprises a transmitting module, a receiving module, a function selection module and a master control module, the transmitting module is used for transmitting a radio frequency signal, the receiving module is used for outputting a wireless starting signal according to the radio frequency signal, the function selection module is used for selecting a deaf-mute function or a blind function, and the master control module is used for controlling the deaf-mute function or the blind function. And the main control module is used for executing lamp-on and horn-off operation according to the deaf-mute function and executing horn-off operation according to the blind function. According to the doorbell, different function modes are switched through the function selection module to meet the use requirements of the deaf-mute blind; for the deaf-mute, the main control module carries out the operation that the lamp is turned on and the loudspeaker does not sound, so that the bright light reminds the deaf-mute that someone comes to visit; and for the blind person, the main control module prompts the blind person to visit through loud sound by executing the operation that the loudspeaker is not turned on, so that not only is the use flexibility of the doorbell improved, but also the independent living ability and living quality of the blind person can be improved.

The invention relates to the technical field of medical first-aid equipment, and discloses a first-aid pager automatic alarm system based on multi-modal perception, the system is internally provided with eight modules, a multi-modal perception module collects user state data online, a special group data collection module pays attention to information of three types of crowds of deaf-mute, blind and cognitive impairment, and a first-aid pager alarm module is provided with a first-aid alarm module and a second-aid alarm module. The data analysis module calculates a session key, a permission compliance rate, a fault influence degree and a success rate of simplified operation of a cognitive impairment, the sensitive information encryption protection module performs end-to-end encryption on collected data, the fine-grained permission management and control module performs dynamic access control judgment, the distributed fault self-healing module performs self-healing strategy judgment, and the distributed fault self-healing module performs self-healing strategy judgment. The special crowd interaction optimization module performs interaction strategy evaluation, and the instruction execution module receives encryption, authority, self-healing and interaction optimization instructions issued by all the modules, uniformly controls underlying hardware and network resources to complete actions after priority arbitration, and returns an execution result log to form a closed loop.

The invention provides a doctor-seeing intelligent guiding system suitable for a deaf-mute, and belongs to the technical field of hospital doctor-seeing guiding, and the system comprises a doctor-seeing guiding module which obtains the appointment registration information and the real-time doctor-seeing condition of a deaf-mute patient, transmits doctor-seeing process guiding information to a user terminal device at the deaf-mute patient, and sends the doctor-seeing process guiding information to the user terminal device; marking the doctor-seeing guide information on a hospital three-dimensional map, and guiding the deaf-mute patient to see a doctor; the doctor seeing interaction module is used for collecting and recognizing sign language video images made by the deaf-mute patient, obtaining sign language recognition results, displaying the sign language recognition results to medical staff, collecting voice information or character information sent by the medical staff, and generating sign language conversion videos; and the doctor-seeing customer service module performs information interaction with the deaf-mute patient through a video interaction device, an information interaction device and a customer service robot which are arranged, and guides the deaf-mute patient to seeing a doctor smoothly. According to the invention, intelligent guide treatment for the deaf-mute patient is realized, information interaction between medical staff and the patient is optimized, the treatment effect is improved, and the treatment experience of the deaf-mute patient is improved.

The present invention relates to a voice-assisted mobile phone system for deaf-mutes, comprising: a display module configured to display text information; a voice input module configured to input voice information of the user's daily communication language; an intelligent voice processing module configured to extract the voice information of the user's daily communication language and establish a database with a one-to-one correspondence with the text information; a voice recognition module configured to, through the input of the user's external voice information, recognize the content of the voice information, obtain the corresponding text information from the database, and output the text information to the display module; a voice playback module configured to be able to convert the text information displayed by the display module into voice information for playback. The present invention integrates a deaf-mute auxiliary communication system into a mobile phone. In daily communication, users can carry the mobile phone to assist deaf-mutes in completing daily communication, and this system helps deaf-mutes build confidence in communication.

According to the automatic program running device of the computer, an instruction is automatically output through voice recognition of a user and is input as a program control instruction, so that the user can automatically start a required program only by voice without operating through an icon on a display screen. Therefore, the program starting time is shortened, and an accurate program execution result can be obtained. Besides, a result obtained through a program device is integrated into a virtual environment platform linked with the 6G technology, and a user can experience a more real simulation test in a non-face-to-face virtual space, so that safer purchase compared with reality is realized, and automatic service of the whole process from shopping to distribution is obtained. Particularly, by introducing various input modes, the blind, the deaf-mute and the old can conveniently use the computer, and meanwhile, the working efficiency of an enterprise can be greatly improved by utilizing a program automation system.

The invention relates to the technical field of computer vision, in particular to a sign language recognition method and system. The method comprises the following steps: acquiring an initial sign language prediction text based on multiple continuous to-be-recognized sign language pictures of a user after preprocessing; and if the text is vacant, outputting the probability that the vacant part belongs to each candidate word through the linear projection layer, and taking the probability as the visual confidence. When the number of vacancies is smaller than a preset value, calculating semantic similarity between the candidate words and the initial sign language prediction text as text confidence; on the contrary, the initial sign language prediction text and the initial sign language prediction text corresponding to multiple frames before and after are spliced into a total prediction text, and then the semantic similarity is calculated to obtain the text confidence coefficient. And obtaining a dynamic adjustment factor in combination with the definition of the sign language picture, fusing the factor with the two confidence coefficients to obtain a target confidence coefficient, and complementing the vacancy with the highest confidence coefficient to obtain a final sign language recognition result. The sign language semantic recognition precision is greatly improved, and the immersion of interaction with the deaf-mute is enhanced.