Pickup system of AR barrier-free hearing-aid glasses

Abstract

The invention relates to the technical field of intelligent equipment, in particular to a pickup system of AR barrier-free hearing-aid glasses. According to the pickup system of the AR barrier-free hearing-aid glasses provided by the invention, the technical problem of low sound pickup precision of a pickup system in the prior art is solved. The invention discloses a pickup system of AR barrier-free hearing-aid glasses. The pickup system comprises a glasses frame, the lenses are mounted on the spectacle frame; and a pickup assembly, wherein the pickup assembly is installed on the spectacle frame. By arranging the pickup unit and the lip language recognition unit, the pickup system can simultaneously take the sound signal and the lip language data in the pickup process, and integrates the contents recognized by the pickup unit and the lip language recognition unit through the data processing unit, so that the sound data can be corrected through the lip language data, or the lip language data can be corrected through the lip language recognition unit. Error correction is performed on the lip language data through the sound data, so that the third text file generated by the data processing unit has higher precision, and the pickup error of the pickup system is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent devices, and in particular to a sound pickup system for an AR barrier-free hearing aid glasses. Background Art

[0002] ‌Due to their inherent physical defects, deaf-mutes cannot communicate with others through language because they do not have the ability to hear and speak, which brings great inconvenience in daily life.

[0003] With the development of technology, hearing aids enable some deaf-mutes to obtain external sounds or texts converted from external sounds through hearing aids or smart glasses. Taking smart glasses as an example, the sound pickup device on the smart glasses can convert external sounds into texts, or the smart glasses are combined with hearing aids, enabling deaf-mutes to observe texts or hearing aids to understand the speech content of others, thus realizing the communication of deaf-mutes.

[0004] Although the sound pickup systems in the prior art for picking up external sounds can meet the function of picking up sound signals, the sound pickup accuracy of the sound pickup systems in the prior art is relatively low, and signal transmission errors may occur during the communication with deaf-mutes.

[0005] Therefore, the sound pickup systems in the prior art have the technical problem of low sound pickup accuracy. Summary of the Invention

[0006] A sound pickup system for an AR barrier-free hearing aid glasses provided by the present invention solves the technical problem of low sound pickup accuracy of the sound pickup systems in the prior art.

[0007] Some implementation schemes for solving the above technical problems include: A sound pickup system for an AR barrier-free hearing aid glasses includes a frame; Lenses, the lenses are installed on the frame; And a sound pickup component, the sound pickup component is installed on the frame; The frame includes a frame body and temple arms provided on the frame body; The sound pickup component includes a sound pickup unit for picking up sound signals and a lip language recognition unit for picking up lip movement data. The sound pickup component further includes a data processing unit, and both the sound pickup unit and the lip language recognition unit communicate with the data processing unit; Among them, the sound pickup unit picks up the sound signal to be extracted and generates a first text file according to the picked sound signal; The lip language recognition unit picks up the lip language signal and generates a second text file according to the picked lip language signal; The data processing unit generates a third text file based on the contents of the first text file and the second text file, and the data processing unit outputs the content picked up by the sound pickup component according to the third text file.

[0008] Preferably, the data processing unit generating the third text file based on the contents of the first text file and the second text file includes the following steps: Determine the memory sizes occupied by the first text file and the second text file; When the memory occupied by the first text file is larger than that occupied by the second text file, correct the content of the first text file using the content of the second text file, and the corrected first text file is the third text file; When the memory occupied by the second text file is larger than that occupied by the first text file, correct the content of the second text file using the content of the first text file, and the corrected second text file is the third text file.

[0009] Preferably, the lip reading recognition unit generates the second text file using a convolutional neural network and a long short-term memory network.

[0010] Preferably, the data processing unit corrects the first text file or the second text file using a natural language processing model.

[0011] Preferably, the sound pickup unit includes a microphone and a noise reducer, and the sound signal picked up by the microphone is sent to the data processing unit after being denoised by the noise reducer.

[0012] Preferably, the sound pickup unit further includes a voiceprint collector, the voiceprint collector collects the voiceprint of the sound signal to be extracted, and the noise reducer denoises the extracted sound signal according to the voiceprint collected by the voiceprint collector.

[0013] Preferably, the sound pickup component includes a housing, the sound pickup unit and the lip reading recognition unit are both integrated in the housing, and the housing is fixed to the spectacle frame.

[0014] Preferably, the sound pickup component includes a front component installed on the frame body and collecting front signals and a rear component arranged on the temple and collecting rear signals, and the front component and the rear component cooperate to collect side signals.

[0015] Preferably, the data processing unit is a cloud server, and the sound pickup unit and the lip reading recognition unit both communicate with the cloud server through a wireless network.

[0016] Preferably, the data processing unit outputs the sound picked up by the sound pickup component according to the third text file, or the data processing unit outputs text according to the third text file.

[0017] Compared with the prior art, the present invention has the following advantages: By setting up a sound pickup unit and a lip-reading recognition unit, during the sound pickup process, the sound pickup system can simultaneously acquire sound signals and lip-reading data, and the data processing unit integrates the content recognized by the sound pickup unit and the lip-reading recognition unit, so that the sound data can be corrected by the lip-reading data, or the lip-reading data can be corrected by the sound data. Furthermore, the text file three generated by the data processing unit has higher accuracy, reducing the sound pickup error of the sound pickup system.

[0018] By converting the sound signals picked up by the sound pickup unit into text file one, and converting the content recognized by the lip-reading recognition unit into text file two, both text file one and text file two are in text form. The data processing unit has less computing pressure during error correction, improving the data processing efficiency of the sound pickup system. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] For purposes of explanation, several embodiments of the technology of the present invention are illustrated in the following drawings. The following drawings are incorporated into this text and form a part of the specific embodiments. In some cases, well-known structures and components are shown in block diagram form to avoid obscuring the concepts of the subject technology of the present invention.

[0020] Figure 1 It is a schematic diagram of the first angle of the present invention.

[0021] Figure 2 It is a schematic diagram of the second angle of the present invention.

[0022] Figure 3 It is a top view of the present invention. Figure 3 The dotted lines in it represent the sound pickup ranges of the respective sound pickup units.

[0023] Figure 4 It is a structural block diagram of the present invention.

[0024] As shown in the figure: 1. Frame, 11. Frame body, 12. Temple.

[0025] 2. Lens.

[0026] 3. Sound pickup assembly, 31. Sound pickup unit, 311. Microphone, 312. Noise reducer, 313. Voiceprint collector, 32. Lip-reading recognition unit, 33. Data processing unit. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] The specific embodiments shown below are intended as descriptions of various configurations of the subject technology of the present invention, and are not intended to represent the only configurations in which the subject technology of the present invention can be practiced. The specific embodiments include specific details intended to provide a thorough understanding of the subject technology of the present invention. However, it will be clear and obvious to those skilled in the art that the subject technology of the present invention is not limited to the specific details shown herein, and can be practiced without these specific details.

[0028] It can be understood that, herein, relational terms such as "first" and "second" are intended to distinguish one entity or operation from another entity or operation, and are not intended to expressly or impliedly indicate any actual relationship or order between these entities or operations.

[0029] The term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0030] Referring to Figures 1 to 4 As shown, a sound pickup system of an AR barrier-free hearing aid glasses includes a frame 1; Lenses 2, and the lenses 2 are mounted on the frame 1; And a sound pickup component 3, and the sound pickup component 3 is mounted on the frame 1; The frame 1 includes a frame body 11 and temple arms 12 provided on the frame body 11; The sound pickup component 3 includes a sound pickup unit 31 for picking up sound signals and a lip language recognition unit 32 for picking up lip movement data. The sound pickup component 3 further includes a data processing unit 33, and both the sound pickup unit 31 and the lip language recognition unit 32 communicate with the data processing unit 33; Wherein, the sound pickup unit 31 picks up the sound signals to be extracted and generates a first text file according to the picked sound signals; The lip language recognition unit 32 picks up lip language signals and generates a second text file according to the picked lip language signals; The data processing unit 33 generates a third text file according to the contents of the first text file and the second text file, and the data processing unit 33 outputs the content picked up by the sound pickup component 3 according to the third text file.

[0031] In some embodiments, the data processing unit 33 generating the third text file according to the contents of the first text file and the second text file includes the following steps: Determine the memory size occupied by text file one and text file two; When the memory occupied by text file one is greater than that occupied by text file two, use the content of text file two to correct the content of text file one, and the corrected text file one is text file three; When the memory occupied by text file two is greater than that occupied by text file one, use the content of text file one to correct the content of text file two, and the corrected text file two is text file three.

[0032] When both the voice signal and the lip movement signal are converted into text, the larger the memory occupied by the text file, the more content the text file contains and the richer its content. Therefore, when text file one occupies more memory, use text file two to correct text file one, which can further reduce the error rate of text file three and make the voice pickup system have higher accuracy.

[0033] In some embodiments, the lip movement recognition unit 32 generates text file two using a convolutional neural network and a long short-term memory network.

[0034] In some embodiments, the data processing unit 33 corrects text file one or text file two using a natural language processing model.

[0035] In some embodiments, the voice pickup unit 31 includes a microphone 311 and a noise reducer 312, and the voice signal picked up by the microphone 311 is sent to the data processing unit 33 after being denoised by the noise reducer 312.

[0036] Refer to Figures 1 to 4 As shown, in some embodiments, the voice pickup unit 31 further includes a voiceprint collector 313, the voiceprint collector 313 collects the voiceprint of the voice signal to be extracted, and the noise reducer 312 denoises the extracted voice signal according to the voiceprint collected by the voiceprint collector 313.

[0037] Specifically, when the user uses the voice pickup system for communication, usually the person being communicated with is a single person. At this time, using the voiceprint information of the person being communicated with can filter out the voices emitted by other people. That is to say, when the voiceprint of the person being communicated with is recognized, the noise reducer 312 can delete all signals that do not contain this voiceprint, and only leave the voice signals that contain this voiceprint, thereby effectively reducing the noise of the voice signal and reducing the data operation pressure during the voice signal processing process.

[0038] In some embodiments, the voice pickup assembly 3 includes a housing, and both the voice pickup unit 31 and the lip movement recognition unit 32 are integrated in the housing, and the housing is fixed to the spectacle frame 1. The housing can be bonded to the spectacle frame 1, or the housing can be fixed to the spectacle frame 1 by threads.

[0039] In some embodiments, the sound pickup component 3 includes a front component installed on the frame body 11 for collecting front signals and a rear component disposed on the temple 12 for collecting rear signals, and the front component and the rear component cooperate to collect side signals.

[0040] Multiple sound pickup components 3 can pick up sound signals and lip language signals in different directions, optimizing the performance of the sound pickup system.

[0041] In some embodiments, the data processing unit 33 is a cloud server, and both the sound pickup unit 31 and the lip language recognition unit 32 communicate with the cloud server through a wireless network.

[0042] In some embodiments, the data processing unit 33 outputs the sound picked up by the sound pickup component 3 according to Text File Three, or the data processing unit 33 outputs text according to Text File Three.

[0043] In some embodiments, when the data processing unit 33 outputs sound according to Text File Three, a hearing aid can be used to output the sound so that the sound is directly transmitted to the user. The hearing aid can be directly installed on the temple 12.

[0044] In some embodiments, when the data processing unit 33 outputs text according to Text File Three, the text can be directly displayed on the lens 2, enabling the user to read it conveniently.

[0045] The above introduces the technical solutions of the present invention's theme and corresponding details. It can be understood that the above introduction is only some implementation schemes of the technical solutions of the present invention's theme, and some details can also be omitted during its specific implementation.

[0046] In addition, in some implementation schemes of the above invention, it is possible to combine multiple implementation schemes. Due to space limitations, various combination schemes are not listed one by one. Those skilled in the art can freely combine and implement the above implementation schemes according to requirements during specific implementation to obtain a better application experience.

[0047] When implementing the technical solutions of the present invention's theme, those skilled in the art can obtain other detailed configurations or drawings according to the technical solutions of the present invention's theme and the drawings. Obviously, without departing from the technical solutions of the present invention's theme, these details still fall within the scope covered by the technical solutions of the present invention's theme.

Claims

1. A sound pickup system for AR barrier-free hearing-aid glasses, characterized in that: The invention comprises a spectacles frame (1); a lens (2), wherein the lens (2) is mounted on the spectacles frame (1); and a sound pickup component (3), wherein the sound pickup component (3) is mounted on the spectacles frame (1); the spectacles frame (1) comprises a frame body (11) and temples (12) arranged on the frame body (11); the sound pickup component (3) comprises a sound pickup unit (31) for picking up sound signals and a lip language recognition unit (32) for picking up lip movement data; the sound pickup component (3) further comprises a data processing unit (33); the sound pickup unit (31) and The lip reading recognition units (32) are in communication with the data processing unit (33); wherein the sound pickup unit (31) picks up the sound signal to be extracted, and generates a text file one according to the picked up sound signal; the lip reading recognition unit (32) picks up the lip reading signal, and generates a text file two according to the picked up lip reading signal; the data processing unit (33) generates a text file three according to the contents of the text file one and the text file two, and the data processing unit (33) outputs the contents picked up by the sound pickup component (3) according to the text file three.

2. The sound pickup system of AR barrier-free hearing-aid glasses according to claim 1, characterized in that: The data processing unit (33) generates text file three according to the contents of text file one and text file two, comprising the following steps: determining the memory sizes occupied by text file one and text file two; when the memory occupied by text file one is larger than the memory occupied by text file two, using the contents of text file two to correct the contents of text file one, and the text file one after error correction is text file three; when the memory occupied by text file two is larger than the memory occupied by text file one, using the contents of text file one to correct the contents of text file two, and the text file two after error correction is text file three.

3. The sound pickup system of AR barrier-free hearing-aid glasses according to claim 1, characterized in that: The lip reading recognition unit (32) generates text file 2 using a convolutional neural network and a long short-term memory network.

4. The sound pickup system of AR barrier-free hearing-aid glasses according to claim 1, characterized in that: The data processing unit (33) uses a natural language processing model to perform error correction on text file one or text file two.

5. The sound pickup system of AR barrier-free hearing-aid glasses according to claim 1, characterized in that: The sound pickup unit (31) comprises a sound pickup (311) and a noise reducer (312); the sound signal picked up by the sound pickup (311) is sent to the data processing unit (33) after being subjected to noise reduction by the noise reducer (312).

6. The sound pickup system of AR barrier-free hearing-aid glasses according to claim 5, characterized in that: The sound pickup unit (31) further comprises a voiceprint collector (313), wherein the voiceprint collector (313) collects the voiceprint of the sound signal to be extracted, and the noise reducer (312) reduces the noise of the extracted sound signal according to the voiceprint collected by the voiceprint collector (313).

7. The sound pickup system of AR barrier-free hearing-aid glasses according to any one of claims 1 to 6, characterized in that: The sound pickup assembly (3) comprises a shell, the sound pickup unit (31) and the lip language recognition unit (32) are both integrated in the shell, and the shell is fixed to the mirror frame (1).

8. The sound pickup system of AR barrier-free hearing-aid glasses according to claim 7, characterized in that: The sound pickup component (3) comprises a front component mounted on the frame (11) and collecting front signals, and a rear component arranged on the temple (12) and collecting rear signals, wherein the front component cooperates with the rear component to collect side signals.

9. The sound pickup system of AR barrier-free hearing-aid glasses according to claim 8, characterized in that: The data processing unit (33) is a cloud server, and the sound pickup unit (31) and the lip reading recognition unit (32) both communicate with the cloud server via a wireless network.

10. The sound pickup system of AR barrier-free hearing-aid glasses according to claim 1, characterized in that: The data processing unit (33) outputs the sound picked up by the sound pickup component (3) according to the text file three, or the data processing unit (33) outputs text to the text file three.

Images