Method for displaying call content of smart phone through AR glasses
AR glasses work in collaboration with smartphones and cloud servers, convert voice data into text and recommend response data, solving the problem of low conversation efficiency for deaf and mute people and achieving efficient communication without handwritten responses.
Patent Information
- Application Number
- CN202510569567.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-04
- Publication Date
- 2025-07-04
AI Technical Summary
Deaf and mute people have low dialogue efficiency during communication, and the reliance on handwritten responses in the existing technology leads to insufficient efficiency.
Through AR glasses working in collaboration with smartphones and cloud servers, voice data is picked up and converted into text and response data is recommended according to the context. After the user selects it, it plays in the form of sound, so that the response is not required without handwritten.
It improves communication efficiency for deaf and mute people, optimizes user experience, reduces the steps of handwritten responses, and improves communication speed.
Smart Images
Figure CN120263894A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of intelligent devices, and particularly to a method for an AR glasses to display the call content of a smart phone. Background Art
[0002] Deaf-mutes face many communication barriers in life, study and work because they cannot hear sounds and cannot speak. With the development of technology, the popularization of hearing aids enables deaf-mutes to hear external sounds using hearing aids. However, although hearing aids can enable deaf-mutes to hear sounds, due to the fact that deaf-mutes cannot speak, there are still significant barriers in the communication process. Usually, deaf-mutes can respond during a conversation by writing.
[0003] That is, in the prior art, deaf-mutes can use hearing aids to hear external sounds and then use writing means to complete responses during a conversation. Although this method enables deaf-mutes to communicate, due to the need to write, the communication efficiency is low.
[0004] Therefore, there is a technical problem of low conversation efficiency when communicating with deaf-mutes in the prior art. Summary of the Invention
[0005] A method for an AR glasses to display the call content of a smart phone provided by the present invention solves the technical problem of low conversation efficiency when communicating with deaf-mutes in the prior art.
[0006] Some implementation schemes for solving the above technical problems include: A method for an AR glasses to display the call content of a smart phone includes the following steps: Establish communication between the AR glasses and the smart phone; Establish communication between the smart phone and the cloud server; The AR glasses pick up the voice stream of the communication party to generate voice data; The cloud server obtains the voice data through the smart phone and converts the voice data into text data; The AR glasses receive the text data generated by the cloud server and display the text data using the AR glasses; When the voice stream of the communication party requires a user response, the cloud server recommends one or more response data according to the context of the voice data and the communication scenario; After the user determines a response data, the response data is played in the form of sound by the speaker.
[0007] Preferably, the cloud server recommending one or more response data according to the context of the voice data and the communication scenario includes the following steps: Determine the current application scenario; Traverse the dialogue database in the current application scenario according to the current application scenario, where the dialogue database includes all dialogue data within the last three months in the current scenario; Match the dialogue database according to the voice data, and determine one or more response data based on the data in the dialogue database.
[0008] Preferably, in determining the current application scenario, the current application scenario is determined by obtaining the location information of the smart phone, or the user manually determines the current application scenario by operating the smart phone.
[0009] Preferably, determining the current application scenario further includes determining the temperature data at the location of the current application scenario, matching the dialogue database according to the voice data, and determining one or more response data based on the data in the dialogue database and the temperature data.
[0010] Preferably, the one or more response data are displayed in text form using AR glasses.
[0011] Preferably, the one or more response data are displayed in text form using a smart phone.
[0012] Preferably, the user determines a response data by operating the screen of the smart phone, or the user determines a response data by operating the keys of the smart phone.
[0013] Preferably, the speaker is the speaker of the smart phone, or the speaker communicates with the smart phone via Bluetooth.
[0014] Preferably, the AR glasses communicate with the smart phone via Bluetooth.
[0015] Preferably, the AR glasses communicate with the smart phone via WIFI.
[0016] Compared with the prior art, the present invention has the following advantages: By picking up the voice information of the communication party to form voice data, and recommending one or more response data according to the context of the voice data and the communication scenario, the user only needs to select one of the response data, and then, the response data is played in the form of sound through the speaker, and the communication party can quickly obtain the user's response in the conversation, thus effectively improving the communication efficiency. And, the user no longer needs to handwrite the response information, and the communication party can get the response, effectively optimizing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] For purposes of explanation, several embodiments of the technology of the present invention are illustrated in the following drawings. The following drawings are incorporated herein and constitute a part of the specific embodiments. In some cases, well-known structures and components are shown in block diagram form in order to avoid obscuring the concepts of the subject technology of the present invention.
[0018] Figure 1 It is a schematic diagram of the present invention.
[0019] Figure 2 It is a flowchart of the present invention.
[0020] As shown in the figure: 1. AR glasses, 2. Smart phone, 3. Cloud server. Detailed implementation manners
[0021] The specific embodiments shown below are intended to be a description of various configurations of the subject technology of the present invention, and are not intended to represent the only configurations in which the subject technology of the present invention can be practiced. The specific embodiments include specific details intended to provide a thorough understanding of the subject technology of the present invention. However, it will be clear and obvious to those skilled in the art that the subject technology of the present invention is not limited to the specific details shown herein, and can be practiced without these specific details.
[0022] It can be understood that in this document, relational terms such as "first" and "second" are intended to distinguish one entity or operation from another entity or operation, and are not intended to explicitly or implicitly imply any actual relationship or order between these entities or operations.
[0023] The term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.
[0024] Referring to Figures 1 to 2 As shown, a method for an AR glasses to display the call content of a smart phone includes the following steps: Establish communication between the AR glasses 1 and the smart phone 2; Establish communication between the smart phone 2 and the cloud server 3; The AR glasses 1 pick up the voice stream of the communication party to generate voice data; The cloud server 3 obtains the voice data through the smart phone 2 and converts the voice data into text data; The AR glasses 1 receive the text data generated by the cloud server 3 and display the text data using the AR glasses 1; When the voice stream of the communication party requires the user to respond, the cloud server 3 recommends one or more response data according to the context of the voice data and the communication scenario; After the user determines a response data, the speaker plays the response data in the form of sound.
[0025] The cloud server 3 usually has the ability of big data processing and can generate different response data for different application scenarios, that is, the conversation content in different scenarios is similar. For example, in application scenarios such as coffee shops and milk tea shops, the corresponding content between customers and store clerks is similar. Usually, the customer selects their favorite products, and the store clerk makes appropriate inquiries or responses. When the user is a store clerk, since the response is usually relatively simple, therefore, the cloud server 3 can effectively provide one or more response data for the user according to the current application scenario.
[0026] In some embodiments, the cloud server 3 recommends one or more response data according to the context of the voice data and the communication scenario, including the following steps: Determine the current application scenario; Traverse the conversation database under the current application scenario according to the current application scenario. The conversation database includes all conversation data within the last three months under the current scenario; Match the voice data with the conversation database and determine one or more response data according to the data in the conversation database.
[0027] The cloud server 3 can adopt artificial intelligence algorithms to recommend one or more response data in different application scenarios.
[0028] In some embodiments, in the determination of the current application scenario, the current application scenario is determined by obtaining the location information of the smart phone 2, or the user manually determines the current application scenario by operating the smart phone 2.
[0029] Usually, the user manually operating the smart phone 2 to determine the current application scenario has a higher accuracy rate. Preferably, the current application scenario can be determined by the user's manual operation method in the determination of the current application scenario.
[0030] In some embodiments, the determination of the current application scenario further includes determining the temperature data of the location where the current application scenario is located, matching the voice data with the conversation database, and determining one or more response data according to the data in the conversation database and the temperature data.
[0031] Understandably, different additional parameters may be required in different application scenarios. For example, in this embodiment, the additional parameter is temperature data, and such additional parameters can be applied to coffee shops, milk tea shops, etc., to recommend response data on whether to add ice according to the temperature data.
[0032] That is to say, other parameters can also be collected in different application scenarios to obtain more or more accurate response data.
[0033] In some embodiments, the one or more response data are displayed in text form using the AR glasses 1.
[0034] Alternatively, the one or more response data are displayed in text form using the smart phone 2.
[0035] In some embodiments, the user determines a response data by operating the screen of the smart phone 2, or the user determines a response data by operating the keys of the smart phone 2.
[0036] In some embodiments, the speaker is the speaker of the smart phone 2, or the speaker communicates with the smart phone 2 via Bluetooth.
[0037] In some embodiments, the AR glasses 1 communicate with the smart phone 2 via Bluetooth.
[0038] Alternatively, the AR glasses 1 communicate with the smart phone 2 via WIFI.
[0039] Refer to Figure 1 As shown, the hardware for implementing the above functions generally includes AR glasses 1. The AR glasses 1 generally include a frame and lenses. The lenses are disposed on the frame. The lenses are transparent display lenses. The frame is provided with a display component for causing the lenses to display corresponding text data.
[0040] The frame can usually be provided with a pickup, and the pickup is usually a microphone for collecting voice streams. The microphone can be one or more. When multiple microphones collect voice streams simultaneously, the collection of voice streams is more accurate. That is, the voice data is more accurate.
[0041] The frame can also be provided with a camera for taking pictures or identifying other people.
[0042] The smart phone 2 can also be replaced by other intelligent devices, such as a tablet computer, etc.
[0043] The cloud server 3 generally has voice AI speech recognition capabilities.
[0044] The corresponding communication module is used to implement the communication between the AR glasses 1, the smart phone 2, and the cloud server 3.
[0045] The technical solution of the present invention will be further introduced below with a specific application example: Taking the current application scenario as a coffee shop as an example: Refer to Figures 1 to 2 As shown, after the user wears the AR glasses 1 and takes their seat, a data link is established among the AR glasses 1, the smart phone 2, and the cloud server 3, and they can communicate with each other. Then, the current application scenario is determined to be a coffee shop automatically or manually. At this time, the cloud server 3 defines the current application scenario of the AR glasses 1 as a coffee shop.
[0046] When a customer enters the coffee shop, the camera on the frame can recognize the presence of the customer. At this time, the cloud server 3 can automatically recommend one or more response data, which is usually a greeting. After the user selects the corresponding greeting, the speaker plays the greeting in the form of sound.
[0047] Alternatively, when the customer enters the coffee shop and greets the user actively, after the cloud server 3 obtains the voice data formed by the greeting, it immediately recommends one or more response data, which is usually a corresponding greeting. After the user selects the corresponding greeting, the speaker plays the greeting in the form of sound.
[0048] The microphone of the AR glasses 1 continues to monitor the customer's voice. For example, the customer asks for a cup of coffee. At this time, after the cloud server 3 obtains the voice data, it traverses the dialogue database and recommends the corresponding response data. For example, "Do you want sugar?" Or, it can also recommend response data on whether to add ice according to the temperature data in the current application scenario. The response data can be displayed on the smart phone 2, and the response data can also be displayed on the AR glasses 1. Usually, when the response data is displayed on the smart phone 2, it is more convenient for the user to operate.
[0049] The user can select one or more greetings by operating the smart phone 2. Then, the speaker can play response data such as "Do you want sugar?" and "Do you want ice?" in the form of sound.
[0050] The above has introduced the technical solution of the present invention theme and the corresponding details. It can be understood that the above introduction is only some implementation schemes of the present invention theme technical solution, and some details can also be omitted during its specific implementation.
[0051] In addition, in some implementation schemes of the above invention, it is possible to combine multiple implementation schemes. Due to space limitations, various combination schemes are not listed one by one. Those skilled in the art can freely combine and implement the above implementation schemes according to their needs during specific implementation to obtain a better application experience.
[0052] Those skilled in the art can obtain other detailed configurations or drawings according to the technical solution of the present invention and the drawings when implementing the technical solution of the present invention. Obviously, these details still fall within the scope covered by the technical solution of the present invention without departing from the technical solution of the present invention.
Claims
1. A method for an AR glasses to display the call content of a smartphone, characterized in that, It includes the following steps: establishing communication between the AR glasses (1) and the smartphone (2); establishing communication between the smartphone (2) and the cloud server (3); the AR glasses (1) picking up the voice stream of the communication party to generate voice data; the cloud server (3) obtaining the voice data through the smartphone (2) and converting the voice data into text data; the AR glasses (1) receiving the text data generated by the cloud server (3) and displaying the text data by using the AR glasses (1); when the voice stream of the communication party requires a user response, the cloud server (3) recommends one or more response data according to the context of the voice data and the communication scenario; After the user determines a response data, the response data is played in the form of sound by the speaker.
2. The method for an AR glasses to display the call content of a smartphone according to claim 1, wherein: The cloud server (3) recommending one or more response data according to the context of the voice data and the communication scenario includes the following steps: determining the current application scenario; traversing the conversation database under the current application scenario according to the current application scenario, and the conversation database includes all conversation data in the current scenario in the past three months; Matching the voice data with the conversation database and determining one or more response data according to the data in the conversation database.
3. The method for an AR glasses to display the call content of a smart phone according to claim 2, characterized in that: In the determination of the current application scenario, the current application scenario is determined by obtaining the location information of the smartphone (2), or the user manually determines the current application scenario by operating the smartphone (2).
4. The method for an AR glasses to display the call content of a smart phone according to claim 3, characterized in that: The determination of the current application scenario further includes determining the temperature data of the location where the current application scenario is located, matching the voice data with the conversation database, and determining one or more response data according to the data in the conversation database and the temperature data.
5. The method for an AR glasses to display the call content of a smart phone according to claim 1, wherein: The one or more response data are displayed by using the AR glasses (1) in text form.
6. The method for an AR glasses to display the call content of a smart phone according to claim 1, characterized in that: The one or more response data are displayed by using the smartphone (2) in text form.
7. The method for an AR glasses to display the call content of a smart phone according to claim 6, wherein: The user determines a response data by operating the screen of the smartphone (2), or the user determines a response data by operating the keys of the smartphone (2).
8. The method for an AR glasses to display the call content of a smart phone according to claim 1, wherein: The speaker is the speaker of the smartphone (2), or the speaker communicates with the smartphone (2) via Bluetooth.
9. The method for an AR glasses to display the call content of a smart phone according to claim 1, characterized in that: The AR glasses (1) communicate with the smartphone (2) via Bluetooth.
10. The method for an AR glasses to display the call content of a smart phone according to claim 1, characterized in that: The AR glasses (1) communicate with the smartphone (2) via WIFI.