An accessibility interaction system based on multi-modal artificial intelligence and glasses

Abstract

This invention discloses an accessible interactive system and glasses based on multimodal artificial intelligence. The accessible interactive system includes an AI chip, a processor, a sensing input module, a voice module, and a hazard warning unit. The sensing input module receives finger swipe signals from the control input sensing area located on the glasses. The hazard warning unit receives and executes hazard action information derived from image data processing and analysis, presenting it as a warning message on the glasses. The advantages of this invention are: fast interactive data processing speed, rapid response to user communication, support for real-time processing of multimodal information, and improved human-computer interaction fluency; enhanced environmental perception capabilities and user safety through dual-camera collaboration and intelligent warning mechanisms; and extended device battery life and optimized data management efficiency through a combination of low-power Bluetooth transmission and local storage.

Description

Technical Field

[0001] This invention relates to the field of intelligent interaction technology, specifically to an accessible interaction system and glasses based on multimodal artificial intelligence. Background Technology

[0002] Smart glasses, brimming with technological sophistication, are popular among users for their ability to project relevant information onto the lenses without notes and switch languages ​​for communication. However, the single-modal interactive system of smart glasses suffers from slow response times and poor fluency. Furthermore, its built-in memory increases power consumption, shortens standby time, and is significantly affected by battery drain during data transfer. Therefore, this paper proposes an accessible interactive system and glasses based on multimodal artificial intelligence to address these issues. Summary of the Invention

[0003] The purpose of this invention is to provide an accessible interaction system and glasses based on multimodal artificial intelligence in order to solve the above-mentioned problems.

[0004] The present invention achieves the above objectives through the following technical solution: an accessible interaction system based on multimodal artificial intelligence, wherein the accessible interaction system is an intelligent integration unit of glasses, and the accessible interaction system includes an AI chip, a processor, a sensing input module, a voice module, and a danger warning unit;

[0005] The AI ​​chip is combined with the processor to accelerate the processing and analysis of language communication data acquired by the voice module and image data acquired by the front and rear cameras.

[0006] The sensing input module is used to receive finger swipe signals implemented by the control input sensing area located on the glasses;

[0007] The danger warning unit is used to receive and execute dangerous action information obtained from the processing and analysis of image data as a signal and present it on the glasses in the form of warning information.

[0008] Preferably, the AI ​​chip, combined with the processor, accelerates the processing and analysis of data information, which is then transmitted via Bluetooth to one of the smartphones or smartwatches for storage.

[0009] A pair of glasses includes a smart integrated lens and temples mounted on a frame, wherein a forward-facing camera is mounted on the smart integrated lens, and the integration technology of the smart integrated lens is not limited to display technology and sensing technology.

[0010] Preferably, a finger touch sensor and a rearview camera are respectively provided on the outer front end of the temple, and the finger touch sensor is electrically connected to the sensing input module.

[0011] Preferably, the operation input methods of the finger touch sensor include forward and backward short and long distance sliding, clicking, and in-place forward and reverse rubbing circle drawing. The forward and backward short and long distance sliding method is used to zoom in and out and adjust the volume of the image information presented by the smart integrated lens. The clicking method is used to switch and confirm the function interface. The in-place forward and reverse rubbing circle drawing method is used to browse the content of a certain function interface.

[0012] Preferably, a voice amplifier and a voice microphone are respectively provided on the inner front end of the temple, and the voice amplifier and the voice microphone are electrically connected to the voice module. The inside of the temple is a hollow structure for installing the system integrated circuit and power supply.

[0013] Preferably, the voice amplifier is used for issuing warning voice prompts and broadcasting the information content presented by the smart integrated lens to the communication recipient.

[0014] Preferably, the voice microphone and the voice amplifier are in an asynchronous startup and operation state.

[0015] Compared with existing technologies, the advantages of this invention are:

[0016] First, it features fast interactive data processing speed, rapid response to user communication and dialogue, and supports real-time processing of multimodal information, thereby improving the smoothness of human-computer interaction.

[0017] Second, by using dual-camera collaboration and intelligent early warning mechanisms, we enhance environmental perception capabilities and ensure user safety.

[0018] Third, by combining low-power Bluetooth transmission with local storage, the device's battery life is extended and data management efficiency is optimized. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram illustrating the interaction principle of an accessible interactive system based on multimodal artificial intelligence.

[0021] Figure 2 This is a schematic diagram of the control principle of an accessible interaction system based on multimodal artificial intelligence.

[0022] Figure 3 This is a schematic diagram of the structure of the glasses in this invention.

[0023] In the diagram: 100, eyeglasses; 110, smart integrated lens; 111, front-facing camera; 120, temples; 121, finger swipe sensor; 122, rear-facing camera; 123, speaker; 124, microphone; 200, smartphone; 300, smartwatch; 400, accessibility system; 410, AI chip; 420, processor; 430, sensor input module; 440, Bluetooth; 450, voice module; 460, hazard warning unit. Detailed Implementation

[0024] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0025] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0026] In the description of this invention, it should be understood that the terms "upper", "lower", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0027] Please see Figure 1-3 As shown, an accessible interaction system based on multimodal artificial intelligence is disclosed. The accessible interaction system 400 is the intelligent integration unit of the glasses 100. The accessible interaction system 400 includes an AI chip 410, a processor 420, a sensing input module 430, a voice module 450, and a danger warning unit 460.

[0028] The AI ​​chip 410 is combined with the processor 420 to accelerate the processing and analysis of the language communication data acquired by the voice module 450 and the image data acquired by the front and rear cameras.

[0029] The sensing input module 430 is used to receive finger swipe signals implemented by the control input sensing area located on the glasses 100;

[0030] The danger warning unit 460 is used to receive and execute dangerous action information obtained from the processing and analysis of image data as a signal and present it on the glasses 100 in the form of warning information.

[0031] Furthermore, the AI ​​chip 410, combined with the processor 420, accelerates the processing and analysis of data information, which is then transmitted via Bluetooth 440 to one of the smartphone 200 and the smartwatch 300 for storage.

[0032] Please see Figure 1 and Figure 3 As shown, a pair of glasses, using the system as described in any one of claims 1 to 2, the glasses 100 includes a smart integrated lens 110 and temples 120 mounted on the frame of the glasses 100, the smart integrated lens 110 is provided with a forward-looking camera 111, wherein the integration technology of the smart integrated lens 110 is not limited to display technology and sensing technology.

[0033] Furthermore, a finger touch sensor 121 and a rear-view camera 122 are respectively provided on the outer front end of the temple 120, and the finger touch sensor 121 is electrically connected to the sensing input module 430. The operation input commands of the finger touch sensor 121 include forward and backward short and long distance sliding, clicking, and in-place forward and reverse rubbing circle. The forward and backward short and long distance sliding method can zoom in and out and adjust the volume of the image information presented by the smart integrated lens 110, the clicking method can switch and confirm the function interface, and the in-place forward and reverse rubbing circle method can browse the content of a certain function interface.

[0034] Furthermore, a voice amplifier 123 and a voice microphone 124 are respectively provided on the inner front end of the temple 120. The voice amplifier 123 and the voice microphone 124 are both electrically connected to the voice module 450. The temple 120 has a hollow structure inside, which is used to install the system integrated circuit and power supply.

[0035] Furthermore, the voice amplifier 123 is used for warning voice prompts and broadcasting the information content presented by the smart integrated lens 110 to the communication object. The voice microphone 124 and the voice amplifier 123 are in an asynchronous start-up and operation state.

[0036] The specific interactive operation process is as follows:

[0037] S1. The barrier-free interaction system 400 connects to the smartphone 200 or smartwatch 300 via Bluetooth.

[0038] S2. The finger acts on the finger touch sensor 121, and selectively performs the following actions as needed: the operation input command methods include forward and backward short and long distance sliding, clicking, and in-place forward and reverse rubbing circle drawing. The forward and backward short and long distance sliding method enlarges and reduces the image information presented by the smart integrated lens 110 and adjusts the volume. The clicking method switches and confirms the function interface. The in-place forward and reverse rubbing circle drawing method flips through the content of a certain function interface, solving the inconvenience of carrying additional control accessories.

[0039] S3. Present the information required by the smartphone 200 or the smartwatch 300 on the smart integrated lens 110, so that the wearer of glasses 100 can see the content to be introduced in front of their eyes. The content of a certain function interface can also be flipped through by rubbing the screen in a circular motion.

[0040] S4, the front-view camera 111 and the rear-view camera 122 are selectively turned on, and at the same time, they are transmitted to the storage of the smartphone 200 or the storage of the smartwatch 300 via Bluetooth 440, which reduces the overall power consumption and increases the diversity of storage methods;

[0041] S5. The rear-view camera 122 is activated to monitor the blind spot environment of the wearer and the front-view camera 111 monitors the front field of view. If a dangerous scene is detected, the danger warning unit 460 is triggered to issue a warning signal, which is displayed on the smart integrated lens 110. At the same time, the voice amplifier 123 emits a prompt sound to improve the safety protection of the user.

[0042] Compared with existing technologies, the advantages of this invention are:

[0043] First, it features fast interactive data processing speed, rapid response to user communication and dialogue, and supports real-time processing of multimodal information, thereby improving the smoothness of human-computer interaction.

[0044] Second, by using dual-camera collaboration and intelligent early warning mechanisms, we enhance environmental perception capabilities and ensure user safety.

[0045] Third, by combining low-power Bluetooth transmission with local storage, the device's battery life is extended and data management efficiency is optimized.

[0046] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalent elements of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0047] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A barrier-free interaction system based on multimodal artificial intelligence, characterized in that: The barrier-free interaction system (400) is the intelligent integration unit of the glasses (100). The barrier-free interaction system (400) includes an AI chip (410), a processor (420), a sensing input module (430), a voice module (450), and a danger warning unit (460). The AI ​​chip (410) is combined with the processor (420) to accelerate the processing and analysis of language communication data acquired by the voice module (450) and image data acquired by the front and rear distributed cameras; The sensing input module (430) is used to receive finger swipe signals implemented by the control input sensing area located on the glasses (100); The danger warning unit (460) is used to receive and execute the dangerous action information obtained from the processing and analysis of image data as a signal and present it on the glasses (100) in the form of warning information.

2. The barrier-free interaction system based on multi-modal artificial intelligence according to claim 1, characterized in that: The AI ​​chip (410) and the processor (420) work together to accelerate the processing and analysis of data information, which is then transmitted via Bluetooth (440) to one of the smartphones (200) and the smartwatch (300) for storage.

3. Eyeglasses applying the system according to any one of claims 1 to 2, characterized in that: The glasses (100) include a smart integrated lens (110) and temples (120) mounted on the frame of the glasses (100). A forward-facing camera (111) is provided on the smart integrated lens (110). The integration technology of the smart integrated lens (110) is not limited to display technology and sensing technology.

4. The eyeglasses of claim 3, wherein: The outer front end of the temple (120) is provided with a finger touch sensor (121) and a rearview camera (122), and the finger touch sensor (121) is electrically connected to the sensor input module (430).

5. The eyeglasses of claim 4, wherein: The operation input methods of the finger touch sensor (121) include forward and backward short and long distance sliding, clicking, and in-place forward and reverse rubbing circle drawing. The forward and backward short and long distance sliding method can zoom in and out and adjust the volume of the image information presented by the smart integrated lens (110). The clicking method can switch and confirm the function interface. The in-place forward and reverse rubbing circle drawing method can browse the content of a certain function interface.

6. The eyeglasses of claim 3, wherein: The inner front end of the temple (120) is provided with a voice amplifier (123) and a voice microphone (124). The voice amplifier (123) and the voice microphone (124) are electrically connected to the voice module (450). The temple (120) has a hollow structure inside, which is used to install the system integrated circuit and power supply.

7. The eyeglasses of claim 6, wherein: The voice amplifier (123) is used to provide early warning voice prompts and broadcast the information content presented by the smart integrated lens (110) to the communication target.

8. The eyeglasses of claim 6, wherein: The voice microphone (124) and the voice amplifier (123) are in an asynchronous start-up and operation state.

Images