A wearable device for remote information sharing that provides a "see," "hear," and "speak" experience with a "help function" for the user.

Abstract

This invention aims to assist users in information gathering and organization, thereby deepening their activities, by investigating, recording, and analyzing synchronized and linked 3E information from the actual location, and by providing immediate feedback of the results not only to third parties in remote locations but also to the users themselves through a help function. [Solution] A wearable information device characterized by having a 3E information collection device (camera, microphone, etc.) as a head-mounted device, and a device having functions such as a power supply, memory, information processing device, and wireless communication for operating the head-mounted device worn outside the head, with the two connected by detachable power supply and transmission lines to form an integrated device.

Description

Technical Field

[0001] The present invention integrates a wearable camera, an earphone (or headset) with a microphone, a monitor, and various sensors, and mounts them on the head like glasses, visors, or headsets. An apparatus that performs power supply, external communication, temporary storage, simple information processing, etc. is mounted outside the head, and the entire set of devices connected to each other by a power supply and a transmission cable is concerned. The present invention aims to share synchronous data of "seeing", "hearing", and "speaking" experiences (hereinafter abbreviated as "3E" three experiences) between the wearer and a third party (or parties) or an assistant AI for a long time, and to provide a high-level 3E information analysis and advice function (hereinafter referred to as the "help function" in this article). By doing so, the "help function" with visual and auditory accompaniment is implemented in the work of the wearer, and the work efficiency is continuously improved during wearing.

Background Art

[0002] Conventionally, there have been earphones with microphones, wearable cameras, glasses-type or goggle-type monitors, and external power supply devices for them. However, there has been no wearable device that has a shape that does not interfere with activities and that can integrate these mechanisms and continuously share synchronous 3E information among multiple people, remote devices, etc. for a long time.

Prior Art Documents

Patent Documents

[0003] Patents for wearable cameras, patents for earphones, patents for monitors, patents for mobile batteries

Patent Document 1

Summary of the Invention

[0004] Due to the declining working population and the promotion of work-style reforms, Japan will increasingly need measures to improve labor productivity. We propose a device to support these measures from the perspective of increasing work efficiency. In most industries, if it becomes possible to simultaneously perform information gathering, information analysis, problem identification and resolution, and feedback on information gathering methods, significant results can be achieved in the implementation of information strategies and immediate responses. The device that can help with this is the "3E Shared Wearable Device with Help Function". The physical problems that this device aims to solve include the following: (i) While it is possible to collect 3E information and utilize that data by using conventional microphone-equipped earphones, wearable cameras, and monitors in combination, wearing several devices individually and synchronizing and linking the acquired information requires data conversion between each device and various applications, which requires proficiency in operation. Furthermore, changing the devices used requires time and effort to relearn different operations. (b) Each of the above-mentioned devices was an information collection terminal or an information viewing terminal, and was not capable of information synthesis, recording, or analysis. (h) Each conventional device often has its own power supply, but because it does not have sufficient power capacity, continuous use requires recharging, battery replacement, or the use of spare devices. (ii) In order to share the 3E information obtained from each device in real time at a remote location using personal information devices, it was necessary to use smartphones or tablet devices. (e) It is difficult to choose devices such as earphones or headphones with microphones, wearable cameras, and monitors that are suitable for the usage environment, and there are few that are easy to carry or wear. (h) Until now, there has been no device designed to allow users to receive advice from a third party in a remote location while listening to surrounding conversations and ambient sounds. (T) Conventional glasses-type and goggle-type monitors cover the entire field of view, which worsens the real-world visual environment, making it difficult to perform various real-world tasks and activities while wearing them. (C) There were no devices manufactured with the purpose of collecting the wearer's murmurs of questions and other sounds and providing AI-generated answers to them. (i) No device was found that could withstand use in unsafe locations and fulfill the above-mentioned functions. [Means for solving the problem]

[0005] Use the same symbols as the problem to describe the corresponding solution. (i) Eliminate the power supply and wireless communication devices from earphones with a microphone or headphones with a microphone (regardless of whether they use bone conduction or tympanic conduction), and integrate a wearable camera with a monitor that covers the lower (upper) edge of the field of view, creating a system that provides synchronized information in a unified standard. This integrated device can solve the problem. Hereafter, this will be referred to as a "head-mounted device." (b) A separate device (hereinafter referred to as the "main unit") having functions such as power supply, communication, recording, and analysis for the head-mounted device is prepared and attached to a location other than the head (such as the neck, shoulders, chest, back, waist, or legs, in a place that does not interfere with the purpose of the activity), and the two are connected with a power supply / transmission cable to form a single device. (h) By equipping the main unit with the largest possible capacity battery and supplying power to each device, sufficient operating time for the intended purpose can be ensured, and the weight and volume of individual devices can be reduced. (ii) The main unit will be equipped with a wireless communication function that allows it to connect directly to a telephone line, and will also be equipped with a function that allows it to cooperate with information mobile devices such as smartphones and tablets. (e) Head-mounted devices will be integrated into a spectacle-type or visor-type structure, omitting individual power supplies and I / O interfaces. In doing so, other internal components will be omitted as much as possible to reduce size and weight, and the entire device will be packaged to be as airtight as possible for waterproofing and dustproofing. (h) This device must be designed so as not to impair the user's hearing environment, and it is desirable that it does not directly block the ear canal but is open to the outside air. Furthermore, quietness to the surroundings is also necessary, so it is necessary to sufficiently suppress sound leakage from the earphones and electronic sounds from the equipment. Bone conduction type earphones are suitable for achieving this objective, but since bone conduction may not be audible when the ambient noise is loud, it is also acceptable to have a design that is like conventional earphones or headphones but with an open section. (t) The shape of the display portion will be either a glasses type with a narrow vertical width that covers only the lower part of the field of view, or a visor type that covers only the upper part of the field of view, so as to ensure that the majority of the user's visual environment is secured. (C) The device will be equipped with an AI function that recognizes the wearer's voice and provides simple advice through easy operation or by simply mumbling freehand (referred to as the "help function" in this paper). In order to effectively utilize this function, the head-mounted device will need a microphone that accurately captures the user's mumbling and a function that accurately authenticates that the mumbling is indeed from the user. (Re) The helmet will be a helmet type with a visor, and all functions of the head-mounted device will be integrated into the helmet. Cables and the main unit will not be included. Head-mounted devices and connecting cables are frequently attached and detached, and while prioritizing lightness and design often results in fragile structures, this product, primarily intended for professional use, requires robustness and durability to prevent easy breakage. Its robust, lightweight, and high-quality design makes it user-friendly. When integrated with the main unit, it enables long-term operation, providing intensive information gathering, sharing, and analysis, as well as real-time display and voice guidance of analysis results and advice, thereby assisting the wearer in easily achieving their work tasks. [Effects of the Invention]

[0006] The same symbols are used for the means, and the corresponding effects are described. (i) By fully synchronizing the collected data, 3E information—what was seen, what was heard, and what kind of conversations took place at that time—can be easily extracted without the need for any data transformation or synthesis. Because data collection is performed efficiently and automatically, users can focus solely on utilizing the collected data, rather than wasting time on synthesis and transformation. (b) By connecting the head-mounted device, which has been made as lightweight as possible, with the main unit, which has a full range of functions, using a dedicated cable to integrate them, the portability and activity sustainability of the device are improved, enabling long-term remote 3E sharing and analysis. (h) By guaranteeing sufficient operating time, it is possible to continuously collect local 3E information for extended periods, thereby gathering sufficient information necessary for recording, analysis, etc. (ii) It becomes possible to share information in real time with remote locations. (e) Integration allows for one-action attachment and detachment, and enables miniaturization and weight reduction, resulting in less fatigue during prolonged use and enabling use during moderately strenuous activities. Furthermore, integration physically preserves the connection state between devices, laying the foundation for the next innovation that prioritizes connection stability. Considering weather resistance, toughness, and price rationality, it is also possible to create category divisions according to the main application. (h) It becomes possible to engage in activities while conversing using earphones or headphones with a microphone. Also, by not completely blocking the ear canal, it is possible to always hear the sounds around you while using them. *Since the wearer is exposed to local events, it is necessary to enhance safety and mental stability, and to obtain the most accurate 3E. (T) By not covering the center of the field of vision and maximizing the area the wearer can directly see, reality perception can be made more reliable, making it easier to ensure safety and security during use. In addition, by pre-training the camera and AI functions on points of focus, it may be possible to display warnings even for things outside of the wearer's awareness through immediate analysis of the collected video. (C) When the wearer mutters something as if to confirm something, the AI ​​interprets it as a question and provides an answer, allowing the wearer to immediately understand the gist of the question and concentrate on activities and conversations. (i) When worn on a helmet while driving a car or motorcycle, it can provide immediate feedback on nearby hazards, including construction sites, disaster relief areas, and other dangerous locations. It also allows for checking road navigation information without significantly changing one's line of sight, and because it is helmet-type, various sensors can be mounted on the rear and above, enabling the transmission and display of fall and collision warnings. Furthermore, when used while driving, continuous charging of the main unit from the vehicle's power supply, the handle grips, and contact charging gloves would allow for extended use while mounting many sensors on the helmet, leading to further improvements in safety. This device allows for the easy accumulation of synchronized, integrated, and highly reliable 3E data, enabling analysis and application by user companies, organizations, and software developers, thereby contributing to increased efficiency and accuracy in business operations. Furthermore, the immediate analysis and application of this 3E data allows for immediate feedback to be provided directly to practical work. Furthermore, by allowing practitioners to use AI devices, it becomes possible to clarify the capabilities required of next-generation AI and the nuances of practical support, which directly leads to the feeding back of a vast amount of empirical data to researchers to make it even more useful in society. The purpose of this device is to revolutionize the future with AI, elevating it to new heights from the user's perspective, by providing a different viewpoint from AI developers—by real-world individuals who best embody the "ideal form" of AI. I believe this invention has the potential to become the foundation for the development of the next generation of the AI ​​industry. [Brief explanation of the drawing]

[0007] [Figure 1] This shows the basic components of the present invention. [Figure 2] This is an example of the wearing method of the present invention. (Earphone + Glasses type) [Figure 3] An example of the wearing method of the present invention. (Headphone + visor type) [Figure 4] The invisible part generated by the product of the present invention is illustrated. [Figure 5] The part incorporating a camera, a sensor, etc. is illustrated. [Figure 6] The shape of the head-mounted device as viewed from the wearer's perspective is shown. [Figure 7] The shape of the glasses-type head-mounted device as viewed from four directions is illustrated. [Figure 8] The head-mounted device (headphone + visor type) is illustrated. [Figure 9] An example of the display of the meeting record and the AI answer to personal questions is shown. (Reference example 1 of the help function) [Figure 10] An example of analyzing and displaying a preset attention target from surrounding video information is shown. (Reference example 2 of the help function)

Mode for Carrying Out the Invention

[0008] Hereinafter, embodiments of the present invention will be described. (A) Exclude power supplies, wireless communication devices, etc. from earphones with a microphone or headphones with a microphone (regardless of bone conduction or tympanic conduction), integrate a wearable camera and a monitor sized to cover the lower (upper) end of the field of vision (head-mounted device), and adopt a mechanism that enables obtaining information with a unified standard and synchronized information. (1) In addition, a type that mounts all the functions of the head-mounted device on a helmet with a visor needs to be prepared for activities that require head protection, such as use in unsafe locations or for driving a motorcycle. (B) Prepare a device (main body) having functions such as power supply, communication, recording, and analysis for the head-mounted device, mount it at a position excluding the head (a place where activities such as the neck, shoulders, chest, back, waist, and feet are not hindered), and connect the two with a power transmission and cable to form one device. (Fig. 1) (h) Equipped the main unit with the largest possible capacity battery. (ii) The main unit will be equipped with a wireless communication function that allows it to connect directly to a telephone line, and a Wi-Fi function that allows it to connect with smartphones and tablets. (e) The head-mounted device will be integrated into a spectacle-type or visor-type structure, omitting individual power supplies and I / O interfaces. In doing so, it will be made smaller and lighter by omitting other internal components as much as possible, and the entire device will be packaged to be as airtight as possible for waterproofing and dustproofing. (Figures 7 and 8) (h) This device must be designed so as not to impair the user's hearing environment, and it is desirable that it does not directly block the ear canal but is open to the outside air. Furthermore, quietness to the surroundings is also necessary, so it is necessary to sufficiently suppress sound leakage from the earphones and electronic sounds from the equipment. Bone conduction type earphones are suitable for achieving this objective, but since bone conduction may not be audible when the ambient noise is loud, it is also acceptable to use a conventional earphone or headphone type with an open section. (Figures 7, 8) (t) The shape of the display portion will be either a glasses-type with a narrow vertical width that covers only the lower part of the field of view, or a visor-type that covers part of the field of view, so as to ensure that the majority of the user's visual environment is secured. (Figures 2, 3, 4) (C) The device will use AI to recognize the wearer's voice and provide simple advice based on a few simple actions or freehand whispers. In order to effectively utilize this function, the head-mounted device will need a microphone to accurately capture the user's whispers and a function to accurately authenticate that the whispers are indeed from the user. (Figures 8 and 9) (i) The above-mentioned head-mounted device is mounted on a safety helmet with a visor, a driving helmet, etc. The only external differences from a regular helmet are that the helmet has openings for sensors in the necessary places and a visor for displaying information. The present invention is constructed as described above. Examples of the effects of the present invention are listed below. (i) This invention investigates, records, and analyzes 3E information of the actual site and objects, and provides immediate feedback of the results to the user themselves through a help function, thereby assisting the user in information gathering and organization, and deepening their activities. (b) Because data collection is performed with a single device, it becomes easy to synchronize the time-collected data from video, audio, and various other sensors, so the obtained information can be handled as highly reliable data. (h) By linking video analysis, facial recognition, and business card analysis, and adding transcription and text summarization functions, it becomes possible to easily create meeting minutes that clearly show individual names and timelines. These information processing tasks may be performed by external devices connected via wireless communication rather than by the main unit. (ii) Through video analysis from recordings, image search, temperature analysis (using optional thermal cameras, etc.), and audio analysis, the 3E can be shared remotely simply by the wearer of the device walking while visually observing and explaining. (e) During on-the-job training (OJT), the trainee can have a visual experience almost identical to the trainee's without having to perform cumbersome camera operations, allowing the educator to provide more appropriate instruction even in remote locations. (Ideally, the trainee should be able to perform operations such as zooming in and out using voice commands or remote control, without needing to use their hands.) Conversely, this is also effective in that it allows students to share the perspective of their instructors. (h) When multiple cameras are installed, it may be possible to create a 3D CAD model of the site situation through 3D analysis of the video footage. The site situation can also be visualized by a third party in a remote location. Accuracy can be further improved by incorporating an optical distance sensor. (T) By registering items to be lost with AI and walking around, you can search the area using two eyes: your own eyes and the built-in camera with AI analysis capabilities. (C) In the case of a helmet-type device, by equipping the head-mounted device with monitoring sensors including those on the top, sides, and rear, it becomes possible to observe images from all directions, distance, temperature, relative velocity, etc., so that dangers from collisions, flying objects, openings, and heat sources can be detected and responded to. The above are just examples of the effects, but countless other ways of utilizing them are possible depending on the activity in which they are used. [Explanation of Symbols]

[0009] 1. Head-mounted device 2 Charging / Transmission Cables 3. Main unit 4. Head-mounted device (headphones + visor type) 5. Installation locations for cameras, sensors, etc. 6. Detachable power supply + transmission cable (Not limited to connectors, magnetic types are also acceptable.) 7. Installation locations for cameras, sensors, etc. 8. Ear canal opening (hole, mesh, etc.) 9. Examples of speaker profile displays and conversation summary displays. 10. An example of interpreting a voiceprint user's tweet as a question and displaying an answer. 11. Examples of detecting and displaying hazardous areas and targets for investigation.

Claims

1. This device integrates a wearable camera, microphone-equipped earphones (or headphones), monitor, and various sensors, which are worn on the head like glasses, a visor, or headphones. A main unit that handles power supply, external communication, temporary storage, and information processing is worn elsewhere, and these components are connected to each other by power and transmission cables.

2. This device collects and analyzes synchronized and coordinated data obtained from cameras, microphones, and other sensors of a head-mounted device, utilizing information processing mechanisms at remote locations via the main unit or external communication.

3. This device is equipped with a "help function" that provides users with immediate feedback, such as analysis information and messages from remote locations.

4. This is a wearable device for sharing information remotely, facilitating the "seeing," "hearing," and "speaking" experiences.

5. For activities requiring head protection, we offer a lineup of helmet-type headgear with a visor.

6. For helmet-type devices with visors worn while driving cars, motorcycles, etc., the system will allow power to be supplied from the vehicle's power source through the handlebars and gloves to the main unit.

Images