Smart glasses including protection shield and maintenance·repair management method using the same

Abstract

A smart glass according to an embodiment of the present invention comprises: a body portion; a protection shield connected to both ends of the body portion; a window connected to one side of the body portion; and a projector disposed in the body portion; wherein the projector switches the display to project onto the protection shield, which is a first display, when the shield angle, which is an angle between the body portion and the protection shield, is greater than or equal to a preset maximum angle, and projects onto the window, which is a second display, when the shield angle is less than or equal to a preset minimum angle.

Description

Technology Field

[0001] The present invention relates to smart glasses including a protection shield and a maintenance and management method utilizing the same. More specifically, the invention relates to smart glasses including a protection shield that serves as a protective film for blocking ultraviolet rays while also being used as a display, and a maintenance and management method utilizing the same. Background Technology

[0002] Virtual Reality (VR) is a technology that creates a virtual environment to make it feel real, and Augmented Reality (AR) is a technology that projects virtual information onto the real surrounding environment. Recently, AR and VR have been evolving globally into Mixed Reality (MR) and Augmented Virtual Reality (AV), which create environments by simultaneously fusing real and virtual information, based on the concept of the Reality-Virtuality Continuum.

[0003] Furthermore, a Head Mounted Display (HMD) refers to a display device worn on a user's head. Head Mounted Displays are used as display devices for implementing the aforementioned Virtual Reality or Augmented Reality, and their applications are gradually increasing.

[0004] Recently, these augmented reality devices have been utilized in various industries, such as shipbuilding, nuclear power, medical, and agriculture, for field management systems for maintaining and repairing sites or for collaboration systems where multiple workers collaborate on a single task, thereby enhancing work capabilities and contributing to increased work efficiency.

[0005] However, the maintenance management technology based on the augmented reality providing device described above has disadvantages, such as limited cost for multiple workers to use because it is based on an augmented reality providing device that is expensive, relatively heavy, and cumbersome to attach and detach. Additionally, when workers must wear the device for a long time, dizziness may occur due to focus misalignment, and frequent attachment and detachment of the device during work causes inconvenience, resulting in a low actual usage rate.

[0006] Furthermore, if the industrial site where augmented reality devices are used is outdoors or involves accepting certain risks, problems such as skin aging caused by UV rays due to prolonged exposure to sunlight, malfunction of the augmented reality device due to UV rays, heat, and other hazards, and the need for separate face shields have occurred.

[0007] In other words, there is an urgent need to invent a maintenance management method that utilizes an augmented reality providing device suitable for use in industrial settings in order to solve these problems. Prior art literature

[0008] (Patent Document 0001) KR 10-2084723 B1 The problem to be solved

[0009] The present invention was devised to solve the problems of the prior art as described above, and aims to provide smart glasses including a protection shield that recognizes a predetermined object included in the user's field of vision and displays the results of an AI analysis of the object on at least one display, and a maintenance and management method utilizing the same.

[0010] In addition, the present invention aims to provide smart glasses comprising a protection shield that performs the role of a protective film blocking ultraviolet rays and is also used as a display at a predetermined angle, and a maintenance and management method utilizing the same.

[0011] In addition, the present invention aims to provide smart glasses including a protection shield incorporating a PinTILT method and a maintenance and repair management method utilizing the same.

[0012] In addition, the present invention aims to provide smart glasses including a protection shield that requests a diagnosis from an expert through real-time screen sharing, and a maintenance management method utilizing the same.

[0013] In addition, the present invention aims to provide smart glasses comprising a protection shield that displays the state of an analyzed object on at least one display, and a maintenance management method utilizing the same.

[0014] However, the technical problems that the present invention and the embodiments of the present invention aim to solve are not limited to the technical problems described above, and other technical problems may exist. means of solving the problem

[0015] A smart glass according to an embodiment of the present invention comprises: a body portion; a protection shield connected to both ends of the body portion; a window connected to one side of the body portion; and a projector disposed in the body portion; wherein the projector switches the display to project onto the protection shield, which is a first display, when the shield angle, which is an angle between the body portion and the protection shield, is greater than or equal to a preset maximum angle, and projects onto the window, which is a second display, when the shield angle is less than or equal to a preset minimum angle.

[0016] In addition, the smart glasses according to an embodiment of the present invention include a camera disposed in the body portion; and the camera captures a field of view image displayed on the first display and the second display.

[0017] In addition, the smart glasses according to an embodiment of the present invention include a rotating part that connects the body part and the protection shield and provides a rotatable structure; and a position sensor included in the rotating part; wherein the position sensor senses the shield angle.

[0018] In addition, the above protection shield is composed of a polycarbonate film that blocks ultraviolet rays.

[0019] Additionally, the window includes a projector; a plurality of pin mirrors; and a concave mirror; and displays augmented reality content in a PinTilt manner in which light emitted from the projector is reflected by the concave mirror and then transmitted to the user's pupil via the plurality of pin mirrors.

[0020] Meanwhile, a maintenance management method using smart glasses according to an embodiment of the present invention is a maintenance management method using smart glasses provided by an object management application executed by at least one processor of the smart glasses of claim 1, comprising: a step of calculating a shield angle between the body part and the protection shield and a window angle between the body part and the window; a step of determining a display mode based on the calculated shield angle and window angle; and a step of displaying augmented reality content on the determined display; wherein the step of displaying augmented reality content comprises: a step of capturing a field of view image based on a camera; a step of recognizing at least one object included in the captured field of view image based on an object recognition program; a step of obtaining analysis information for the recognized object based on an object information analysis deep learning neural network; and a step of augmenting the field of view image with the obtained analysis information.

[0021] Additionally, the step of determining a display mode based on the shield angle includes: determining a first display mode for projecting the augmented reality content onto the protection shield if the shield angle is between a preset minimum angle and a preset maximum angle; and determining a second display mode for projecting the augmented reality content onto the window if the shield angle is less than or equal to a preset minimum angle.

[0022] Additionally, the step of determining a display mode based on the window angle includes, if the window angle is a first angle, a step of determining a second display mode for projecting the augmented reality content onto the window, and if the window angle is smaller or larger than the first angle, a step of determining a stop mode for stopping the operation of the smart glasses.

[0023] In addition, the step of determining the first display mode includes the step of adjusting the projection angle of the projector.

[0024] Additionally, the step of adjusting the projection angle of the projector comprises: a step of dividing the field of view image into a shield-transmitting area and a shield-non-transmitting area; a step of calculating the ratio of the shield-non-transmitting area to the shield-transmitting area; and a step of determining to adjust the projector by a first projection angle according to the projection angle pre-set for each ratio of the shield-non-transmitting area.

[0025] Additionally, the step of obtaining analysis information for the recognized object includes the step of inputting at least one recognized object into the object information analysis deep learning neural network, and the step of obtaining analysis information for each of the at least one object output from the object information analysis deep learning neural network.

[0026] Additionally, the step of displaying augmented reality content including the acquired analysis information includes a first object box surrounding a first object included in the field of view image, and a step of augmenting the first analysis information for the first object.

[0027] In addition, a maintenance management method using smart glasses according to an embodiment of the present invention further includes the step of transmitting the field of view image to at least one expert terminal and the step of performing remote consultation by displaying the answer data generated from the at least one expert terminal as augmented reality content on the determined display.

[0028] Meanwhile, a maintenance management system utilizing smart glasses according to an embodiment of the present invention includes the smart glasses of claim 1 and further includes a terminal that provides augmented reality content to the smart glasses, and the terminal performs edge computing with a service providing server. Effects of the invention

[0029] The smart glasses including a protection shield according to an embodiment of the present invention and the maintenance management method using the same recognize a predetermined object included in the user's field of vision and display the content of the analysis of the object by AI on at least one display, thereby allowing the user to immediately acquire information related to the object, such as problems and current status of the object, and manage the object in real time, thus shortening the time required to acquire information related to the object and significantly increasing the speed of management tasks.

[0030] In addition, the smart glasses including a protection shield according to an embodiment of the present invention and the maintenance and management method using the same perform the role of a protective film that blocks ultraviolet rays and are also used as a display at a predetermined angle, thereby eliminating the need to wear a sun cap separately from the smart glasses and preventing skin aging and eye diseases that may occur when used outdoors, while increasing the utility of the smart glasses.

[0031] In addition, the smart glasses including a protection shield according to an embodiment of the present invention and the maintenance and management method using the same have the effect of increasing mobility and providing comfort to the user during device usage by incorporating a PinTILT method, with high light efficiency and light weight due to the thin lens thickness.

[0032] In addition, the smart glasses including a protection shield according to an embodiment of the present invention and the maintenance management method using the same have the effect of significantly increasing the overall efficiency of an object by preventing economic, time, procedural, and environmental losses that may occur due to a lack of familiarity with the characteristics related to the object by requesting a diagnosis from an expert through real-time screen sharing.

[0033] However, the effects obtainable from the present invention are not limited to those mentioned above, and other unmentioned effects can be clearly understood from the description below. Brief explanation of the drawing

[0034] FIG. 1 is a conceptual diagram of an object management service provision system according to an embodiment of the present invention. FIG. 2 is an internal block diagram of smart glasses according to an embodiment of the present invention. FIG. 3 is an example showing the shape of a smart glass including a protection shield according to an embodiment of the present invention. FIG. 4 is an example of a drawing illustrating the pin tilt method used by smart glasses in an embodiment of the present invention. FIGS. 5 and FIGS. 6 are examples showing the shape of a body portion of a smart glass according to an embodiment of the present invention. FIG. 7 is a flowchart illustrating a method for a smart glass to provide an object management service according to an embodiment of the present invention. FIGS. 8 and 9 are examples for explaining display modes according to the angle of a protection shield according to an embodiment of the present invention. FIG. 10 is an example showing a field of view image displayed on a first display according to a projection angle according to an embodiment of the present invention. FIG. 11 is an example of an object management interface displaying analysis information according to an embodiment of the present invention. Specific details for implementing the invention

[0035] The present invention is capable of various modifications and may have various embodiments; therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, and the methods for achieving them, will become clear by referring to the embodiments described in detail below together with the drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various forms. In the following embodiments, terms such as "first," "second," etc., are used not in a limiting sense but for the purpose of distinguishing one component from another. Furthermore, singular expressions include plural expressions unless the context clearly indicates otherwise. Also, terms such as "include" or "have" mean that the features or components described in the specification exist, and do not preclude the possibility that one or more other features or components may be added. Additionally, in the drawings, the size of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are arbitrarily depicted for convenience of explanation, so the present invention is not necessarily limited to what is illustrated.

[0036] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same reference numerals, and redundant descriptions thereof will be omitted.

[0038] FIG. 1 is a conceptual diagram of an object management service provision system according to an embodiment of the present invention.

[0039] Referring to FIG. 1, an object management service providing system (hereinafter, service providing system) according to an embodiment of the present invention can provide a maintenance and management service for an object (hereinafter, object management service) by recognizing a predetermined object included in a user's field of vision on smart glasses including a protection shield, and displaying the content of the analysis of the object by AI on at least one display included in the smart glasses.

[0040] In an embodiment, a service providing system that implements the above object management service can be connected through smart glasses (100), a service providing server (200), and a network (10: Network).

[0041] Here, the network (10) according to the embodiment refers to a connection structure capable of exchanging information between each node, such as the smart glasses (100) and / or the service provider server (200), and examples of such a network (10) include, but are not limited to, a 3GPP (3rd Generation Partnership Project) network, an LTE (Long Term Evolution) network, a WIMAX (World Interoperability for Microwave Access) network, the Internet, a LAN (Local Area Network), a Wireless LAN (Wireless Local Area Network), a WAN (Wide Area Network), a PAN (Personal Area Network), a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, a DMB (Digital Multimedia Broadcasting) network, etc.

[0042] Hereinafter, the smart glasses (100) and service provision server (200) implementing the service provision system will be described in detail with reference to the attached drawings.

[0044] - Smart Glass (100: Smart Glass)

[0045] The smart glasses (100) according to an embodiment of the present invention may be a predetermined computing device having an object management application (hereinafter, application) installed that provides an object management service.

[0046] Here, the smart glasses (100) according to the embodiment is a device that simultaneously provides the function of glasses for viewing objects in front and the function of a computer, and may be a wearable type computing device that can be worn on a user's face. In the following, a person who receives an object management service by wearing the smart glasses (100) may be referred to as a user.

[0047] Additionally, according to an embodiment, the smart glasses (100) may further include a predetermined server computing device that provides an object management service environment.

[0048] FIG. 2 is an internal block diagram of smart glasses according to an embodiment of the present invention.

[0049] Referring to FIG. 2, from a functional perspective, the smart glasses (100) may include a memory (110), a processor assembly (120), a communication processor (130), an interface module (140), an input system (150), a sensor system (160), and a display system (170). These components may be configured to be included within the housing of the smart glasses (100).

[0050] Specifically, in memory (110), an application (111) is stored, and the application (111) can store one or more of various applications, data and commands for providing an object management service environment.

[0051] Additionally, the memory (110) according to the embodiment may include augmented reality content, an image buffer, a position engine, an augmented reality content display engine, etc., for providing an augmented reality environment.

[0052] That is, the memory (110) can store commands and data, etc., that can be used to create an object management service environment and / or an augmented reality environment.

[0053] In addition, the memory (110) may include a program area and a data area.

[0054] Here, the program area according to the embodiment may be linked between the operating system (OS) and functional elements that boot the smart glasses (100), and the data area may store data generated by the use of the smart glasses (100).

[0055] Additionally, the memory (110) may include at least one non-transient computer-readable storage medium and a transient computer-readable storage medium.

[0056] For example, the memory (110) may be various storage devices such as ROM, EPROM, flash drive, hard drive, etc., and may include web storage that performs the storage function of the memory (110) on the internet.

[0057] The processor assembly (120) may include at least one processor capable of executing instructions of an application (111) stored in memory (110) to perform various tasks for creating an object management service environment.

[0058] In an embodiment, the processor assembly (120) can control the overall operation of the components through an application (111) of the memory (110) to provide object management services.

[0059] This processor assembly (120) may be a system-on-chip (SOC) suitable for smart glasses (100) including a central processing unit (CPU) and / or a graphics processing unit (GPU), etc., and may execute an operating system (OS) and / or application programs stored in memory (110), and may control each component mounted on the smart glasses (100).

[0060] Additionally, the processor assembly (120) can communicate internally with each component via a system bus and may include one or more predetermined bus structures, including a local bus.

[0061] Additionally, the processor assembly (120) may be implemented by including at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and other electrical units for performing functions.

[0062] The communication processor (130) may include one or more devices for communicating with an external device. The communication processor (130) may communicate via a wireless network.

[0063] More specifically, the communication processor (130) can communicate with smart glasses (100) that store content sources for implementing an object management service environment, and can communicate with various user input components such as a controller that receives user input.

[0064] In an embodiment, the communication processor (130) can transmit and receive various data related to object management services to other smart glasses (100) and / or external servers, etc.

[0065] This communication processor (130) can wirelessly transmit and receive data with at least one of a base station, an external terminal, and any server on a mobile communication network built through a communication device capable of performing technical standards for mobile communication or communication methods (e.g., LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G NR (New Radio), WIFI) or short-range communication methods.

[0066] The interface module (140) can connect the smart glasses (100) to communicate with one or more other devices. Specifically, the interface module (140) may include one or more wired and / or wireless communication devices compatible with different communication protocols.

[0067] Through this interface module (140), the smart glasses (100) can be connected to various input / output devices.

[0068] For example, the interface module (140) can be connected to an audio output device, such as a headset port or a speaker, to output audio.

[0069] Although it has been described as an example in which an audio output device is connected through an interface module (140), an embodiment in which it is installed inside the smart glasses (100) may also be included.

[0070] Additionally, for example, the interface module (140) may be connected to an input device such as a keyboard and / or mouse to obtain user input.

[0071] This interface module (140) may be configured to include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module, an audio I / O (Input / Output) port, a video I / O (Input / Output) port, an earphone port, a power amplifier, an RF circuit, a transceiver, and other communication circuits.

[0072] The input system (150) can detect user input related to the object management service (e.g., gestures, voice commands, button operation, or other types of input).

[0073] Specifically, the input system (150) may include a predetermined button, a touch sensor and / or an image sensor (161) that receives user motion input.

[0074] More specifically, the input system (150) can detect input based on a user's hand movements based on a non-contact sensor (e.g., an infrared sensor and / or a gesture sensor) of the sensor system (160) described later.

[0075] Additionally, the input system (150) can be connected to an external controller through an interface module (140) to receive user input.

[0076] The sensor system (160) may include various sensors such as an image sensor (161), a position sensor (IMU, 163), an audio sensor (165), a distance sensor, a proximity sensor, a contact sensor, an infrared sensor, and a gesture sensor.

[0077] Here, the image sensor (161) can capture images and / or videos of the physical space around the smart glasses (100).

[0078] In an embodiment, the image sensor (161) can capture and acquire various images and / or videos related to the object management service.

[0079] Additionally, the image sensor (161) can be positioned on the front or / and rear of the smart glasses (100) to capture an image of the positioned direction, and can capture a physical space through a camera positioned toward the outside of the smart glasses (100).

[0080] Such an image sensor (161) may include an image sensor device and an image processing module. Specifically, the image sensor (161) may process still images or video obtained by an image sensor device (e.g., CMOS or CCD).

[0081] Additionally, the image sensor (161) can process a still image or video obtained through the image sensor device using an image recognition process (e.g., OCR, etc.) and / or an image processing module to extract necessary information and transmit the extracted information to a processor.

[0082] The image sensor (161) may be a camera assembly including at least one camera. The camera assembly may include a general camera that captures the visible light band, and may further include special cameras such as an infrared camera, a stereo camera, and an all-around camera capable of capturing 360 degrees.

[0083] Additionally, the image sensor (161) described above may be included in and operated by the smart glasses (100) according to the embodiment, or it may be included in an external device (e.g., an external server, etc.) and operated through interlocking based on the communication processor (130) and / or interface module (140) described above. Hereinafter, the image sensor (161) may be referred to as a camera.

[0084] The position sensor (IMU, 163) can detect at least one of the movement and acceleration of the smart glasses (100). For example, it may be composed of a combination of various position sensors such as an accelerometer, a gyroscope, and a magnetometer.

[0085] In addition, the location sensor (IMU, 163) can recognize spatial information about the physical space around the smart glasses (100) by linking with a location communication processor (130), such as a GPS of the communication processor (130).

[0086] The audio sensor (165) can recognize sounds around the smart glasses (100).

[0087] Specifically, the audio sensor (165) may include a microphone capable of detecting voice input from a user using smart glasses (100).

[0088] In the embodiment, the audio sensor (165) can receive voice data necessary for object management services from the user.

[0089] The display system (170) can output various information related to object management services as graphic images.

[0090] Specifically, the display system (170) may include a transparent glass display that transmits light from the physical space around the smart glasses (100) to reach the user's eyes, while simultaneously reflecting augmented reality content displayed by the display system (170) toward the user's eyes.

[0091] This display system (170) may include a left display corresponding to the left eye of a user wearing smart glasses (100) and a right display corresponding to the right eye.

[0092] By outputting different images with a parallax offset between the left display and the right display as augmented reality content, the user can perceive the augmented reality content as a three-dimensional image.

[0093] In an example, the display system (170) can display various user interfaces for object management services (in an example, an object management interface, etc.).

[0094] Such displays may include at least one of a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, a 3D display, and an e-ink display.

[0095] The smart glasses (100) including the components described above may store at least one field of view image, object analysis information, and / or augmented reality content in a memory (110) according to an embodiment.

[0096] Meanwhile, according to an embodiment, the smart glasses (100) may further perform at least some of the functional operations performed by the service providing server (200) described later.

[0097] These smart glasses (100) may be equipped with a protection shield (310) in hardware.

[0098] FIG. 3 is an example showing the shape of a smart glass including a protection shield according to an embodiment of the present invention.

[0099] Referring to FIG. 3, in the embodiment, the smart glass (100) may include a protection shield (310), a window (320) and / or a body part (330).

[0100] The protection shield (310) may be a curved protective film to protect the user's face from ultraviolet rays.

[0101] Additionally, the protection shield (310) may be positioned at a certain distance from the user's face when the user wears the smart glasses (100). At this time, the size of the protection shield (310) may be large enough to cover at least the user's entire face.

[0102] This protection shield (310) can be implemented with a material that blocks ultraviolet rays (e.g., polycarbonate).

[0103] Additionally, the protection shield (310) can operate as a first display that displays a predetermined content output by a projector provided on one side of the body part (330). At this time, the predetermined content can be displayed on the inner surface of the protection shield (310).

[0104] In another embodiment, the protection shield (310) may be a flexible display panel capable of displaying content on its own without a projector.

[0105] Additionally, the protection shield (310) can be connected to the body part (330) through the rotating part (340).

[0106] Additionally, the protection shield (310) can rotate up and down to have a predetermined angle with respect to the body part (330) based on the rotating part (340).

[0107] The window (320) may be a transparent glass display that reflects augmented reality content displayed by the display system (170) toward the user's eyes.

[0108] Specifically, the window (320) may be a glass display that uses a PinTILT method to obtain an image by arranging small mirrors at an angle inside a lens.

[0109] FIG. 4 is an example of a drawing illustrating the pin tilt method used by smart glasses in an embodiment of the present invention.

[0110] Referring to FIG. 4, the window (320) may include a projector (321), a plurality of pin mirrors (322) and / or concave mirrors (323).

[0111] The projector (321) may be a display source that projects light to show an image.

[0112] Additionally, light from the projector (321) can be reflected by a concave mirror (323) and then transmitted to the pupil via a plurality of pin mirrors (322).

[0113] That is, the window (320) using the pin-tilt method can be implemented with a thin lens thickness because the light reflection angle is wide.

[0114] These windows (320) can operate as a projector and / or a second display depending on the angle of the protection shield (310).

[0115] Specifically, the projector (321) of the window (320) can project a predetermined text, image, video and / or augmented reality content (hereinafter collectively referred to as augmented reality content) onto the first display and / or the second display.

[0116] More specifically, the projector (321) of the window (320) can project augmented reality content onto the second display when the projection shield (310) rotates upward, as this is a situation where the first display is removed. Meanwhile, when the projection shield (310) rotates downward, it can project augmented reality content onto the first display as a projector. Thus, the projection shield (310) can operate as a first display for displaying augmented reality content.

[0117] Additionally, the projector (321) of the window (320) may have its projection angle adjusted when projecting augmented reality content onto the first display. Further details regarding this will be described later.

[0118] Additionally, the window (320) may be shaped like glasses so that an image can be formed in the user's eye.

[0119] Additionally, the window (320) may be positioned at a certain distance from the user's eyes when the user wears the smart glasses (100). At this time, the size of the window (320) may be large enough to cover at least the user's eyes completely.

[0120] Additionally, the window (320) may be implemented with a structure capable of vertical rotation. Depending on the rotation of the window (320), the operation of the smart glasses (100) may be determined. At this time, the rotating window (320) may operate only at a preset first window angle (e.g., 90 degrees), and may stop operating if the angle is smaller or larger than the first window angle. Additionally, the direction in which the window (320) rotates may be outward from the user's eyes.

[0121] Specifically, when the window (320) rotates upward, the operation of the smart glasses (100) may be stopped. Also, when the window (320) rotates downward, the operation of the smart glasses (100) may be started.

[0122] At this time, the upward rotation of the window (320) may mean a situation in which the window (320), which was perpendicular to the body part (330), moves to become horizontal or is inserted into the body part (330), or is removed from the user's sight. Conversely, the downward rotation of the window (320) may mean a situation in which the window (320) moves to become perpendicular to the body part (330) or is protruded from inside the body part (330), or is installed in the user's sight.

[0123] The operation of the smart glasses (100) may be determined by combining whether the window (320) rotates and whether the protection shield (310) rotates.

[0124] When the protection shield (310) rotates upward, the projector (321) is automatically turned off and the second display window (320) can be operated (in an example, in a second display mode). In this state, when the window (320) rotates upward, the operation of the smart glasses (100) can be stopped (in an example, in a stop mode).

[0125] Additionally, when the protection shield (310) rotates downward, the projector (321) is automatically turned on and the protection shield (310), which is the first display, can be operated (in an example, in a first display mode). In this state, when the window (320) rotates upward, the operation of the smart glasses (100) is stopped (in an example, in a stop mode), and when the window (320) rotates downward, the operation of the smart glasses (100) can be resumed (in an example, in the first display mode).

[0126] The body part (330) may be an elliptical headband (band) with one side open to mount the smart glasses (100) on the user's head.

[0127] Additionally, the size of the body part (330) may be small enough to be mounted on the user's head and may provide a function that is adjustable according to the user's head circumference.

[0128] Additionally, a protection shield (310) and a window (320) may be provided on one side of the body part (330).

[0129] At this time, the window (320) may be positioned on the bottom surface (D) of the body part (330) so as to be located near the user's eyes when the user wears the smart glasses (100).

[0130] Additionally, the protection shield (310) may be positioned on the outer surface of the body part (330) so as to be freely rotatable up and down. That is, when the user rotates the protection shield (310) downward, the front of the window (320) is covered, and when the user rotates the protection shield (310) upward, the front of the window (320) is exposed.

[0131] Additionally, a first camera (161-1), a second camera (161-2), and / or a rotating part (340) may be disposed in the body part (330).

[0132] FIGS. 5 and 6 are examples showing the shape of a body portion of a smart glass according to an embodiment of the present invention. Specifically, FIG. 5 is a view of the body portion of the smart glass from the front, and FIG. 6 is a view of the body portion of the smart glass from above.

[0133] Referring to FIGS. 5 and 6, in the embodiment, the projector (321) may be positioned at the bottom of the first camera (161-1) and at the top of the window (320).

[0134] These projectors (321) may be ultra-small beam projectors that emit light to display a display on a protection shield (310) and / or a window (320).

[0135] To this end, the projector (321) is implemented with a structure that can rotate in a circular motion, and the angle of the projector (321) can be determined by setting a reference point (focus) on the protection shield (310) and / or window (320).

[0136] Specifically, when the user lowers the protection shield (310), the projector (321) can display content on the protection shield (310), which is the first display, and when the user raises the protection shield (310), the projector (321) can display content on the window (320), which is the second display. Further details regarding this will be described later.

[0137] In the embodiment, a first camera (161-1) may be positioned on the upper side of the center in the forward direction (F) of the body part (330).

[0138] Additionally, a second camera (161-2) may be positioned on the upper side of the center of the window (320) positioned in the body part (330) in the forward direction (F).

[0139] In the following description, the positions or operations of the first camera (161-1) and the second camera (161-2) are described separately, but the main camera is the first camera (161-1), and the content performed by the second camera (161-2) can be attributed to the content performed by the first camera (161-1).

[0140] The operation of the first and second cameras (161-1, 161-2) can be switched according to the angle of the protection shield (310).

[0141] In an embodiment, when the protection shield (310) rotates downward, the second camera (161-2) is covered, and a first display mode in which only the first camera (161-1) performs shooting can be operated. Meanwhile, when the protection shield (310) rotates upward, the first camera (161-1) is covered, and a second display mode in which only the second camera (161-2) performs shooting can be operated.

[0142] These first and second cameras (161-1, 161-2) can capture and recognize recognizable information. For example, the recognizable information may be certain objects including plants, insects, and pesticides.

[0143] The rotating part (340) may be a rotatable structure that allows the connected protection shield (310) to rotate up and down.

[0144] Additionally, the rotating part (340) can perform a stopper function that fixes the projection shield (310) at a predetermined angle. At this time, the display mode may be switched according to the predetermined angle. For example, if the projection shield (310) has an angle of 80 degrees or more relative to the body part (330), it may be determined as a first display mode. On the other hand, if the projection shield (310) has an angle of 20 degrees or less, it may be determined as a second display mode.

[0145] These rotating parts (340) can be symmetrically arranged one each in the left (L) direction and the right (R) direction so as to come into contact with the protection shield (310).

[0146] Additionally, each rotating part (340) may include a first position sensor (163-1) and / or a second position sensor (163-2).

[0147] The first and second position sensors (163-1, 163-2) can be positioned on corresponding sides based on the center of the body part.

[0148] These first and second position sensors (163-1, 163-2) can detect the movement of the protection shield (310) and / or the movement of the user.

[0149] At this time, if the sensing value sensed by the first and second position sensors (163-1, 163-2) changes gradually, it can be recognized that the user is moving their head. However, if the sensing value sensed by the first and second position sensors (163-1, 163-2) stops at a specific value, it is considered that the value is temporarily fixed according to the stopper, and it can be recognized that the user has rotated the projection shield (310).

[0150] These smart glasses (100) can detect a user's gesture input and / or voice input based on at least one sensor included in the sensor system (160) (e.g., a non-contact sensor including an infrared sensor and / or a gesture sensor).

[0151] Additionally, in the embodiment, the smart glasses (100) can detect the user's head movements based on the first and second position sensors (163-1, 163-2).

[0152] Additionally, in the embodiment, the smart glass (100) can calculate the angle between the protection shield (310) and the body part (330).

[0153] In addition, in the embodiment, the smart glasses (100) can switch the display mode based on the calculated angle.

[0154] In addition, in the embodiment, the smart glasses (100) can recognize and analyze objects captured based on the first and second cameras (161-1, 161-2).

[0156] - Service Providing Server (200: Service Providing Server)

[0157] Meanwhile, the service providing server (200) according to an embodiment of the present invention can perform a series of processes to provide object management services.

[0158] In detail, in an embodiment, the service providing server (200) can provide the object management service by exchanging data necessary to enable the object management service process to run on an external device such as smart glasses (100) with said external device.

[0159] More specifically, in the embodiment, the service providing server (200) can provide an environment in which an application (111) can operate on an external device (in the embodiment, smart glasses (100), etc.).

[0160] To this end, the service providing server (200) may include an application program, data and / or commands, etc., for the application (111) to operate, and may transmit and receive various data based thereon with the external device.

[0161] In addition, in the embodiment, the service providing server (200) can perform various deep learning for object management services by linking with a deep-learning neural network.

[0162] Here, the deep learning neural network according to the embodiment may include a Convolutional Neural Network (CNN), R-CNN (Regions with CNN features), Fast R-CNN, Faster R-CNN, Mask R-CNN, YOLO, etc., and may include any deep learning neural network that includes an algorithm capable of performing the embodiments described below, and the embodiments of the present invention do not limit or restrict such deep learning neural networks themselves.

[0163] In the embodiment, the deep learning neural network may be performed based on YOLO Object Detection, a deep learning-based approach for object detection and object recognition.

[0164] At this time, according to the embodiment, the deep learning neural network may be directly installed on the service providing server (200) or may operate as a separate device from the service providing server (200) to perform deep learning for the object management service.

[0165] In the following embodiments, the deep learning neural network is directly installed on the service provider server (200) to perform deep learning, and the description is based on the embodiment.

[0166] In addition, in the embodiment, the service providing server (200) can read a predetermined deep learning neural network driving program built to perform the deep learning from memory and perform the deep learning described below according to the predetermined deep learning neural network system read out.

[0167] In an embodiment, the service providing server (200) can communicate with the smart glasses (100) using an edge computing method.

[0168] Edge computing refers to a method of performing computing at or near the physical location of a user or data source. By utilizing edge computing, complex tasks such as analysis can be processed closer to the user's terminal device, allowing the user to receive faster and more stable services.

[0169] At this time, the main computer of the central server can store and analyze the acquired information.

[0170] Accordingly, in the embodiment, the service providing server (200) may be an edge computing platform connected to a central server or may be implemented as a user's smartphone.

[0171] In addition, in the embodiment, the service providing server (200) can store and manage various applications, commands and / or data, etc. for implementing object management services.

[0172] In an example, the service providing server (200) can store and manage a field of view image, object analysis information, augmented reality content and / or an object management interface, etc.

[0173] However, in the embodiment of the present invention, the functional operations that the service providing server (200) can perform are not limited to those described above, and other functional operations may also be performed.

[0174] Meanwhile, referring further to FIG. 1, in the embodiment, the service providing server (200) as described above may be implemented as a predetermined computing device comprising at least one processor module (210) for data processing, at least one communication module (220) for exchanging data with an external device, and at least one memory module (230) for storing various applications, data and / or instructions for providing object management services.

[0175] Here, the memory module (230) can store one or more of an operating system (OS), various applications, data, and instructions for providing object management services.

[0176] In addition, the memory module (230) may include a program area and a data area.

[0177] Here, the program area according to the embodiment may be linked between the operating system (OS) and functional elements that boot the server, and the data area may store data generated according to the use of the server.

[0178] In the embodiment, the memory module (230) may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and may also be web storage that performs the storage function of the memory module (230) on the internet.

[0179] Additionally, the memory module (230) may be a recording medium that is detachable on the server.

[0180] Meanwhile, the processor module (210) can control the overall operation of each unit described above in order to implement an object management service.

[0181] This processor module (210) may be a system-on-chip (SOC) suitable for a server including a central processing unit (CPU) and / or a graphics processing unit (GPU), etc., and may execute an operating system (OS) and / or application programs stored in a memory module (230), and may control each component installed in the server.

[0182] Additionally, the processor module (210) can communicate internally with each component via a system bus and may include one or more predetermined bus structures, including a local bus.

[0183] Additionally, the processor module (210) may be implemented using at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and other electrical units for performing functions.

[0184] In the above description, it has been explained that the service providing server (200) according to the embodiment of the present invention performs functional operations as described above. However, depending on the embodiment, at least a portion of the functional operations performed by the service providing server (200) may be performed by an external device (e.g., smart glasses (100), etc.), and at least a portion of the functional operations performed by the external device may be further performed by the service providing server (200), and various other embodiments may be possible.

[0186] - A method in which smart glasses (100) provide object management services

[0187] In an embodiment, the smart glasses (100) may provide an object management service that displays an analysis of an object recognized by the protection shield (310) as the first display and / or the window (320) as the second display.

[0188] Hereinafter, a method in which at least one processor assembly (120) of a smart glass (100) according to an embodiment of the present invention provides an object management service will be described in detail with reference to the attached FIGS. 7 to 11.

[0189] In an embodiment of the present invention, at least one processor assembly (120) of the smart glasses (100) can execute at least one application (111) stored in at least one memory (110) or operate in a background state.

[0190] Hereinafter, the method of providing the object management service described above by the smart glasses (100) is described in a shortened manner, wherein at least one processor assembly (120) operates to execute instructions of the application (111) to perform the method.

[0191] FIG. 7 is a flowchart illustrating a method for a smart glass to provide an object management service according to an embodiment of the present invention.

[0192] Referring to FIG. 7, in the embodiment, the smart glasses (100) can determine the camera to use among the first camera and / or the second camera according to the shield angle of the protection shield (310). (S101)

[0193] In the embodiment, when the camera to be used is determined to be the first camera, the smart glasses (100) can operate a first display mode. On the other hand, when the camera to be used is determined to be the second camera, the second display mode can operate.

[0194] To determine this display mode, in the embodiment, the smart glasses (100) can calculate the angle of the protection shield (310).

[0195] FIGS. 8 and FIGS. 9 are examples for explaining display modes according to the angle of a protection shield according to an embodiment of the present invention. Specifically, FIG. 8 is an example for explaining a first display mode based on a first camera according to an embodiment of the present invention, and FIG. 9 is an example for explaining a second display mode based on a second camera according to an embodiment of the present invention.

[0196] Referring to FIG. 8, in the embodiment, the angle of the protection shield (310) may mean the angle that the protection shield (310) has when the body part (330) of the smart glass (100) is assumed to be a reference horizontal line.

[0197] In other words, in the embodiment, the smart glass (100) can calculate the shield angle (SA) between the horizontal line (L) and any point (P1, hereinafter the first point) of the protection shield (310).

[0198] At this time, the calculation of the shield angle (SA) can be performed based on position sensors (163-1, 163-2) mounted on the rotating part (340).

[0199] That is, in the embodiment, the smart glasses (100) can determine the first camera (161-1) as the camera to use if the shield angle (SA) is greater than or equal to a preset maximum angle (e.g., 80 degrees).

[0200] Additionally, in the embodiment, the smart glasses (100) can operate in a first display mode when the camera used is determined to be the first camera.

[0201] At this time, the first display mode may be a mode that displays a field of view image captured by the first camera on the first display, which is the protection shield (310).

[0202] That is, in the embodiment, the smart glasses (100) can display a field of view image (IMG) captured by a first camera as shown in FIG. 8 on a first display which is the inner side of the protection shield (310).

[0203] Also, referring again to FIG. 9, in the embodiment, the smart glasses (100) can determine the second camera (161-2) as the camera to use if the shield angle (SA) between the horizon (L) and the first point (P1) is less than or equal to a preset minimum angle (e.g., 20 degrees).

[0204] At this time, the minimum angle may have a negative value if the protection shield (310) is tilted further beyond the horizontal line of the body part (330). In the embodiment, the smart glasses (100) may determine the second camera (161-2) as the camera to use even when the shield angle (SA) is a negative value.

[0205] Additionally, in the embodiment, the smart glasses (100) can operate in a second display mode when the camera used is determined to be a second camera.

[0206] At this time, the second display mode may be a mode that displays a field of view image captured by the second camera on a window (320), which is the second display.

[0207] That is, in the embodiment, the smart glasses (100) can display a field of view image (IMG) captured by a second camera, as shown in FIG. 9, on a second display which is a window (320). An example shown in FIG. 9 may be a view seen from the user's perspective when the user (1) is wearing the smart glasses (100).

[0208] Meanwhile, the first display mode uses the protection shield (310) as a display, but depending on the shield angle (SA), a part of the view image (IMG) moves outside the area of ​​the protection shield (310), so the user will not be able to see the view image (IMG) displayed on the protection shield (310) accurately.

[0209] Accordingly, in the embodiment, the smart glasses (100) can adjust the angle of the projector (321) that projects a field of view image (IMG) onto the protection shield (310).

[0210] In the embodiment, if the shield angle (SA) is greater than or equal to the preset maximum angle, the smart glasses (100) are considered to have secured a protection shield (310) area sufficient to project the entire view image (IMG), and can transmit the view image (IMG) forward without adjusting the angle of the projector (321).

[0211] In addition, in the embodiment, the smart glasses (100) switch to a second display mode when the shield angle (SA) is less than or equal to a preset minimum angle, so there is no need to adjust the angle of the projector (321) likewise.

[0212] FIG. 10 is an example showing a field of view image displayed on a first display according to a projection angle according to an embodiment of the present invention.

[0213] However, referring to FIG. 10, in the embodiment, the smart glasses (100) can be divided into a shield-transmitting area (SC1) that is projected onto the protection shield (310) area and / or a shield-non-transmitting area (SC2) that is not projected onto the protection shield (310) area, when the shield angle (SA) is between a preset minimum angle and a maximum angle.

[0214] In this embodiment, the smart glass (100) can adjust the projection angle of the projector (321) to include the entire shield non-transmitting area (SC2) as a shield transmitting area (SC1). Specifically, the projection angle can be rotated by the amount of the shield non-transmitting area (SC2).

[0215] To this end, in the embodiment, the smart glass (100) can pre-set the projection angle of the projector (321) according to the ratio of the shield non-transmitting area (SC2).

[0216] That is, in the embodiment, the smart glass (100) can rotate the projector (321) by the projection angle of the projector (321) set according to the ratio of the shield non-transmitting area (SC2).

[0217] For example, if the ratio of the shield non-transmittance area (SC2) in the field of view image (IMG) is 30%, the smart glasses (100) can rotate the projection angle of the projector (321) by 10 degrees, which is a preset ratio of 30%.

[0218] Thus, all shield non-transparent areas (SC2) are converted into shield transparent areas (SC1), and the user according to the embodiment can view all content displayed on the first display.

[0219] Accordingly, the smart glasses (100) according to the embodiment of the present invention allow the user to view the display being played without necessarily covering the entire face with the protection shield (310), thereby improving the autonomy to freely adjust the angle of the protection shield (310) and relieving the user's discomfort from using the smart glasses (100), thus improving the overall satisfaction with the device.

[0220] Additionally, in the embodiment, the smart glasses (100) can capture a field of view image with at least one of the first camera and / or the second camera. (S103)

[0221] In the embodiment, the field of view image may be an image of the same scene as the scene viewed by the user's field of view, captured through a first camera and / or a second camera positioned in the direction of the user's field of view.

[0222] Additionally, in the embodiment, the smart glasses (100) may provide an object management interface including a field of view image.

[0223] In an embodiment, the object management interface may be a screen that has a view image as a background and displays a predetermined menu and / or description overlaid on the view image. Further details regarding this will be described later.

[0224] In this object management interface, analysis information obtained by analyzing a specific object detected in a field of view image (IMG) can be displayed as augmented reality content.

[0225] To this end, in the embodiment, the smart glasses (100) can recognize at least one object included in the captured field of view image (IMG). (S105)

[0226] The type of the object mentioned above may be any object containing information that can be analyzed; however, for the convenience of explanation in the following examples, the object is considered to be part and / or all of a plant containing a fruit.

[0227] At this time, in the embodiment, the smart glasses (100) can recognize at least one object included in the field of view image based on an object recognition program (e.g., YOLO).

[0228] Additionally, in the embodiment, the smart glasses (100) can transmit at least one recognized object to a service provider server (200).

[0229] Here, in the embodiment, the service providing server (200) can obtain analysis information about the recognized object based on an object information analysis deep learning neural network.

[0230] In the embodiment, the object information analysis deep learning neural network may use at least one object included in the field of view image as input data and analysis information about the object as output data.

[0231] Accordingly, in the embodiment, the service providing server (200) can obtain at least one piece of analysis information output from the object information analysis deep learning neural network. (S107)

[0232] Here, the object analysis information according to the embodiment (hereinafter, analysis information) may include information regarding the variety, disease name, pesticide, etc. of the fruit, assuming that the recognized object is a fruit.

[0233] In addition, the service providing server (200) may be an edge computing platform connected to a central server or a user's smartphone.

[0234] In an embodiment, the service providing server (200) can transmit this analysis information to the smart glasses (100).

[0235] Accordingly, in the embodiment, the smart glasses (100) can receive analysis information from the service providing server (200).

[0236] Meanwhile, in the embodiment, the smart glasses (100) may independently perform the steps performed by the service providing server (200). That is, in the embodiment, the smart glasses (100) may recognize at least one object included in the field of view image based on an object recognition program, and obtain analysis information about the recognized object based on an object information analysis deep learning neural network.

[0237] Additionally, in the embodiment, the smart glasses (100) can display augmented reality content including acquired analysis information on one of the first display and / or the second display. (S109)

[0238] In this case, in the embodiment, the smart glasses (100) may display additional analysis information based on an object management interface. Also, in the embodiment, the augmented reality content may be a concept that includes all content displayed on the object management interface.

[0239] FIG. 11 is an example of an object management interface displaying analysis information according to an embodiment of the present invention.

[0240] Referring to FIG. 11, in an embodiment, the object management interface (500) may provide an object management interface (500) in which analysis information of a predetermined object detected in a field of view image (IMG) is displayed as augmented reality content.

[0241] Additionally, in the embodiment, the object management interface (500) may display certain information overlaid on the field of view image (IMG) to provide object management services for the detected object.

[0242] Additionally, in an embodiment, the user may share the screen with an expert in real time by transmitting and receiving with a predetermined terminal and / or server based on the object management interface (500). In another embodiment, images and / or videos captured based on a camera may be transmitted to an expert terminal.

[0243] At this time, if there are multiple expert terminals, the smart glasses (100) can upload images and / or videos captured based on the camera to a cloud system used by multiple expert terminals. Additionally, it can obtain answer data generated from the cloud system. Accordingly, it has the effect of easily asking multiple experts about the situation at the site at once and immediately obtaining answers to solve problems that occur at the site.

[0244] This object management interface (500) may include a view image (IMG), a basic information area (510), a menu area (520), at least one object box (OB), analysis information (530), first and second buttons (BT1, BT2) and / or an auxiliary explanation area (540).

[0245] The basic information area (510) may be an area that displays information about the user's surrounding environment based on the sensor system (160). For example, the basic information area (510) may display the date, time, weather, temperature, humidity, etc.

[0246] The menu area (520) may be an area that displays information about certain additional functions provided by the smart glasses (100). For example, the menu area (520) may display applications such as phone, internet, music, and settings.

[0247] In the embodiment, the object box (OB) and analysis information (530) may be displayed as augmented reality content on the first object. At this time, the first object may be manually selected by the user with a predetermined input or may be automatically selected.

[0248] To this end, in the embodiment, the smart glasses (100) can place a first object as a reference point for the view.

[0249] In addition, in the embodiment, the smart glasses (100) can determine the position and orientation of the smart glasses (100) based on the first object and place and display the object box (OB) and analysis information (530) as augmented reality content on the recognized first object.

[0250] An object box (OB) may be content that displays at least one object recognized within a view image (IMG) in the form of a box. In this case, among the multiple object boxes (OB), a first object box selected automatically and / or manually may be displayed in a color different from the other boxes. For example, among the multiple object boxes (OB), the object box (OB) may be displayed in red only for the first object presumed to be in an abnormal state.

[0251] The analysis information (530) may include first and second buttons (BT1, BT2). At this time, the first and second buttons (BT1, BT2) can be entered by the user when they want to view the simplified information displayed in the analysis information (530) in more detail. When entered, a predetermined website may be connected or a pre-set control command may be executed.

[0252] For example, pressing the first button connects to a website containing 'pesticide usage instructions,' and pressing the second button can execute a control command via 'expert call.'

[0253] The auxiliary explanation area (540) may be activated and displayed when a specific auxiliary explanation is required. For example, a detailed explanation of the analysis information (530), a real-time call screen with an expert, a music selection screen and / or an internet search screen may be displayed.

[0254] At this time, the expert terminal used by the expert can acquire a scene viewed in real time by the smart glasses (100), and the expert can add certain educational content (e.g., a check mark, etc.) to the scene acquired based on the expert terminal. In this way, the expert terminal can perform remote consultation by generating and sharing answer data with added educational content. Here, there may be at least one expert terminal. Accordingly, at least one answer data generated by each expert terminal can be immediately displayed on the display of the smart glasses (100). That is, at least one expert can immediately transmit work instructions regarding the site from a distance.

[0255] In addition, in the embodiment, the smart glasses (100) can perform a predetermined control command based on user input.

[0256] For example, the above control command may include selecting an application in the menu area (520), selecting a first object, selecting first and second buttons (BT1, BT2) included in the analysis information (530), and inputting call reception and termination.

[0257] To this end, in the embodiment, the smart glasses (100) may further include a predetermined control button on one side.

[0258] In addition, in the embodiment, the smart glasses (100) can pre-set a predetermined control command for each angle of the protection shield (310).

[0259] For example, 'Yes' is pre-set for the act of rotating the protection shield (310) upward and 'No' is set for the act of rotating the protection shield (310) downward, so that the user can perform various commands provided by the smart glasses (100) through coordination with a control button placed on one side.

[0260] Meanwhile, if the user cannot use their hands and therefore cannot rotate the protection shield (310), the smart glasses (100) can be controlled based on gestures and / or voice commands.

[0261] To this end, in the embodiment, the smart glasses (100) can match a predetermined action to a predetermined gesture and / or voice command. Accordingly, in the embodiment, when the smart glasses (100) detect input of a gesture and / or voice command, the smart glasses (100) can be controlled based on the matched action.

[0262] In summary, the smart glasses including a protection shield according to an embodiment of the present invention and the maintenance management method utilizing the same have the effect of significantly increasing the speed of management tasks by shortening the time required to acquire information related to the object, as the user can immediately acquire information related to the object, such as problems and current status of the object, and manage the object in real time by recognizing a predetermined object included in the user's field of vision and displaying the content of the analysis of the object by AI on at least one display.

[0263] In addition, the smart glasses including a protection shield according to an embodiment of the present invention and the maintenance and management method using the same perform the role of a protective film that blocks ultraviolet rays and are also used as a display at a predetermined angle, thereby eliminating the need to wear a sun cap separately from the smart glasses and preventing skin aging and eye diseases that may occur when used outdoors, while increasing the utility of the smart glasses.

[0264] In addition, the smart glasses including a protection shield according to an embodiment of the present invention and the maintenance and management method using the same have the effect of increasing mobility and providing comfort to the user during device usage by incorporating a PinTILT method, with high light efficiency and light weight due to the thin lens thickness.

[0265] In addition, the smart glasses including a protection shield according to an embodiment of the present invention and the maintenance management method using the same have the effect of significantly increasing the overall efficiency of an object by preventing economic, time, procedural, and environmental losses that may occur due to a lack of familiarity with the characteristics related to the object by requesting a diagnosis from an expert through real-time screen sharing.

[0266] The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. Hardware devices may be modified into one or more software modules to perform processing according to the present invention, and vice versa.

[0267] The specific embodiments described in this invention are examples and do not limit the scope of the invention in any way. For the sake of brevity of the specification, descriptions of prior electronic configurations, control systems, software, and other functional aspects of said systems may be omitted. Additionally, the connections of lines or connecting members between components shown in the drawings are illustrative of functional connections and / or physical or circuit connections, and may be replaced or additionally represented as various functional connections, physical connections, or circuit connections in actual devices. Furthermore, unless specifically stated as “essential,” “importantly,” etc., a component may not be strictly necessary for the application of the invention.

[0268] Furthermore, although the detailed description of the present invention has been explained with reference to preferred embodiments of the invention, those skilled in the art or those with ordinary knowledge in the relevant technical field will understand that various modifications and changes can be made to the invention without departing from the spirit and technical scope of the invention as set forth in the claims below. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be determined by the claims.

Claims

Claim 1 A maintenance and management method utilizing smart glasses comprising: a body portion; a protection shield connected at both ends to the body portion; a window connected at one side to the body portion; a projector fixed to the body portion and projecting toward the protection shield or the window; a camera disposed in the body portion; and at least one processor controlling the projector and the camera, wherein the at least one processor comprises: a step of calculating a shield angle between the body portion and the protection shield and a window angle between the body portion and the window; a step of activating a display mode for displaying augmented reality content on the window or the protection shield only when the window angle is a preset first angle, and stopping the operation of the smart glasses when the window angle is an angle other than the first angle; and a step of, while the display mode is activated, if the shield angle is between a preset minimum angle and a maximum angle, calculating the ratio of the shield non-transmittance area to the shield transmittance area of ​​the protection shield on a field of view image captured by the camera. A maintenance and management method utilizing smart glasses, characterized by performing the step of displaying augmented reality content on the protection shield by adjusting the physical projection angle of the projector in correspondence with a preset projection angle according to the ratio of the calculated shield non-transmittance area. Claim 2 A maintenance and management method using smart glasses according to claim 1, comprising the step of capturing a field of view image displayed on at least one of the protection shield and the window using the camera disposed in the body part. Claim 3 A maintenance and management method using smart glasses according to claim 1, comprising the step of sensing the shield angle using a position sensor included in a rotating part that connects the body part and the protection shield and provides a rotatable structure. Claim 4 A maintenance and repair management method using smart glass, characterized in that, in claim 1, the protection shield is composed of a polycarbonate film that blocks ultraviolet rays. Claim 5 A maintenance and management method using smart glasses, wherein, in claim 1, the step of displaying augmented reality content on the window is performed using a PinTILT method in which light emitted from the projector is reflected by a concave mirror and then transmitted to the user's pupil via a plurality of pin mirrors. Claim 6 A maintenance and management method using smart glasses according to claim 1, wherein the step of activating the display mode comprises: a step of recognizing at least one object included in the captured field of view image based on an object recognition program; a step of obtaining analysis information about the recognized object based on an object information analysis deep learning neural network; and a step of augmenting the field of view image with the obtained analysis information. Claim 7 delete Claim 8 delete Claim 9 delete Claim 10 delete Claim 11 A maintenance and management method using smart glasses according to claim 6, wherein the step of acquiring analysis information for the recognized object comprises the step of inputting at least one recognized object into the object information analysis deep learning neural network and the step of acquiring analysis information for each of the at least one object output from the object information analysis deep learning neural network. Claim 12 In claim 6, the step of displaying augmented reality content including the acquired analysis information comprises a first object box surrounding a first object included in the field of view image and a step of augmenting the first analysis information for the first object, a maintenance and management method using smart glasses. Claim 13 A maintenance management method using smart glasses according to claim 6, further comprising the steps of transmitting the field of view image to at least one expert terminal and performing remote consultation by displaying answer data generated from the at least one expert terminal as augmented reality content on at least one of the window or the protection shield. Claim 14 A maintenance management system utilizing smart glasses configured to perform the method of claim 1, further comprising a terminal that provides augmented reality content to the smart glasses, wherein the terminal performs edge computing with a service providing server.

Images