Method and electronic device for providing ar information using image

Abstract

An augmented reality (AR) device according to an embodiment may include a communication module, a camera, a display, a wear detection sensor, a memory, and a processor. A memory according to an embodiment may include instructions to cause a processor to receive, via a communication module, indicia configuration information related to a display form and a display method of a dial AR indicia and configured in an electronic device. A memory according to an embodiment may include instructions to cause a processor to detect, via a wear detection sensor, that an AR device is worn on a user's body and to activate a camera. A memory according to an embodiment may include instructions to cause a processor to identify a dial AR marker and a watchband pointing marker included in a camera video based on marker configuration information configured in an electronic device. A memory according to an embodiment may include instructions to cause a processor to determine at least one of a position, a gesture, and a tilt of an electronic device based on an identified dial AR marker and an identified strap pointing marker. A memory according to an embodiment may include instructions to cause a processor to output AR information corresponding to a gesture or tilt based on a location of an electronic device.

Description

Technical Field

[0001] Various embodiments relate to an electronic device and method for providing AR information by using an image. Background Art

[0002] Recently, electronic devices (hereinafter, "AR devices") that support augmented reality (AR) or mixed reality (MR) services have proliferated, and augmented reality (AR) or mixed reality (MR) services provide information by superimposing virtual images on images or backgrounds of real-world elements.

[0003] For ease of understanding of the present disclosure, the above information may be provided as related art. No determination is made as to the applicability of any of the foregoing as prior art relevant to the present disclosure. Summary of the Invention

[0004] Technical Problem

[0005] An augmented reality (AR) service may use a specific pattern of a marker to synthesize or map a virtual object or virtual information on a real environment or a real-world image. An AR device may identify a marker in a camera video based on where the marker is located to synthesize and provide AR information (e.g., a virtual object or an additional object). An AR device may detect a marker in a camera video based on where the marker is located to output AR information.

[0006] Markers that can be recognized by an AR device are printed separately and attached to positions for outputting AR information. However, the marker recognition rate of separately printed markers may change according to the surrounding environment, timing, or angle. For example, when the surrounding environment is dark, there are patterns similar to the marker, or the marker is damaged, it may be difficult to detect the marker. Therefore, when it is difficult to detect the marker or the recognition rate of the marker continuously decreases, there may be an inconvenience of having to print new markers again to replace existing markers.

[0007] Technical Solution

[0008] An augmented reality (AR) device according to an embodiment may include: a communication module including a communication circuit, a camera, a display, a wearing detection sensor, a memory, and a processor. The memory according to an embodiment may include instructions that, when executed by the processor, cause the AR device to receive, via the communication module, marker configuration information related to a display form and a display method of a dial AR marker and configured in an electronic device. The memory according to an embodiment may include instructions that cause the processor to detect, via the wearing detection sensor, that the AR device is worn on a user's body and activate the camera. The memory according to an embodiment may include instructions that cause the processor to identify a dial AR marker and a strap pointing marker included in a camera video based on the marker configuration information configured in the electronic device. The memory according to an embodiment may include instructions that cause the processor to determine at least one of a position, a posture, and a tilt of the electronic device based on the identified dial AR marker and the identified strap pointing marker. The memory according to an embodiment may include instructions that cause the processor to output AR information corresponding to the posture or the tilt based on the position of the electronic device.

[0009] A method for an AR device according to an embodiment to provide AR information by using a dial image may include: receiving, from an electronic device, marker configuration information related to a display form and a display method of a dial AR marker and configured in the electronic device (e.g., a wrist-worn electronic device). The method according to an embodiment may include detecting that the AR device is worn on a user's body and activating the camera. The method according to an embodiment may include identifying a dial AR marker and a strap pointing marker included in a camera video based on the marker configuration information configured in the electronic device. The method according to an embodiment may include determining at least one of a position, a posture, and a tilt of the electronic device based on the identified dial AR marker and the identified strap pointing marker. The method according to an embodiment may include outputting AR information corresponding to the posture or the tilt based on the position of the electronic device.

[0010] Technical effects

[0011] According to various embodiments, an electronic device and a method may increase the usability of the electronic device by adaptively displaying AR information according to the movement or rotation of the electronic device (e.g., a wrist-worn electronic device) by using a dial image on the electronic device (e.g., a wrist-worn electronic device) without a separately printed marker.

[0012] According to various embodiments, an electronic device and a method may provide an augmented reality service in cooperation with an AR device by using a dial image in a manner invisible to a user. Brief description of the drawings

[0013] The above and other aspects, features, and advantages of certain embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0014] Figure 1 is a block diagram showing an electronic device 101 in a network environment 100 according to various embodiments.

[0015] Figure 2 is a block diagram showing an AR (AR: Augmented Reality) device according to an embodiment of the present disclosure.

[0016] Figure 3 is a diagram showing an AR (AR: Augmented Reality) device according to an embodiment of the present disclosure.

[0017] Figure 4 is a simplified block diagram of an electronic device 101 and an AR device 201 according to an embodiment.

[0018] Figure 5 is an example of a dial AR marker image displayed on an electronic device according to an embodiment.

[0019] Figure 6 shows an example of a strap configuration of an electronic device according to an embodiment.

[0020] Figure 7 shows a method of providing AR information by using a dial image in an electronic device and an AR device according to an embodiment.

[0021] Figure 8 shows a screen describing a first display method of a dial AR marker image in an electronic device according to an embodiment.

[0022] Figure 9 shows an example of displaying AR information in an AR device 201 by using a dial image of an electronic device according to an embodiment.

[0023] Figure 10 shows an example of providing AR information in an AR device according to an embodiment.

[0024] Figure 11 shows a method of displaying a dial marker in an electronic device according to an embodiment.

[0025] Figure 12 shows a method of providing AR information by using a dial marker in an AR device 201 according to an embodiment. Detailed Description

[0026] An electronic device according to various embodiments may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. According to an embodiment of the present disclosure, the electronic device is not limited to the above-described electronic devices.

[0027] Figure 1 is a block diagram showing an electronic device 101 in a network environment 100 according to various embodiments. Refer to Figure 1 .

[0028] The electronic device 101 in the network environment 100 may communicate with an electronic device 102 via a first network 198 (e.g., a short-range wireless communication network), or communicate with at least one of an electronic device 104 or a server 108 via a second network 199 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 via the server 108. According to an embodiment, the electronic device 101 may include a processor 120, a memory 130, an input module 150, a sound output module 155, a display module 160, an audio module 170, a sensor module 176, an interface 177, a connection end 178, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a subscriber identification module (SIM) 196, or an antenna module 197. In some embodiments, at least one of the above components (e.g., the connection end 178) may be omitted from the electronic device 101, or one or more other components may be added to the electronic device 101. In some embodiments, some of the above components (e.g., the sensor module 176, the camera module 180, or the antenna module 197) may be implemented as a single integrated component (e.g., the display module 160).

[0029] The processor 120 may run software (e.g., program 140) to control at least one other component (e.g., a hardware component or a software component) connected to the processor 120 of the electronic device 101, and may perform various data processing or computations. According to one embodiment, as at least part of the data processing or computation, the processor 120 may store commands or data received from another component (e.g., the sensor module 176 or the communication module 190) in the volatile memory 132, process the commands or data stored in the volatile memory 132, and store the resulting data in the non-volatile memory 134. According to an embodiment, the processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or an application processor (AP)) or an auxiliary processor 123 (e.g., a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that is operationally independent of or combined with the main processor 121. For example, when the electronic device 101 includes the main processor 121 and the auxiliary processor 123, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or may be adapted to be dedicated to a specific function. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of the main processor 121.

[0030] When the main processor 121 is in an inactive (e.g., sleep) state, the auxiliary processor 123 (instead of the main processor 121) may control at least some of the functions or states related to at least one of the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190), or when the main processor 121 is in an active state (e.g., running an application), the auxiliary processor 123 may control at least some of the functions or states related to at least one of the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190) together with the main processor 121. According to an embodiment, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., the camera module 180 or the communication module 190) that is functionally related to the auxiliary processor 123. According to an embodiment, the auxiliary processor 123 (e.g., a neural processing unit) may include a hardware structure dedicated to artificial intelligence model processing. The artificial intelligence model may be generated through machine learning. For example, such learning may be performed by the electronic device 101 where the artificial intelligence is executed or via a separate server (e.g., the server 108). The learning algorithm may include, but is not limited to, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The artificial intelligence model may include multiple artificial neural network layers. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q-network, or a combination of two or more of them, but is not limited thereto. Additionally or alternatively, the artificial intelligence model may include a software structure in addition to the hardware structure.

[0031] The memory 130 may store various data used by at least one component of the electronic device 101 (e.g., the processor 120 or the sensor module 176). The various data may include, for example, software (e.g., the program 140) and input data or output data for commands related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134.

[0032] The program 140 may be stored as software in the memory 130, and the program 140 may include, for example, an operating system (OS) 142, middleware 144, or an application 146.

[0033] The input module 150 may receive commands or data to be used by other components of the electronic device 101 (e.g., the processor 120) from the outside of the electronic device 101 (e.g., a user). The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus).

[0034] The sound output module 155 may output a sound signal to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. The speaker may be used for general purposes such as playing multimedia or playing a record. The receiver may be used for receiving an incoming call. According to an embodiment, the receiver may be implemented separately from the speaker or as a part of the speaker.

[0035] The display module 160 may visually provide information to the outside of the electronic device 101 (e.g., to a user). The display device 160 may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling a corresponding one of the display, the holographic device, and the projector. According to an embodiment, the display module 160 may include a touch sensor adapted to detect a touch or a pressure sensor adapted to measure an intensity of a force caused by the touch.

[0036] The audio module 170 may convert a sound into an electrical signal and vice versa. According to an embodiment, the audio module 170 may obtain a sound via the input module 150, or output a sound via the sound output module 155 or a headset of an external electronic device (e.g., electronic device 102) directly (e.g., wired) or wirelessly connected to the electronic device 101.

[0037] The sensor module 176 may detect an operating state of the electronic device 101 (e.g., power or temperature) or an environmental state outside the electronic device 101 (e.g., a state of a user), and then generate an electrical signal or a data value corresponding to the detected state. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0038] The interface 177 may support one or more specific protocols used to directly (e.g., wired) or wirelessly connect the electronic device 101 to an external electronic device (e.g., electronic device 102). According to an embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0039] The connection terminal 178 may include a connector through which the electronic device 101 may be physically connected to an external electronic device (e.g., electronic device 102). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headset connector).

[0040] The haptic module 179 may convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that can be recognized by the user via his sense of touch or kinesthesia. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0041] The camera module 180 may capture a still image or a moving image. According to an embodiment, the camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0042] The power management module 188 may manage the power supply to the electronic device 101. According to an embodiment, the power management module 188 may be implemented as at least part of, for example, a power management integrated circuit (PMIC).

[0043] The battery 189 may supply power to at least one component of the electronic device 101. According to an embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0044] The communication module 190 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (e.g., the electronic device 102, the electronic device 104, or the server 108), and performing communication via the established communication channel. The communication module 190 may include one or more communication processors capable of operating independently of the processor 120 (e.g., an application processor (AP)), and support direct (e.g., wired) communication or wireless communication. According to an embodiment, the communication module 190 may include a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). Each of these communication modules may communicate with an external electronic device via a first network 198 (e.g., a short-range communication network, such as Bluetooth, Wi-Fi Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network, such as a traditional cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or wide area network (WAN))). These various types of communication modules may be implemented as a single component (e.g., a single chip), or these various types of communication modules may be implemented as multiple separate components (e.g., multiple chips). The wireless communication module 192 may identify and authenticate the electronic device 101 in a communication network (such as the first network 198 or the second network 199) using user information (e.g., an international mobile subscriber identity (IMSI)) stored in the user identification module 196.

[0045] The wireless communication module 192 may support 5G networks after 4G networks and next-generation communication technologies (e.g., New Radio (NR) access technology). The NR access technology may support enhanced mobile broadband (eMBB), massive machine type communication (mMTC), or ultra-reliable low-latency communication (URLLC). The wireless communication module 192 may support high frequency bands (e.g., millimeter wave bands) to achieve, for example, high data transfer rates. The wireless communication module 192 may support various technologies for ensuring performance in high frequency bands, such as, for example, beamforming, massive multiple-input multiple-output (massive MIMO), full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. The wireless communication module 192 may support various requirements specified in the electronic device 101, an external electronic device (e.g., the electronic device 104), or a network system (e.g., the second network 199). According to an embodiment, the wireless communication module 192 may support a peak data rate for implementing eMBB (e.g., 20 Gbps or greater), a loss coverage for implementing mMTC (e.g., 164 dB or less), or a U-plane latency for implementing URLLC (e.g., 0.5 ms or less for each of downlink (DL) and uplink (UL), or a round trip of 1 ms or less).

[0046] The antenna module 197 may transmit a signal or power to the outside of the electronic device 101 (e.g., an external electronic device) or receive a signal or power from the outside of the electronic device 101 (e.g., an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a radiating element formed of a conductive material or a conductive pattern formed in a substrate (e.g., a printed circuit board (PCB)) or formed on the substrate. According to an embodiment, the antenna module 197 may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme to be used in a communication network (such as the first network 198 or the second network 199) may be selected from the plurality of antennas by, for example, the communication module 190 (e.g., the wireless communication module 192). Subsequently, a signal or power may be transmitted or received between the communication module 190 and an external electronic device via the at least one selected antenna. According to an embodiment, additional components (e.g., a radio frequency integrated circuit (RFIC)) other than the radiating element may be additionally formed as part of the antenna module 197.

[0047] According to various embodiments, the antenna module 197 may form a millimeter-wave antenna module. According to an embodiment, the millimeter-wave antenna module may include a printed circuit board, a radio frequency integrated circuit (RFIC), and a plurality of antennas (e.g., an array antenna), wherein the RFIC is disposed on a first surface (e.g., a bottom surface) of the printed circuit board, or adjacent to the first surface and capable of supporting a specified high-frequency band (e.g., a millimeter-wave band), and the plurality of antennas are disposed on a second surface (e.g., a top surface or a side surface) of the printed circuit board, or adjacent to the second surface and capable of transmitting or receiving signals of the specified high-frequency band.

[0048] At least some of the above components may be interconnected via an inter-peripheral communication scheme (e.g., a bus, a general-purpose input / output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)) and communicatively transmit signals (e.g., commands or data) therebetween.

[0049] According to an embodiment, commands or data may be sent or received between the electronic device 101 and the external electronic device 104 via the server 108 connected to the second network 199. Each of the electronic devices 102 or 104 may be a device of the same type as the electronic device 101 or a device of a different type from the electronic device 101. According to an embodiment, all or some of the operations running on the electronic device 101 may be run on one or more of the external electronic device 102, the external electronic device 104, or the server 108. For example, if the electronic device 101 is supposed to automatically execute a function or service or is supposed to execute a function or service in response to a request from a user or another device, the electronic device 101 may request one or more of the external electronic devices to execute at least part of the function or service instead of running the function or service, or in addition to running the function or service, the electronic device 101 may also request one or more of the external electronic devices to execute at least part of the function or service. The one or more external electronic devices that receive the request may execute the requested at least part of the function or service, or execute an additional function or additional service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may provide the result as at least part of a reply to the request with or without further processing of the result. For this purpose, for example, cloud computing technology, distributed computing technology, mobile edge computing (MEC) technology, or client-server computing technology may be used. The electronic device 101 may use, for example, distributed computing or mobile edge computing to provide ultra-low latency services. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. The server 108 may be an intelligent server using machine learning and / or neural networks. According to an embodiment, the external electronic device 104 or the server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology or IoT-related technology.

[0050] Figure 2 is a block diagram showing an AR (AR: Augmented Reality) device according to an embodiment of the present disclosure.

[0051] Referring to Figure 2 , according to an embodiment, the augmented reality (AR) device 201 that provides an image related to an AR service to a user may be configured in the form of at least one of glasses, goggles, a helmet, or a hat, but is not limited thereto. In an example, the AR device 201 may be a head-mounted device (HMD), a head-mounted display (HMD), or AR glasses.

[0052] In an embodiment, the AR device 201 may be combined with an electronic device 101 (or a host device or a controller device) (e.g., Figure 1 the electronic device 101) to provide an AR service.

[0053] In an embodiment, the AR device 201 may provide an AR service that outputs at least one virtual object such that the at least one virtual object appears to overlap with a region determined by a field of view (FoV) of a user. For example, the region determined by the field of view of the user is a region determined to be perceivable by the user via the AR device 201 and may include all or at least a part of a display module 240 of the AR device 201.

[0054] In an embodiment, the AR device 201 may be at least partially controlled by the electronic device 101 (e.g., Figure 1 the electronic device 101), and at least one function may be executed under the control of the electronic device 101 (e.g., Figure 1 the electronic device 101). The AR device 201 may be at least partially controlled by the electronic device 101 (e.g., Figure 1 the electronic device 101), and at least one function may be executed under the control of the electronic device 101 (e.g., Figure 1 the electronic device 101).

[0055] In an embodiment, the AR device 201 may include a communication module (e.g., including a communication circuit) 210, a processor (e.g., including a processing circuit) 220, a memory 230, a display module (e.g., including a display 240), an audio module (e.g., including an audio circuit) 250, a sensor module (e.g., including at least one sensor) 260, and a camera module (e.g., including at least one camera) 270. Although not shown in the figure, the AR device 201 may further include a power management module and a battery.

[0056] In an embodiment, the communication module 210 (e.g., a wireless communication circuit) may include various communication circuits and perform communication with the electronic device 101 (e.g., Figure 1 the electronic device 101) via a wireless communication network (e.g., Figure 1 the first network 198 {e.g., a short-range wireless communication network}) or perform communication with a server device via a long-range wireless network (e.g., Figure 1 the second network 199). In an example, the AR device 201 or 101 may perform wireless communication with the electronic device 101 (e.g., Figure 1 the electronic device 101) to exchange commands and / or data with each other.

[0057] In an embodiment, the communication module 210 may support 5G networks and next-generation communication technologies after 4G networks, for example, New Radio (NR) access technology. The NR access technology may support high-speed transmission of high-capacity data (Enhanced Mobile Broadband {eMBB}), minimization of terminal power, and connection of multiple terminals (Massive Machine Type Communication {mMTC}), or Ultra-Reliable and Low-Latency Communication (URLLC). The communication module 210 may support, for example, high frequency bands (e.g., millimeter wave bands) to achieve high data rates. The communication module 210 may support various technologies for ensuring performance in high frequency bands, such as beamforming, massive Multiple-Input and Multiple-Output (MIMO), full-dimensional MIMO, array antennas, analog beamforming, or massive antennas.

[0058] In an embodiment, the display module 240 may display at least one virtual object on at least a part of the display panel such that the virtual object is added to an image related to the real space acquired by a user wearing the AR device 201 via the camera module 270.

[0059] According to various embodiments, the display module 240 may include a first display module 241 corresponding to the left eye of the user's binocular vision and / or a second display module 243 corresponding to the right eye of the user.

[0060] In an embodiment, the display module 240 may include a transparent or semi-transparent display.

[0061] In an embodiment, the display module 240 may include a lens. The lens may include a lens having a transparent waveguide. The lens may transmit light output from the display panel to the user's eyes. In an example, the light emitted from the display panel may pass through the lens and be transmitted to the user via a waveguide formed in the lens. The waveguide may include at least one of at least one diffractive element (e.g., diffractive optical element {DOE} or holographic optical element {HOE}) or a reflective element (e.g., a mirror). In another example, the waveguide may guide the display light emitted from the light source unit to the user's eyes by using at least one diffractive element or reflective element. The user may perceive the real space (or real environment) on the rear surface of the display via the display module 240.

[0062] In an embodiment, the audio module 250 may include various audio circuits and convert sound into an electrical signal or convert an electrical signal into sound based on the control of the processor 220. For example, the audio module 250 may include a speaker and / or a microphone.

[0063] In an embodiment, the sensor module 260 may include at least one sensor and detect the movement of the AR device 201. The sensor module 260 may detect a physical quantity related to the movement of the AR device 201, such as the speed, acceleration, angular velocity, angular acceleration, or geographical location of the AR device 201.

[0064] In an embodiment, the sensor module 260 may include various sensors. In an example, the sensor module 260 may include a proximity sensor 261, an illuminance sensor 262, and / or a gyro sensor 263, but is not limited thereto. The proximity sensor 261 may detect an object adjacent to the AR device 201. The illuminance sensor 262 may measure the brightness level around the AR device 201. In another embodiment, the processor 220 may use the illuminance sensor 262 to identify the brightness level around the AR device 201 and change the setting information related to the brightness of the display module 240 based on the brightness level. The gyro sensor 263 may detect, for example, the state (or posture or orientation) and position of the AR device 201. The gyro sensor 263 may detect the movement of the AR device 201 or the user wearing the AR device 201.

[0065] In an embodiment, the camera module 270 may include at least one camera and capture still images and moving images. According to another embodiment, the camera module 270 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0066] The camera module 270 may include at least one of a gesture camera 271, a gaze tracking camera 273, a distance measurement camera 275 (depth camera), and / or a red, green, blue (RGB) camera 277. According to one embodiment, the gesture camera 271 may detect the movement of the user. For example, at least one gesture camera 271 may be disposed in the AR device 201 and may detect the hand movement of the user within a predetermined distance. The gesture camera 271 may include a simultaneous localization and mapping (SLAM) camera for identifying information (e.g., position and / or orientation) related to the surrounding space of the AR device 201. The gaze tracking camera 273 may track the movement of the left and right eyes of the user. According to another embodiment, the processor 220 may use the gaze tracking camera 273 to confirm the gaze direction of the left eye and the gaze direction of the right eye. The distance measurement camera 275 may measure the distance to an object located in front of the AR device 201. According to another embodiment, the distance measurement camera 275 may include a time-of-flight (TOF) camera and / or a depth camera. The distance measurement camera 275 may capture the front direction of the AR device 201, and the gaze tracking camera 273 may capture the direction opposite to the capture direction of the distance measurement camera 275. The red, green, blue (RGB) camera 277 may detect color-related information of an object and distance information to the object.

[0067] In an embodiment, the gesture camera 271, the gaze tracking camera 273, the distance measurement camera 275, and / or the RGB camera 277 included in the camera module 270 may be included in the AR device 201 respectively, or some of them may be implemented as an integrated camera. The distance measurement camera 275 and the RGB camera 277 may be implemented as an integrated camera, for example.

[0068] In an embodiment, the processor 220 may include various processing circuits and may run, for example, a program (e.g., Figure 1 program 140) stored in the memory 230 to control at least one other component related to the functions of the AR device 201 (e.g., the communication module 210, the display module 240, the audio module 250, the sensor module 260, or the camera module 270), and may perform data processing and operations required for tasks related to the AR service (e.g., AR tasks). In another example, the processor 220 may include a computing processing unit.

[0069] In an embodiment, the processor 220 may capture a real-space-related image corresponding to the field of view of a user wearing the AR device 201 via the camera module 270 to obtain image information. The processor 220 may identify information corresponding to a region determined by the field of view (FoV) of the user in the real-space-related image acquired via the camera module 270 of the AR device 201. The processor 220 may generate a virtual object based on the virtual information based on the image information. In another embodiment, the processor 220 may display a virtual object related to the AR service together with the image information via the display module 240.

[0070] In an embodiment, the processor 220 may measure physical quantities related to the movement of the AR device 201 (e.g., the geographical location, speed, acceleration, angular velocity, and angular acceleration of the AR device 201) via the sensor module 260, and may use the measured physical quantities or a combination thereof to obtain movement information of the AR device 201.

[0071] In an embodiment, the processor 220 may analyze the movement information and the image information of the AR device 201 in real time to control the execution of AR tasks, such as a head tracking task, a hand tracking task, and an eye tracking task.

[0072] Figure 3 is a diagram showing an AR (AR: Augmented Reality) device according to an embodiment of the present disclosure.

[0073] Referring to Figure 3 According to various embodiments, an augmented reality (AR) device (e.g., Figure 2The AR device 201) can be worn on the user's head to provide the user with images related to augmented reality (AR) services and / or virtual reality (VR) services. The AR device 201 may also include Figure 2 at least some of the elements and / or features of the AR device in Figure 2 and may have overlapping elements substantially the same as those in

[0074] In an embodiment, the AR device 201 may include a first display 305 (e.g., Figure 2 the first display module 241 of Figure 2 ), and / or a second display 310 (e.g.,

[0075] the second display module 243 of

[0076] ), a screen display unit 315, an input optical member 320, a first transparent member 325a, a second transparent member 325b, illumination units 330a and 330b, a first printed circuit board 335a, a second printed circuit board 335b, a first hinge 340a, a second hinge 340b, a first camera 345, a plurality of microphones (e.g., a first microphone 350a, a second microphone 350b, and a third microphone 350c), a plurality of speakers (e.g., a first speaker 355a and a second speaker 355b), a battery 360, a second camera 365a, and a third camera 365b.

[0075] According to an embodiment, the display (e.g., the first display module 305 and the second display module 310) may include, for example, a liquid crystal display (LCD), a digital micromirror device (DMD), a light emitting diode (LED), a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro-LED). Although not shown, when the display is one of a liquid crystal display, a digital mirror display, or a liquid crystal on silicon display device, the AR device 201 may include a light source for emitting light to the screen output area of the display. In an embodiment, when the display is capable of self-generating light, e.g., when the display is made of an organic light emitting diode or a micro-LED, the AR device 201 may provide a high-quality virtual video to the user without including a separate light source. In an embodiment, when the display is implemented as an organic light emitting diode or a micro-LED, a light source is unnecessary, and thus the AR device 201 may be lightweight.

[0076] Displays according to various embodiments (e.g., the first display 305 and the second display 310) may include at least one micro light-emitting diode (micro-LED). For example, the micro-LEDs may emit red (R), green (G), and blue (B) by self-emission and may be small in size (e.g., 100 μm or less) such that a single chip can implement one pixel (e.g., one of R, G, and B). Thus, when the display includes micro-LEDs, the display can provide high resolution without a backlight unit (BLU).

[0077] The present disclosure is not limited thereto, and one pixel may include R, G, and B, and one chip may implement multiple pixels including R, G, and B.

[0078] In an embodiment, a display (e.g., the first display 305 and the second display 310) may include a display area including pixels for displaying a virtual video and light-receiving pixels (e.g., photoelectric sensor pixels) disposed between the pixels to receive light reflected from the eyes, convert the light into electrical energy, and output the light.

[0079] In an embodiment, the AR device 201 (e.g., Figure 2 the processor 220 therein) may detect the user's line-of-sight direction (e.g., pupil movement) via the light-receiving pixels. For example, the AR device 201 may detect and track the line-of-sight direction of the user's left eye and the line-of-sight direction of the user's right eye via one or more light-receiving pixels constituting the first display 305 and one or more light-receiving pixels constituting the second display 310. The AR device 201 may determine the position of the center of the virtual video based on the line-of-sight directions of the user's right and left eyes detected via one or more light-receiving pixels (e.g., the pupils of the user's right eye and the directions in which the pupils of the user's right and left eyes are looking).

[0080] In an embodiment, light emitted from a display (e.g., the first display 305 and the second display 310) may pass through a lens (not shown) and a waveguide to reach a screen display unit 315 formed in a first transparent member 325a disposed to face the user's right eye and a screen display unit 315 formed in a second transparent member 325b disposed to face the user's left eye. For example, light emitted from a display (e.g., the first display 305 and the second display 310) may pass through the waveguide, be reflected by a grating region formed on the input optical member 320 and the screen display unit 315, and reach the user's eyes. The first transparent member 325a and / or the second transparent member 325b may be formed of a glass plate, a plastic plate, or a polymer and may be made transparent or translucent.

[0081] In an embodiment, a lens (not shown) may be disposed on each of the front surfaces of the displays (e.g., the first display module 305 and the second display module 310). The lens may include a concave lens and / or a convex lens. For example, the lens may include a projection lens or a collimating lens.

[0082] In an embodiment, the screen display unit 315 or the transparent member (e.g., the first transparent member 325a and / or the second transparent member 325b) may include a lens having a transparent waveguide and a reflective lens.

[0083] In an embodiment, the waveguide may be made of glass, plastic, or polymer and may include a nano-pattern formed on one surface inside or outside thereof. The nano-pattern may include a polygon or a curved grating structure. According to an embodiment, light incident on one surface of the transparent member (e.g., the first transparent member 325a and / or the second transparent member 325b) may be transmitted or reflected by the nano-pattern inside the waveguide to be transmitted to the user. According to an embodiment, the waveguide may include at least one of at least one diffractive element (e.g., a diffractive optical element (DOE) or a holographic optical element (HOE)) or a reflective element (e.g., a mirror). According to an embodiment, the waveguide may use at least one diffractive element or a reflective element to guide the light emitted from the displays (e.g., the first display module 305 and the second display module 310) (e.g., a light source) to the user's eyes.

[0084] In an embodiment, the diffractive element may include an input optical member 320 / output optical member (not shown). The input optical member 320 may indicate an input grating region, and the output optical member (not shown) may indicate an output grating region. The input grating region may be used to diffract (or reflect) the light output from the displays (e.g., the first display module 305 and the second display module 310) (e.g., a micro LED) to transmit it to the input end of the transparent member (e.g., the first transparent member 325a and / or the second transparent member 325b) of the screen display unit 315. The output grating region may be used to diffract (or reflect) the light transmitted to the transparent member (e.g., the first transparent member 325a and / or the second transparent member 325b) of the waveguide to the exit of the user's eyes.

[0085] In an embodiment, the reflective element may include a total reflection optical element or a total reflection waveguide for total internal reflection (TIR). Total internal reflection is a way of guiding light, in which an incident angle is formed such that the light (e.g., a virtual image) input via the input grating region is reflected by one surface (e.g., a designated surface) of the waveguides 420 and 430 by nearly 100%, so that nearly 100% of the light is transmitted to the output grating region.

[0086] In an embodiment, the optical paths of light emitted from displays 305 and 310 may be optically guided into a waveguide via an input optical member 320. The light traveling in the waveguide may be guided toward a user's eyes via an output optical member. The screen display unit 315 may be determined based on the light emitted toward the eyes.

[0087] According to an embodiment, the first camera 345 may include a high-resolution camera, such as a high-resolution (HR) camera and / or a photo-video (PV) camera. For example, the first camera 345 may utilize an autofocus function and an optical image stabilizer (OIS) function to obtain high-quality images. The capture camera 380 may be implemented as a global shutter (GS) camera and a rolling shutter (RS) camera in addition to a color camera.

[0088] According to an embodiment, the second camera 365a and the third camera 365b may be used for 3DoF or 6DoF head tracking, hand detection and / or hand tracking, gesture recognition and / or recognition spatial recognition. For example, the second camera 365a and the third camera 365b may be utilized to perform 6DoF spatial recognition and a simultaneous localization and mapping (SLAM) function via depth imaging. The second camera 365a and the third camera 365b may include GS cameras to detect and track hand movements and / or head movements.

[0089] In an embodiment, at least one sensor (e.g., Figure 2 the sensor module 260 in

[0090] ), such as a gyro sensor, an acceleration sensor, a geomagnetic sensor, and / or a gesture sensor, the second camera 365a, and the third camera 365b may perform at least one of 6DoF head tracking, pose estimation and prediction, gesture and / or spatial recognition, and a slam function via depth imaging.

[0091] In an embodiment, the lighting units 330a and 330b may be used for different purposes according to the attachment positions of the lighting units. For example, the lighting units 330a and 330b may be attached to the second camera 365a and the third camera 365b, which are disposed around hinges (e.g., the first hinge 340a and the second hinge 340b) connecting the frame and the temple, or around a bridge connecting the frames to each other. When shooting with a GS camera, each of the lighting units 330a and 330b may be used as a means to supplement the ambient brightness. For example, when it is difficult to detect a subject to be photographed due to a dark environment or due to mixed and reflected light from multiple light sources, the lighting units 330a and 330b may be used.

[0092] According to an embodiment, a printed circuit board (e.g., the first printed circuit board 335a and the second printed circuit board 335b) may be provided for respective components (e.g., the processor 220 and / or the memory 230, etc.) of the AR device 201. The PCB may send electrical signals to respective components of the AR device 201.

[0093] In an embodiment, multiple microphones (e.g., the first microphone 350a, the second microphone 350b, and the third microphone 350c) may process external acoustic signals into electrical voice data. Depending on the function (or application being run) performed in the AR device 201, the processed voice data may be used in various ways.

[0094] In an embodiment, multiple speakers (e.g., the first speaker 355a and the second speaker 355b) may output audio data received from a communication module (e.g., Figure 2 the communication module 210 therein) or stored in a memory (e.g., Figure 2 the memory 230 therein).

[0095] In an embodiment, the battery 360 may include one or more batteries and may supply power to components constituting the AR device 201.

[0096] Figure 4 is a simplified block diagram of an electronic device 101 and an AR device 201 according to an embodiment, and Figure 5 is an example of a dial AR marker image displayed on an electronic device according to an embodiment.

[0097] Referring to Figure 4 , according to an embodiment, an electronic device (e.g., Figure 1 the electronic device 101 therein) and an augmented reality (AR) device (e.g., Figure 2 the AR device 201 therein) may provide an augmented reality service in combination with each other.

[0098] According to an embodiment, the electronic device 101 may generate and display an AR marker (e.g., a dial AR marker) via a display. The AR device 201 may detect the AR marker while recording a camera video and may provide AR information (e.g., a virtual object) to the position of the AR marker.

[0099] For example, the electronic device 101 may be a wrist-worn electronic device (or a watch-type electronic device or a smartwatch), and the AR device 201 may be a glasses-type electronic device or a head-mounted display (HMD) device. However, the present disclosure is not limited thereto. According to some embodiments, when the electronic device 101 is a wrist-worn electronic device, the electronic device 101 and the AR device 201 may be connected to other electronic devices (e.g., a smartphone, a mobile device, or a server device) via account linking, and the operation of the electronic device 101 may be controlled based on the control configuration (or remote configuration) of the other electronic devices.

[0100] According to an embodiment, the electronic device 101 may include a display 411 (e.g., Figure 1 the display module 160 in Figure 1 ), a communication module (e.g., including a communication circuit) 412 (e.g., Figure 1 the communication module 190 in Figure 1 ), a wearing detection sensor 413 (e.g., Figure 1 the sensor module 176 in Figure 1 ), a motion sensor 414 (e.g., Figure 4 the sensor module 176 in Figure 1 ), a memory 415 (e.g., Figure 1 the memory 130 in Figure 1 ), and a processor (e.g., including a processing circuit) 410 (e.g.,

[0101] the processor 120 in

[0102] According to an embodiment, the display 411 may display a dial image or a watch AR marker image.

[0103] According to an embodiment, the communication module 412 includes various communication circuits and may communicate with the AR device 201 via wireless communication (e.g., at least one of WiFi-P2P, Bluetooth, and Bluetooth Low Energy (BLE) communication). The communication module 412 may send the dial image information and / or the dial marker configuration information to the AR device 201 via the account linking with the AR device 201.

[0104] According to an embodiment, the wearing detection sensor 413 may detect that the electronic device 101 is worn on the user's body and may send the sensed information to the processor 410. For example, when it is recognized that the electronic device 101 is worn on the user's body and the AR device 201 is also worn on the user's body, the processor 410 may control the AR marker image to be displayed on the dial image, and when the electronic device 101 or the AR device 201 is removed (or not worn), the processor 410 may control the AR marker image not to be displayed.

[0105] According to an embodiment, the motion sensor 414 may detect the motion of the electronic device 101 and send the sensed information to the processor 410. For example, the processor 410 may determine the movement or direction of the electronic device 101 or the tilt / pose of the electronic device 101 based on the sensed information sent from the motion sensor 414.

[0106] According to an embodiment, the memory 415 may store instructions for causing the processor 410 to control the electronic device 101. The instructions may be stored as software in the memory 415 and may be run by the processor 410.

[0107] According to an embodiment, the memory 415 may store dial marker configuration information.

[0108] According to an embodiment, the processor 410 may include various processing circuits and control the electronic device 101, or may process (or execute) the operations of the elements of the electronic device 101. The processor 410 may generate a dial AR marker image to be displayed on the dial image and may control the display of the AR marker image. The processor may control the display of the AR marker image according to the AR marker display configuration method.

[0109] According to an embodiment, in the first display configuration method, the processor 410 may display the dial AR marker image individually only in 1 to n (for example, n < 5) frames while displaying the dial image (= representative image) at the frame scan rate (frame rate) of the display. When the display 411 outputs the dial image at a scan rate of 60 to 120 Hz (scan rate: number of frames displayed per second), the processor 410 may perform control to display the dial AR marker image only in 1 to n frames of the scan rate so that the AR device 201 can recognize the AR marker. For example, since the dial AR marker image is displayed only in 1 to n frames per second, the user cannot visually recognize the AR marker image and can only view the dial image configured to be representative on the display of the AR device.

[0110] For example, as Figure 5As shown in <501>, the processor 410 may generate a dial AR marker image 510 separately from the dial image configured to be representative. The dial AR marker image 510 may be configured to be displayable on the display 411 of the wrist-worn electronic device (e.g., the electronic device 101), and may include a graphic (e.g., a QR code) having a specific pattern (or unique pattern) 515, which includes marker identification information (e.g., device information of the wrist-worn electronic device). However, the present disclosure is not limited thereto. For example, when driving a display that shows a 60hz screen, the processor 410 may control the display to display the representative dial image in frames 1 to 59 and the dial AR marker image in frame 60 on a per-frame basis.

[0111] According to an embodiment, in the second display configuration method, the processor 410 may synthesize and generate an AR marker pattern 525 recognizable by the AR device 201 and a pointing marker 527 with the dial image 520 itself (e.g., including the hour hand and the second hand 521), and may display the dial image 520 synthesized with the AR marker pattern 525 as the representative dial image. For example, as Figure 5 shown in <502>, the processor 410 may generate a dial image 520 including a graphic (e.g., a QR code) having a specific pattern (or unique pattern) 525, and may configure the dial image 520 including the graphic (e.g., a QR code) having a specific pattern (or unique pattern) 525 as the representative dial image.

[0112] According to an embodiment, in the third display configuration method, as Figure 5 shown in <503>, the processor 410 may be configured such that the electronic device 101 may pre-synchronize the screen configuration information (e.g., frames, shapes and colors of the second hand and the minute hand, and background image / thumbnail image) of the dial image (e.g., 530, 531 or 532) configured to be the representative image with the AR device, and in the augmented reality service, the AR device 201 may recognize the dial image of the electronic device 101 faster.

[0113] The dial image of the electronic device 101 is intended to provide time information, and thus may have a simple normalized pattern related to time display. For example, as Figure 5As shown in <503>, when the representative faceplate image configured in the electronic device 101 is the first image 530, the normalized image 530a of the first image 530 can be designated as the AR marker recognition pattern. In another example, when the representative faceplate image currently configured in the electronic device is the second image 531, the normalized image 531a of the second image 531 can be designated as the AR marker identification pattern, or when the representative faceplate image is the third image 532, the normalized image 532a of the third image 532 can be designated as the AR marker identification pattern. The AR device 201 stores the screen configuration information of the dial image configured in the electronic device 101 and configured to represent the image, and thus can more quickly recognize the AR marker recognition pattern from the dial image when tracking the electronic device in the camera video.

[0114] According to an embodiment, the electronic device 101 can adjust the number of times the dial AR marker image is displayed according to the marker recognition situation of the AR device 201.

[0115] When receiving the wearing detection of the AR device 201 sent via wireless communication, the processor 410 can adjust the number of times the dial AR marker image is displayed. For example, when the electronic device 101 is configured to display the dial AR marker image 1 or 2 times within 60 frames, the display frequency can be adjusted to 3 to 5 times to improve the recognition rate in the AR device. When the AR device 201 is in the initial state where no AR marker is detected, faster recognition can be achieved by increasing the number of times the dial AR marker image is displayed. When receiving the AR marker initial recognition information from the AR device 201, the electronic device 101 can restore the number of times the dial AR marker image is displayed to the original value.

[0116] According to an embodiment, when the electronic device 101 is a wrist-worn electronic device, the electronic device 101 can include a strap structure for attaching the electronic device 101 to the user's wrist and detaching it from the user's wrist. The strap structure of the electronic device 101 can include, in at least a part thereof, a pattern decoration in a shape or form that the AR device 201 can recognize as a strap pointing marker.

[0117] According to an embodiment, the AR device (e.g., Figure 2 the AR device 201 in Figure 2 can include an AR display 421 (e.g., Figure 2 the display module 240 in Figure 2 ), an AR communication module (e.g., including a communication circuit) 422 (e.g., Figure 2in the sensor module 260), the AR memory 425 (e.g., Figure 2 the memory 230) in, and the AR processor (e.g., including processing circuitry) 420 (e.g., Figure 2 the processor 220) in. For example, Figure 4 the AR device 201 shown in may have the same configuration as Figure 2 and Figure 3 the AR device 201 shown in, or may further include Figure 2 and Figure 3 the configuration of the AR device 201 shown in. A detailed description of elements identical to those in Figure 2 and Figure 3 will be omitted, and operations related to augmented reality services will be described.

[0118] According to an embodiment, the AR display 421 may perform a function of providing AR information (e.g., virtual objects) while superimposing the AR information on an external real image / video. For example, the AR display 421 may be implemented in the form of optical see-through (OST) or video see-through (VST), and may display at least a part of the AR display 421 and at least one piece of AR information such that the AR information appears to be superimposed on the real video acquired via the AR camera module 423.

[0119] According to an embodiment, the AR communication module 422 may include various communication circuits and be connected to the electronic device 101 via wireless communication (e.g., at least one of WiFi-P2P, Bluetooth, and Bluetooth Low Energy (BLE) communication). The AR communication module 422 may send (or share) various types of data for supporting augmented reality services to the electronic device 101 in real time.

[0120] According to an embodiment, the AR communication module 422 may receive information about a dial image configured as a representative image in the electronic device 101 and / or dial marker configuration information (e.g., communication forwarding or synchronization based on account linking).

[0121] According to an embodiment, the AR communication module 422 may receive motion sensing information of the electronic device 101 from the electronic device 101.

[0122] According to an embodiment, the AR camera module 423 may perform a function of capturing an external reality image / video. For example, the AR camera module 423 may capture a video including the electronic device 101 and send the captured video data to the AR processor 420.

[0123] According to an embodiment, the AR camera module 423 may be configured to operate at a scanning rate equal to or higher than the display driving scanning rate of the electronic device 101. The AR camera module 423 may capture images at dozens of frames per second to identify an AR marker displayed in a specific frame of the screen of the display 411 of the electronic device.

[0124] According to an embodiment, the AR wearing detection sensor 424 may detect that the AR device 201 is worn on a user's body and may send sensing information to the AR processor 420. For example, the AR processor 420 may determine whether to activate the AR camera module 423 based on whether the AR device is worn. The AR processor 420 may send a signal (wearing state information or detachment state information) to the electronic device 101 via the AR communication module 422, the signal being based on whether the AR device is detected as being worn.

[0125] According to an embodiment, the AR memory 425 may store instructions for operating the AR processor 420. The instructions may be stored as software in the AR memory 425 and may be run by the AR processor 420. The AR memory 425 may store information configured as a dial image as a representative image in the electronic device 101 and / or dial marker configuration information.

[0126] According to an embodiment, the AR processor 420 may include various processing circuits and send commands (or control information) to the electronic device 101 (e.g., via the AR communication module 422) to control the AR device 201 or process (or execute) the operation of elements of the electronic device 101. The AR processor 420 may receive the captured video from the AR camera 423 and may perform a function for detecting an AR marker related to an augmented reality service from the video.

[0127] According to an embodiment, the AR processor 420 may analyze the camera video based on the marker configuration information sent from the electronic device 101 to track at least one of an AR marker and a band pointing marker from the camera video. For example, the AR processor 420 may capture the camera video in units of frames and may identify the AR marker and / or the band pointing marker from a dial AR marker image or a dial image by using a tracking algorithm using a specific value (e.g., color or shape).

[0128] According to an embodiment, the AR processor 420 may identify the electronic device 101 based on the marker identification information included in the AR marker and may identify the position of the electronic device 101 (e.g., coordinate information).

[0129] The AR processor 420 may identify the tilt and / or posture of the electronic device 101 via the strap pointing markers. For example, when two strap pointing markers are identified in the camera video, the AR processor 420 may determine that the electronic device is in a posture where the front surface of the electronic device faces the user's line of sight, and when one strap pointing marker is identified, the AR processor 420 may determine that the electronic device is in a posture where the side surface of the electronic device faces the user's line of sight.

[0130] According to an embodiment, when receiving the motion sensor information of the electronic device 101 sent from the electronic device 101, the AR processor 420 may supplement the marker tracking with the result of analyzing the motion sensor information of the electronic device 101, so that when the AR marker (or the electronic device 101) is outside the camera video, the position and / or orientation of the electronic device 101 can be estimated.

[0131] According to an embodiment, the AR processor 420 may perform control to configure the position (i.e., coordinate information) of the identified electronic device 101 in the camera video as the display coordinates of the AR information, and output the AR information corresponding to (or suitable for or optimized for) the posture and tilt of the identified electronic device. The AR information may include an AR object (e.g., a message icon) for controlling the functions of the electronic device 101, but the present disclosure is not limited thereto.

[0132] For example, when the front surface of the electronic device 101 faces the user's line of sight, the AR processor 420 may perform control to output the AR information (e.g., the first type of AR information) in the full version mode, indicating the detailed information associated with the electronic device 101. When the side surface of the electronic device 101 faces the user's line of sight, the AR processor 420 may perform control to output the AR information (e.g., the second type of AR information) in the simple version mode on the AR display 421 (e.g., AR glasses), indicating the schematic information.

[0133] According to an embodiment, the AR processor 420 may track whether the user's finger is located at the coordinates where the AR information is displayed or whether a specific movement is detected via the AR camera module 423 to determine whether the user controls the AR information with the finger. For example, when a movement for selecting one of the AR objects is identified, the AR processor 420 may send the control information of the selected AR object to the electronic device 101. The electronic device 101 may perform control to run the function of the selected AR object based on the control information sent from the AR device 201, or display the information related to the selected AR object on the display 411.

[0134] Figure 6 An example of the strap configuration of an electronic device according to an embodiment is shown.

[0135] Refer toFigure 6 , according to an embodiment, an electronic device 101 (e.g., a wrist-worn electronic device) may include a housing 610 for mounting components of the electronic device 101, a strap structure 620 located at both ends of the housing 610 for attaching the electronic device 101 to a user's wrist and detaching it from the user's wrist, and a display 630 for displaying visual information. For example, Figure 6 the electronic device 101 in Figure 1 may include at least some of the components and / or functions of the electronic device 101 in

[0136] According to an embodiment, the strap structure 620 may include pattern decorations 621 and 622, and the AR device 201 may recognize the pattern decorations 621 and 622 as strap pointing markers, which are located in at least a portion adjacent to both ends of the housing 610. For example, the pattern decoration 621 is a strap design element and may be designed at a first portion 640 and a second portion 641 for expressing a center line. For example, the pattern decorations 621 and 622 may include a first pattern decoration 621 and a second pattern decoration 622. The first pattern decoration 621 has a first direction and is located at one end connected to the housing 610, and the second pattern decoration 622 has a second direction and is located at the other end of the housing 610, where the first direction and the second direction are designed to have a single axis. The shapes and / or colors of the pattern decorations 621 and 622 are designed in various ways and are not limited to Figure 6 the example shown.

[0137] An augmented reality (AR) device according to an embodiment (e.g., Figure 2 the AR device 201 in Figure 2 the communication module 210 in Figure 4 or the AR communication module 422 in Figure 2 the camera module 270 in Figure 4 or the AR camera module 423 in Figure 2 the display module 240 in Figure 4 or the AR display 421 in Figure 2 the sensor module 260 in Figure 4 or the AR wear detection sensor 424 in Figure 2 the memory 230 in Figure 4 or the AR memory 425 in Figure 2 the processor 220 in Figure 4The AR processor 420 in). According to an embodiment, the memory 230 or 425 may include instructions that cause the processor 220 or 420 to receive, via a communication module (e.g., Figure 2 the communication module 210 in Figure 4 the AR communication module 422 in), marker configuration information related to the display form and display method of the dial AR marker and configured in an electronic device (e.g., a wrist-worn electronic device) (e.g., Figure 4 the electronic device 101 in).

[0138] According to an embodiment, the memory may include instructions that cause the processor to detect, via a wear detection sensor (e.g., Figure 4 the AR wear detection sensor 424 in), that the AR device 201 is worn on the user's body and activate a camera (e.g., Figure 2 the camera module 270 in or Figure 4 the AR camera module 423 in).

[0139] According to an embodiment, the memory may include instructions that cause the processor to identify a dial AR marker and a strap pointing marker included in a camera video based on the marker configuration information configured in the wrist-worn electronic device.

[0140] According to an embodiment, the memory may include instructions that cause the processor to determine at least one of the position, posture, and tilt of the wrist-worn electronic device based on the identified dial AR marker and the identified strap pointing marker.

[0141] According to an embodiment, the memory may include instructions that cause the processor to output AR information corresponding to the posture or tilt based on the position of the wrist-worn electronic device.

[0142] According to an embodiment, the processor may be configured to adjust the driving frequency of a camera (e.g., Figure 2 the camera module 270 in or Figure 4 the AR camera module 423 in) based on the marker configuration information to have a performance equal to or higher than the scanning rate of the display (e.g., Figure 4 the display 411 in) of the wrist-worn electronic device.

[0143] According to an embodiment, the processor may be configured to identify an AR marker from a dial AR marker image or a dial image and identify a strap pointing marker included in the strap of the wrist-worn electronic device.

[0144] According to an embodiment, the processor may be configured to perform account synchronization with the wrist-worn electronic device via a communication module (e.g., Figure 2 the communication module 210 in or Figure 4 the AR communication module 422 in) to receive the marker configuration information.

[0145] The processor according to an embodiment may be configured to identify an electronic device by analyzing camera video in units of frames and identifying an AR marker displayed on a display of a wrist-worn electronic device (e.g., the display 411 in Figure 4 ), identify position information or coordinate information of the wrist-worn electronic device, and identify the posture of the wrist-worn electronic device based on the number of strap-pointing markers identified in the camera video.

[0146] The processor according to an embodiment may be configured to determine whether the identified display of the wrist-worn electronic device (e.g., the display 411 in Figure 4 ) is in a posture where the front surface of the display faces the user's line of sight or in a posture where the side surface of the display faces the user's line of sight.

[0147] The processor according to an embodiment may be configured to, when the display of the wrist-worn electronic device is in a posture where the front surface of the display faces the user's line of sight, output AR information in a full version mode, indicating all information provided in combination with the wrist-worn electronic device, and when the display of the wrist-worn electronic device (e.g., the display 411 in Figure 4 ) is in a posture where the side surface faces the user's line of sight, output AR information in a simple version mode, indicating schematic information about the wrist-worn electronic device.

[0148] The processor according to an embodiment may be configured to receive, via a communication module (e.g., the communication module 210 in Figure 2 or the AR communication module 422 in Figure 4 ), screen configuration information of a dial image that is configured to be a representative image of a display of the wrist-worn electronic device (e.g., the display 411 in Figure 4 ), monitor the dial image displayed on the display of the wrist-worn electronic device (e.g., the display 411 in Figure 4 ), and calculate the tilt or rotation of the wrist-worn electronic device based on deformation information between feature points of an image stored based on the received screen configuration information and feature points of the monitored dial image.

[0149] The processor according to an embodiment may be configured to receive, via a communication module (e.g., the communication module 210 in Figure 2 or the AR communication module 422 in Figure 4 ), motion sensor information collected in the wrist-worn electronic device from the wrist-worn electronic device, and supplement the position, posture, and tilt of the wrist-worn electronic device based on the motion sensor information of the wrist-worn electronic device.

[0150] The processor according to an embodiment may be configured to track via a display (e.g.,Figure 4 Whether the movement of the user's finger or a specific gesture is detected in the camera video shown in the AR display 421), and when the movement of the user selects to output AR information, the control information about the selected AR information is sent to the wrist-worn electronic device via the communication module (e.g., Figure 2 the camera module 270 in Figure 4 or the AR camera module 423 in

[0151] According to an embodiment, the processor may be configured to, when the AR device 201 is worn on the user's body, send the wearing detection information to the wrist-worn electronic device via the communication module (e.g., Figure 2 the camera module 270 in Figure 4 or the AR camera module 423 in

[0152] According to an embodiment, an electronic device (e.g., Figure 1 and Figure 4 the electronic device 101 in Figure 1 may include: a communication module including a communication circuit (e.g., Figure 4 the communication module 190 in Figure 1 or the communication module 412 in Figure 4 ; a display (e.g., Figure 1 the display module 160 in Figure 4 or the display 411 in Figure 1 ; a wearing detection sensor (e.g., Figure 4 the sensor module 176 in Figure 1 or the wearing detection sensor 413 in Figure 4 ; a memory (e.g.,

[0153] the memory 130 in

[0154] Figure 4 or the memory 415 in Figure 1 and a processor (e.g., Figure 4 the processor 120 in

[0153] The memory may include instructions that cause the processor to perform the following operations: display a dial image configured to represent an image on the display; send the marker configuration information about the display form and display method of the dial AR marker to the AR device 201 via the communication module; and display the dial AR marker image on the display based on the marker configuration information, and display the dial AR marker image on the display based on detecting that the electronic device (e.g., the wrist-worn electronic device) is worn on the user's body via the wearing detection sensor and detecting that the AR device 201 is worn on the user's body from the AR device 201.

[0154] According to an embodiment, the processor may be configured to display a dial image configured to represent an image on a display based on a scan rate frame of the display, and then display a dial AR marker image 1 to 5 times per second.

[0155] According to an embodiment, the processor may control the communication module to send marker configuration information to the AR device 201 via synchronization with the account of the AR device 201.

[0156] According to an embodiment, the electronic device 101 may further include: a housing in which a communication module, a display, a wear detection sensor, a memory, and a processor are installed; a strap structure configured to be wearable on or detachable from a user's body at both ends of the housing, wherein the strap structure includes two pattern decorations having a shape or form for identification in the AR device.

[0157] According to an embodiment, the two pattern decorations may include a first pattern decoration having a first direction and located at one end connected to the housing, and a second pattern decoration having a second direction and located at the other end of the housing, and the first direction and the second direction may be designed to have a single axis.

[0158] According to an embodiment, the processor may be configured to send screen configuration information of a dial image configured to represent an image to the AR device 201 via account synchronization.

[0159] According to an embodiment, the electronic device 101 may further include a motion sensor (e.g., Figure 4 the motion sensor 414 in

[0160] Figure 7 A method of providing AR information by using a dial image in an electronic device and an AR device according to an embodiment is shown. In the following embodiments, each operation may be performed sequentially, but not necessarily sequentially. For example, the order of each operation may be changed, or at least two operations may be performed in parallel.

[0161] Referring to Figure 7 According to an embodiment, an electronic device (e.g., Figure 1 and Figure 4 the electronic device 101 in

[0162] For example, as Figure 5As shown at 501, the electronic device 101 may be configured to generate a separate dial AR marker image, display a dial image configured to represent an image, and then display the dial AR marker image only in a specified (specific) frame.

[0163] In another example, as Figure 5 shown at 502, the electronic device 101 may display a dial image including a pattern as a representative image, where the pattern includes marker identification information. In another example, as Figure 5 shown at 503, the electronic device 101 may send screen configuration information configured to represent an image to an AR device (e.g., Figure 2 , Figure 3 and Figure 4 the AR device 201 in), and may specify a normalized image of the dial image as the AR marker image.

[0164] In operation 720, the electronic device 101 may synchronize marker configuration information via an account connection with the AR device 201.

[0165] According to an embodiment, the electronic device 101 may send marker configuration information configured in the electronic device 101 to the AR device 201 via wireless communication.

[0166] In operation 730, the AR device 201 (or the AR processor of the AR device 201 (e.g., Figure 2 the processor 220 in or Figure 4 the AR processor 420 in)) may detect the wearing of the AR device. For example, the AR device 201 may detect the wearing or removal of the AR device 201 on the user's body based on sensing information obtained from an AR wearing detection sensor (e.g., Figure 4 the AR wearing detection sensor 424 in). In operation 735, the AR device 201 may send wearing information or removal information to the electronic device 101. According to some embodiments, operation 735 may be omitted.

[0167] In operation 740, the electronic device 101 may control the display of the dial AR marker image on a display (e.g., Figure 1 the display module 160 in or Figure 4 the display 411 in) according to the display configuration of the AR marker. For example, when a first display method is configured for the AR marker, the electronic device 101 may perform control to display a dial image configured to represent an image, and then output the dial AR marker image in a specific frame.

[0168] According to an embodiment, in response to a situation in which AR device wearing information is received from AR device 201, electronic device 101 may perform control to display a dial image configured to represent an image on display 411, and then display a dial AR marker image on display 411.

[0169] In operation 750, AR device 201 may activate an AR camera (e.g., the camera module 270 in Figure 2 or the AR camera module 423 in Figure 4 ) based on detection of wearing of the AR device.

[0170] In operation 755, AR device 201 may obtain a camera video of electronic device 101 from AR camera module 423. For example, AR camera module 423 may obtain video information obtained by capturing an image related to real space corresponding to the field of view of a user wearing AR device 201, and may send the video information to an AR processor (e.g., the processor 220 in Figure 2 or the AR processor 420 in Figure 4 ).

[0171] In operation 760, AR device 201 may identify a dial AR marker in the camera video, and in operation 765, AR device 201 may identify a strap pointing marker. Operations 760 and 765 may be performed as one operation.

[0172] For example, AR processor 420 may analyze the camera video based on marker configuration information sent from electronic device 101, and may track at least one of the AR marker and the strap pointing marker from the camera video. For example, AR processor 420 may capture the camera video frame by frame, and may identify the AR marker and / or the strap pointing marker from the dial AR marker image or the dial image by using a tracking algorithm using a specific value (e.g., color or shape).

[0173] In operation 780, AR device 201 may identify electronic device 101 based on marker identification information included in the AR marker, may identify the position of electronic device 101 (e.g., coordinate information), and may determine the tilt and pose of the identified electronic device 101.

[0174] For example, AR processor 420 may determine the tilt and pose of electronic device 101 identified in the camera video based on the identified strap pointing marker information. For example, when two strap pointing markers are identified in the camera video, AR processor 420 may determine that the electronic device is in a pose where the front surface of the electronic device faces the user's line of sight, and when one strap pointing marker is identified, AR processor 420 may determine that the electronic device is in a pose where the side surface of the electronic device faces the user's line of sight.

[0175] In operation 790, the AR device 201 may output AR information associated with the electronic device to the identified position of the electronic device (e.g., coordinate information) based on the determined pose and tilt of the electronic device.

[0176] The AR processor 420 may perform control to configure the position (i.e., coordinate information) of the identified electronic device 101 in the camera video as the display coordinates of the AR information, and output AR information corresponding to (or suitable for or optimized for) the pose and tilt of the identified electronic device.

[0177] For example, when the front surface of the electronic device 101 faces the user's line of sight, the AR processor 420 may perform control to output AR information (e.g., first type of AR information) in the full version mode, indicating detailed information associated with the electronic device 101. When the side surface of the electronic device 101 faces the user's line of sight, the AR processor 420 may perform control to output AR information in the simple version mode (e.g., second type of AR information) on the AR display 421 (e.g., AR glasses), indicating schematic information.

[0178] In addition, in operation 770, the electronic device 101 may detect motion sensor information from a motion sensor (e.g., Figure 4 the motion sensor 414 in ), and in operation 775, the electronic device 101 may send the motion sensor information to the AR device 201 in real time. According to some embodiments, operations 770 and 775 may be omitted.

[0179] According to some embodiments, when receiving the motion sensor information of the electronic device 101 sent from the electronic device 101, the AR processor 420 may use the result of analyzing the motion sensor information of the electronic device 101 to supplement marker tracking so that when the AR marker (or the electronic device 101) is outside the camera video, the position and / or orientation of the electronic device 101 can be estimated.

[0180] Figure 8 A screen showing a first display method of a dial AR marker image in an electronic device according to an embodiment is shown.

[0181] Referring to Figure 8 , according to an embodiment, when the display method is configured as the first display method, the electronic device (e.g., Figure 1 and Figure 4 the electronic device 101 in ) may adjust and display the dial image 810 configured to represent an image and the separately generated dial AR marker image 820 on a frame-by-frame basis.

[0182] According to an embodiment, when the display of the electronic device 101 (e.g., the display module 160 in Figure 1 or the display 411 in Figure 4 outputs an image at a scanning rate of 60 to 120 Hz (i.e., the number of frames displayed per second), the electronic device 101 may perform control to display the dial AR marker image 820 only in one or two frames of the scanning rate, such that the AR device (e.g., the AR device 201 in Figure 2 , Figure 3 and Figure 4 ) can recognize the AR marker. For example, since the dial AR marker image is displayed only in 1 to n frames per second, the user cannot visually recognize the AR marker image and can only view the dial image configured to be representative on the display of the AR device.

[0183] For example, assuming that the display 411 of the electronic device 101 displays the screen at a scanning rate of 60 frames per second (e.g., 60 Hz), as shown in <801>, the electronic device 101 may control the display 411 to display the dial image 810 configured to be representative on a frame-by-frame basis in frames 1 to 59, and then display the dial AR marker image 820 in frame 60.

[0184] In another example, as shown in <802>, the electronic device 101 may control the display 411 to display the dial image 810 configured to be representative on a frame-by-frame basis in frames 1 to 29 and 31 to 59, and then display the dial AR marker image 820 in frames 30 and 60.

[0185] The electronic device 101 may perform control to display the dial AR marker image 820 only in several frames of the display scanning rate unit, such that the AR device 201 can recognize the AR marker within a range where the dial AR marker image 820 is not visible to the user's eyes. In this case, only the dial image 810 configured to be representative of the image can be seen by the actual user's eyes.

[0186] Figure 9 Shows an example of displaying AR information in the AR device 201 by using the dial image of the electronic device according to an embodiment.

[0187] Referring to Figure 9 , the dial image of the electronic device according to an embodiment (e.g., the electronic device 101 in Figure 1 and Figure 4 ) is designated as the AR device (e.g., Figure 2 , Figure 3 and Figure 4In the AR device 201), for an AR marker (e.g., the third display method), the AR device 201 can extract a pattern for identifying the marker based on the dial image of the electronic device 101 in the camera video, calculate the axis for outputting AR information, and then synthesize a 3D rendered image to output AR information 950.

[0188] For example, the electronic device 101 can specify the form of the first image 910 as the representative image to be displayed on the dial. The electronic device 101 can send the screen configuration information of the first image 910 configured as the representative image to the AR device 201 via an account link with the AR device. The AR device 201 can extract specific points for identifying the first image 910 based on the pre-sent screen configuration information of the first image 910, and extract the normalized image 911 of the first image 910. The AR device 201 can identify the electronic device 101 in the camera video by monitoring the camera video based on the normalized image 911 of the first image 910. The AR device 201 can monitor the first image 910 included in the camera video, and can calculate how many degrees the axis of the image has rotated or tilted based on the degree of deformation of the feature (e.g., the bar indicating time) in the normalized image 911, thereby determining the direction of the electronic device 101 and estimating the posture / tilt of the electronic device 101. In Figure 9 this case, each feature point can be at an angle of rotating approximately 32 degrees on the x-axis, rotating approximately -36.4 degrees on the y-axis, and rotating approximately -32 degrees on the z-axis. The AR device 201 can output AR information on the electronic device 101 based on the rotated angle. The AR information can be displayed based on the identified position and rotation degree of the electronic device 101, and when the electronic device 101 moves, the display position of the AR information can also move along with the movement of the electronic device 101.

[0189] In the case of the second image 920 without the bar indicating time, the AR device 201 can extract the normalized image 921 of the second image 920 having the hour hand and the minute hand as feature points, and analyze the degree of deformation of the feature point pattern of the hour hand and the minute hand, thereby identifying the direction of the electronic device 101 and estimating the posture / tilt of the electronic device 101.

[0190] Optionally, in the case of the third image 930 without the hour hand and the minute hand and in the case of a digital clock, the AR device 201 can extract the normalized image 931 of the third image 930 based on the hour display information that changes only once per hour, and analyze the information on the deformation of the feature point pattern regarding the hour display information, thereby identifying the direction of the electronic device 101 and estimating the posture / tilt of the electronic device 101.

[0191] Figure 10Shows an example of providing AR information in an AR device according to an embodiment.

[0192] Referring to Figure 10 , according to an embodiment, an AR device (e.g., the AR device 201 in Figure 2 , Figure 3 and Figure 4 ) can perform control to output AR information together with a camera video via an AR display, where the AR information corresponds to (or is suitable for or optimized for) the identified pose and tilt of an electronic device (e.g., the electronic device 101 in Figure 1 and Figure 4 ). The electronic device 101 can display a dial image 1015 on the display 1010.

[0193] When it is determined, based on a strap pointing marker implemented in the strap 1020 of the electronic device 101, that the front surface of the electronic device 101 faces the user's line of sight, the AR device 201 can perform control to output AR information (e.g., first type AR information 1030) in the full version mode as shown in <1001>, indicating detailed information associated with the electronic device 101. For example, when two strap pointing markers 1040 and 1041 are identified in the camera video, the AR device 201 can determine the distance (d) between the two strap pointing markers and can display AR information (e.g., function icons of the electronic device) to be output along a circular line with a diameter equal to this distance. The screen configuration of the AR information is only an example and can be provided in various forms.

[0194] When it is determined that the side surface of the electronic device 101 faces the user's line of sight, as shown in <1002>, the AR device 201 can perform control to output AR information (e.g., second type AR information 1035) in the simple version mode on the AR display 421 (e.g., AR glasses), indicating schematic simple information. For example, when one strap pointing marker is identified in the camera video, the AR device 201 can control the AR information to be output as coordinates that are between the virtual extension lines 1050 and 1055 on both sides of the identified strap pointing marker. The simple version of the AR information can include schematic simple information such as time, date, battery information, etc., but is not necessarily limited thereto.

[0195] Figure 11 Shows a method for displaying a dial marker in an electronic device according to an embodiment.

[0196] Referring to Figure 11 , an electronic device according to an embodiment (e.g., the electronic device 101 in Figure 1 and Figure 4 ) can support an AR device (e.g., Figure 2 ,Figure 3 and Figure 4 the function of whether the AR device 201 in Figure 4 is worn to control the display of the dial AR marker image.

[0197] In operation 1110, the electronic device 101 may detect the wearing of the electronic device. The electronic device 101 may detect whether the electronic device 101 is worn on the user's body based on sensor information obtained via a wearing detection sensor.

[0198] In operation 1120, the electronic device 101 may determine whether the wearing of the AR device 201 is detected. For example, the electronic device 101 may receive AR device wearing detection information or AR device detachment information from the AR device 201. When receiving the AR device wearing detection information from the AR device 201, the electronic device 101 may identify the wearing of the AR device 201.

[0199] In operation 1130, when the wearing of the AR device is detected, the electronic device 101 may control the dial AR marker image to be displayed on the display 411.

[0200] In operation 1140, when the wearing of the AR device is not detected, the electronic device 101 may control a normal dial image configured to represent an image to be displayed on the display 411.

[0201] Figure 12 Illustrates a method for providing AR information by using a dial marker in an AR device 201 according to an embodiment.

[0202] Referring to Figure 12 , in operation 1201, a processor of an AR device (e.g., the AR device 201 in Figure 12 , Figure 2 , Figure 3 , Figure 4 ) (e.g., the processor 220 in Figure 2 or the AR processor 420 in Figure 4 ) may obtain, via account synchronization, marker configuration information related to the display form and / or display method of the dial marker and configured in an electronic device (e.g., the electronic device 101 in Figure 1 , Figure 4 ). Figure 2 , Figure 3 and Figure 4 the AR device 201 in Figure 4 ) (e.g., the processor 220 in Figure 2 or the AR processor 420 in Figure 4 ) may obtain, via account synchronization, marker configuration information related to the display form and / or display method of the dial marker and configured in an electronic device (e.g., the electronic device 101 in Figure 1 , Figure 4 ). Figure 2 the processor 220 in Figure 2 or Figure 4 the AR processor 420 in Figure 4 ) may obtain, via account synchronization, marker configuration information related to the display form and / or display method of the dial marker and configured in an electronic device (e.g., the electronic device 101 in Figure 1 , Figure 4 ). Figure 1 and Figure 4 the electronic device 101 in Figure 4 ).

[0203] In operation 1203, the processor 220 or 420 of the AR device 201 may detect that the AR device 201 is worn on the user's body.

[0204] The processor 220 or 420 may detect that the AR device 201 is worn on the user's body based on an AR wearing detection sensor.

[0205] In operation 1205, the processor 220 or 420 of the AR device 201 may obtain a camera video by activating a camera (e.g., Figure 4 the AR camera module 423 in

[0206] In operation 1207, the processor 220 or 420 of the AR device 201 may identify a dial AR marker and / or a pointing marker included in the camera video based on marker configuration information configured in the electronic device.

[0207] The processor 220 or 420 may analyze the camera video based on the marker configuration information sent from the electronic device 101 to track at least one of the AR marker and / or the strap pointing marker from the camera video. For example, the processor 220 or 420 may capture the camera video frame by frame and may identify the AR marker and / or the strap pointing marker from the dial AR marker image or the dial image by using a tracking algorithm using specific values (e.g., color and / or shape).

[0208] In operation 1209, the processor 220 or 420 of the AR device 201 may identify a position and may determine the pose and / or tilt of the electronic device 101.

[0209] The processor 220 or 420 may identify the electronic device 101 based on the marker identification information included in the AR marker and may identify the position (e.g., coordinate information) of the electronic device 101.

[0210] The processor 220 or 420 may identify the tilt and pose of the electronic device 101 via the strap pointing marker. For example, when two strap pointing markers are identified in the camera video, the processor 220 or 420 may determine that the electronic device is in a pose where the front surface of the electronic device faces the user's line of sight, and when one strap pointing marker is identified, the processor 220 or 420 may determine that the electronic device is in a pose where the side surface of the electronic device faces the user's line of sight.

[0211] In operation 1211, the processor 220 or 420 of the AR device 201 may output AR information suitable for the position of the electronic device and the pose and / or tilt of the electronic device.

[0212] For example, when the front surface of the electronic device 101 faces the user's line of sight, the processor 220 or 420 may execute control to output AR information (e.g., first type of AR information) in the full version mode, indicating detailed information associated with the electronic device 101. When the side surface of the electronic device 101 faces the user's line of sight, the processor 220 or 420 may execute control to output AR information (e.g., second type of AR information) in the simple version mode on the AR display 421 (e.g., AR glasses), indicating schematic information.

[0213] The AR device according to an embodiment (e.g., Figure 2 , Figure 3 and Figure 4 the AR device 201 among them) The method of providing AR information by using the dial image may include: receiving, from the electronic device 101, the marker configuration information related to the display form and display method of the dial AR marker and configured in the electronic device (e.g., Figure 1 and Figure 4 the electronic device 101 among them). The method according to an embodiment may include detecting that the AR device 201 is worn on the user's body and activating the camera. The method according to an embodiment may include identifying the dial AR marker and the strap pointing marker included in the camera video based on the marker configuration information configured in the electronic device 101. The method according to an embodiment may include determining at least one of the position, posture, and tilt of the electronic device 101 based on the identified dial AR marker and the identified strap pointing marker. The method according to an embodiment may include: outputting AR information corresponding to the posture or tilt based on the position of the electronic device 101.

[0214] According to an embodiment, the electronic device 101 may be a wrist-worn electronic device.

[0215] According to an embodiment, the wrist-worn electronic device may be an electronic device configured to display a dial image configured to represent an image based on a scan rate frame of a display and then selectively display a dial AR marker image within 1 to 5 times per second.

[0216] As used in connection with various embodiments of the present disclosure, the term "module" may include a unit implemented in hardware, software, or firmware and may be used interchangeably with other terms (e.g., "logic", "logic block", "part", or "circuit"). A module may be a single integrated component adapted to perform one or more functions or the smallest unit or part of the single integrated component. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0217] The various embodiments described herein can be implemented as software (e.g., program 140) including one or more instructions readable by a machine (e.g., electronic device 101) stored in a storage medium (e.g., internal memory 136 or external memory 138). For example, under the control of a processor, a processor (e.g., processor 120) of the machine (e.g., electronic device 101) can call at least one of the one or more instructions stored in the storage medium and run the at least one instruction with or without using one or more other components. This enables the machine to operate to perform at least one function in accordance with the at least one instruction called. The one or more instructions can include code generated by a compiler or code that can be run by an interpreter. The machine-readable storage medium can be provided in the form of a non-transitory storage medium. Herein, the term "non-transitory" only means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but this term does not distinguish between data being stored semi-permanently in the storage medium and data being stored temporarily in the storage medium.

[0218] According to an embodiment, a method according to various embodiments of the present disclosure can be included and provided in a computer program product. The computer program product can be traded between a seller and a purchaser as a product. The computer program product can be distributed (e.g., downloaded or uploaded) in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or can be distributed online via an application store (e.g., Play StoreTM) (e.g., downloaded or uploaded), or can be directly distributed (e.g., downloaded or uploaded) between two user devices (e.g., smart phones). If it is distributed online, at least part of the computer program product can be generated temporarily, or at least part of the computer program product can be stored at least temporarily in a machine-readable storage medium (such as the memory of a manufacturer's server, an application store's server, or a forwarding server).

[0219] According to various embodiments, each of the above components (e.g., a module or a program) may include a single entity or multiple entities, and some of the multiple entities may be separately provided in different components. According to various embodiments, one or more of the above components may be omitted, or one or more other components may be added. Optionally or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform the one or more functions of each of the multiple components in the same or similar manner as the corresponding one of the multiple components performed the one or more functions before integration. According to various embodiments, the operations performed by a module, a program, or another component may be performed sequentially, in parallel, repeatedly, or in a heuristic manner, or one or more of the operations may be run in a different order or omitted, or one or more other operations may be added.

[0220] Although the present disclosure has been shown and described with reference to various exemplary embodiments, it should be understood that the various exemplary embodiments are illustrative and not restrictive. Those skilled in the art will further understand that various changes can be made in form and detail without departing from the true spirit and scope of the present disclosure (including the appended claims and their equivalents). It will also be understood that any embodiment described herein can be used in combination with any other embodiment described in the present disclosure.

Claims

1. An augmented reality (AR) device, comprising: a communication module, which includes a communication circuit; a camera; a display; a wearing detection sensor; a memory; and a processor, wherein the memory includes instructions that, when run by the processor, cause the AR device to: receive, via the communication module, marker configuration information related to the display form and display method of a dial AR marker and configured in an electronic device; detect, via the wearing detection sensor, that the AR device is worn on a user's body and activate the camera; identify a dial AR marker and a strap pointing marker included in the camera video based on the marker configuration information configured in the electronic device; determine at least one of the position, posture, and tilt of the electronic device based on the identified dial AR marker and the identified strap pointing marker; and output AR information corresponding to the posture or tilt based on the position of the electronic device.

2. The AR device according to claim 1, wherein the electronic device is a wrist-worn electronic device, and wherein the processor is configured to adjust the driving frequency of the camera based on the marker configuration information to have a performance equal to or higher than the scanning rate of the display of the electronic device.

3. The AR device according to claim 1, wherein the processor is configured to identify an AR marker from a dial AR marker image or a dial image, and identify a strap pointing marker included in the strap of the electronic device.

4. The AR device according to claim 3, wherein the processor is configured to receive the marker configuration information by performing account synchronization with the electronic device via the communication module.

5. The AR device according to claim 1, wherein the processor is configured to: identify the electronic device by analyzing the camera video frame by frame and identifying an AR marker displayed on the display of the electronic device, and identify the position information or coordinate information of the electronic device; and identify the posture of the electronic device based on the number of strap pointing markers identified in the camera video.

6. The AR device according to claim 5, wherein the processor is configured to determine whether the display of the identified electronic device is in a posture where the front surface of the display faces the user's line of sight or in a posture where the side surface of the display faces the user's line of sight.

7. The AR device according to claim 6, wherein the processor is configured to: in the case where the display of the electronic device is in a posture where the front surface of the display faces the user's line of sight, output AR information in a full version mode, the AR information indicating all information provided in combination with the electronic device; and in the case where the display of the electronic device is in a posture where the side surface of the display faces the user's line of sight, output AR information in a simple version mode, the AR information indicating schematic information about the electronic device.

8. The AR device according to claim 1, wherein the processor is configured to: Receiving, via the communication module, screen configuration information of a dial image configured as a representative image of a display of the electronic device; Monitoring, in the camera video, a dial image displayed on the display of the electronic device; And Calculating an inclination or rotation of the electronic device via deformation information between feature points of an image stored based on the received screen configuration information and feature points of the monitored dial image.

9. The AR device according to claim 1, Wherein, The processor is configured to: Receive, via the communication module, motion sensor information collected in the electronic device from the electronic device; and Supplement a position, a posture, and an inclination of the electronic device based on the motion sensor information of the electronic device.

10. The AR device according to claim 1, Wherein, The processor is configured to track whether a movement of a user's finger or a specific gesture is detected in the camera video shown via the display, and in a case where the movement of the user selects the output AR information, send control information regarding the selected AR information to the electronic device via the communication module.

11. The AR device according to claim 1, Wherein, The processor is configured to, in a case where the AR device is worn on a user's body, send wearing detection information to the electronic device via the communication module, and in a case where the AR device is detached from the user's body, send detachment detection information to the electronic device via the communication module.

12. An electronic device, Comprising: A communication module including a communication circuit; A display; A wearing detection sensor; A memory; And A processor, Wherein, the memory includes instructions that, when run by the processor, control the electronic device to: Display a dial image configured as a representative image on the display; Send, via the communication module, marker configuration information regarding a display form and a display method of a dial AR marker to an AR device; and Based on detecting that the electronic device is worn on a user's body via the wearing detection sensor and detecting that the AR device is worn on the user's body from the AR device, display a dial AR marker image on the display based on the marker configuration information, and Wherein, the processor is configured to display the dial image configured as a representative image on the display based on a scan rate frame of the display, and then display the dial AR marker image within 1 to 5 times per second.

13. The electronic device according to claim 12, including a wrist-worn electronic device, Wherein, The processor is configured to control the communication module to send the marker configuration information to the AR device via an account synchronization with the AR device.

14. The electronic device according to claim 12, Wherein, The processor is configured to send screen configuration information of the dial image configured as a representative image to the AR device via the account synchronization.

15. The electronic device according to claim 12, further including a motion sensor, Wherein, The processor is configured to send the motion sensor information detected by the motion sensor to the AR device.

Images