Movable projector and operation method for movable projector

Abstract

A mobile projector device comprises: a drive assembly comprising at least one wheel; a transceiver; a projector; a memory for storing instructions; and at least one processor, wherein the instructions, when executed by the at least one processor, instruct the mobile projector device to: determine whether to set the mobile projector device as an extended display device associated with a source device; receive information about an image from the source device through the transceiver on the basis of determining to set the mobile projector device as the extended display device; determine a projection position of the mobile projector device on which the image is to be projected, on the basis of position information of the source device; and control the projector to project the image while the mobile projector device is at the projection position.

Description

Mobile Projectors and How They Work

[0001] The present disclosure relates to a mobile projector and a method of operating the mobile projector.

[0002] Robots have been developed in industrial sectors like manufacturing and are being used as part of factory automation. Recently, new technologies have been researched, leading to the development of various robots, such as surgical robots for medical use or military use. Their use is expanding beyond industrial fields to homes as well.

[0003] Mobile robots can autonomously navigate specific areas and execute commands either previously entered or received in real time. Mobile robots can be used as cleaning robots, telepresence robots, or security robots, and are being developed for a variety of purposes across diverse fields.

[0004] The information disclosed in this Background section may be information already known or derived by the inventors during the process of completing the embodiments of the present application, or may be technical information acquired during the process of completing the embodiments. Accordingly, it may include information that does not constitute prior art already known to the public.

[0005] Mobile robots can operate based on an operating system and, based on communication capabilities, receive various content from servers on a network. Mobile robots can include a display and display content provided by the server. Mobile robots with displays can perform display operations while moving. However, since mobile robots can only display content on a display, they are limited to functioning as mobile display devices, similar to mobile terminals.

[0006] Typical mobile robots don't include projectors. Projectors can display screens of various sizes and locations, offering greater versatility than fixed-size displays. However, mobile robots that incorporate projectors have not been considered or utilized in the past, necessitating further development.

[0007] Additional aspects are some of which are set forth in the description below, some of which will be apparent from the description, or some of which may be learned by practice of the embodiments presented.

[0008] According to one aspect of the present disclosure, a mobile projector device may include a drive assembly including at least one wheel, a transceiver, a projector, a memory storing commands, and at least one processor, wherein the commands, when executed by the at least one processor, cause the mobile projector device to determine whether to set the mobile projector device as an extended display device associated with a source device, receive information about an image from the source device through the transceiver based on the determination to set the mobile projector device as the extended display device, determine a projection position at which to project the image (e.g., a projection position of the mobile projector device at which to project the image) based on position information of the source device, and control the projector to project the image at the projection position (e.g., while the mobile projector device is at the projection position).

[0009] The above instructions, when executed by the at least one processor, may further cause the mobile projector device to receive, from the source device through the communication unit, a request message requesting the mobile projector device to set the mobile projector device as the extended display device, and, based on reception of the request message, determine whether a projection operation by the projection unit is being performed, and, based on determining that the projection operation is being performed, determine not to set the mobile projector device as the extended display device and control the communication unit to transmit a rejection message to the source device, and, based on determining that the projection operation is not being performed, determine to set the mobile projector device as the extended display device and control the communication unit to transmit an acceptance message to the source device.

[0010] The above instructions, when executed by the at least one processor, may further cause the mobile projector device to control the driving unit to move the mobile projector device to the projection position.

[0011] The above instructions, when executed by the at least one processor, may further cause the mobile projector device to identify an area adjacent to the source device while at the projection position and control the projection unit to project the image onto the identified area.

[0012] The mobile projector device may further include a user interface (UI), and the instructions, when executed by the at least one processor, may further cause the mobile projector device to identify a control signal based on interaction with the UI, and to stop projection of the image and deactivate the mobile projector device as the extended display device based on the identified control signal, wherein the identified control signal may include information instructing to project another image or information instructing movement of the mobile projector device.

[0013] The information about the image may include information about at least one of an image received in real time by the source device, an image associated with an application running on the source device, an image associated with a camera connected to the source device, and an image provided from a mobile terminal connected to the source device.

[0014] The mobile projector device may further include a main body, a driving unit, a plurality of sensors, and at least one camera, and the instructions, when executed by the at least one processor, may further cause the mobile projector device to determine, by the at least one camera and / or the plurality of sensors, the presence or absence of an obstacle in a direction in which the image is projected, and to control the driving unit so that the main body of the mobile projector device is rotated by a predetermined angle based on determining the presence of the obstacle.

[0015] The mobile projector device may further include a plurality of sensors and at least one camera, and the instructions, when executed by the at least one processor, may further cause the mobile projector device to determine the presence or absence of a change in height at the projection location by a first sensor among the plurality of sensors, and, based on determining the presence of the change in height, determine the presence or absence of a fall zone at a location adjacent to the mobile projector device by at least one of the at least one camera or a second sensor among the plurality of sensors, the fall zone indicating an area into which the mobile projector device may fall, and, based on determining the absence of the fall zone, control the projection unit to project the image using a first projection method, and, based on determining the absence of the change in height, control the projection unit to project the image using a second projection method, and the first projection method and the second projection method may differ in projection distance or projection direction.

[0016] The above instructions, when executed by the at least one processor, may further cause the mobile projector device to perform a face recognition operation by the at least one camera based on determining the presence of the fall section, determine whether a user's face is recognized based on the face recognition operation, and control the projector to project the image using the first projection method based on whether the user's face is not recognized.

[0017] The instructions, when executed by the at least one processor, may cause the mobile projector device to perform a face recognition operation by the at least one camera based on determining the presence of the fall section, determine whether a user's face is recognized based on the face recognition operation, determine whether a user's gesture is detected based on the recognition of the user's face, by the at least one camera, in a direction in which the user's face is recognized, identify whether the user's gesture is a first gesture or a second gesture based on the detection of the user's gesture, and control the projector to project the image using the first projection method based on the user's gesture being the first gesture, control the projector to project the image using the second projection method based on the user's gesture being the second gesture, and control the projector to project the image using the second projection method based on the user's gesture not being detected.

[0018] According to one aspect of the present disclosure, a method of operating a mobile projector device may include an operation of determining whether to set the mobile projector device as an extended display device associated with a source device, an operation of receiving information about an image from the source device based on the determination to set the mobile projector device as the extended display device, an operation of determining a projection position at which to project the image (e.g., a projection position of the mobile projector device at which to project the image) based on location information of the source device, and an operation of projecting the image at the projection position (e.g., while the mobile projector device is at the projection position).

[0019] The operation of determining whether to set the mobile projector device as the extended display device may include the operation of receiving, from the source device, a request message requesting to set the mobile projector device as the extended display device, the operation of determining whether a projection operation is being performed based on the reception of the request message, the operation of determining not to set the mobile projector device as the extended display device based on the determination that the projection operation is being performed, and transmitting a rejection message to the source device, and the operation of determining to set the mobile projector device as the extended display device based on the determination that the projection operation is not being performed, and transmitting an acceptance message to the source device.

[0020] The method may further include an operation of controlling a driving unit of the mobile projector device to move the mobile projector device to the projection position.

[0021] The operation of projecting the image may include an operation of identifying an area adjacent to the source device and an operation of projecting the image onto the identified area.

[0022] The method may include an operation of identifying a control signal generated based on an interaction with a user interface (UI) of the mobile projector device, and an operation of stopping projection of the image and deactivating the setting of the mobile projector device as the extended display device based on the identified control signal, wherein the identified control signal may include information instructing to project another image or information instructing movement of the mobile projector device.

[0023] The information about the image may include information about at least one of an image received in real time by the source device, an image associated with an application running on the source device, an image associated with a camera connected to the source device, and an image provided from a mobile terminal connected to the source device.

[0024] The method may further include an operation of determining the presence or absence of an obstacle in the direction in which the image is projected by at least one camera or at least one of a plurality of sensors of the mobile projector device, and an operation of changing the direction in which the image is projected by rotating the main body of the mobile projector device by a predetermined angle based on determining the presence of the obstacle.

[0025] The above-described operation of projecting an image may include an operation of determining the presence or absence of a change in height at the projection position by a first sensor among a plurality of sensors included in the mobile projector, an operation of determining the presence or absence of a fall zone at a location adjacent to the mobile projector device by at least one camera of the mobile projector device or at least one second sensor among the plurality of sensors based on the determination of the presence of the change in height, an operation of projecting the image using a first projection method based on the determination of the absence of the fall zone, wherein the fall zone indicates an area into which the mobile projector device may fall, and an operation of projecting the image using a second projection method based on the determination of the absence of the change in height. The first projection method and the second projection method may differ in projection distance or projection direction.

[0026] The method may include an operation of performing a face recognition operation by the at least one camera based on determining the existence of the fall section, an operation of determining whether a user's face is recognized based on the face recognition operation, and an operation of projecting the image using the first projection method based on whether the user's face is not recognized.

[0027] The method may include, based on determining the existence of the fall section, performing a face recognition operation by the at least one camera, determining whether a user's face is recognized based on the face recognition operation, determining whether a user's gesture is detected based on the recognition of the user's face, by the at least one camera, in a direction in which the user's face is recognized, determining whether the user's gesture is a first gesture or a second gesture based on the detection of the user's gesture, projecting the image using the first projection method based on the user's gesture being the first gesture, projecting the image using the second projection method based on the user's gesture being the second gesture, and projecting the image using the second projection method based on the user's gesture not being detected.

[0028] According to one aspect of the present disclosure, a mobile projector device includes a driving unit, a projector, a memory storing commands, and at least one processor, wherein the commands, when executed by the at least one processor, cause the mobile projector device to set the mobile projector device as an extended display device associated with a source device, receive information about an image from the source device, determine a projection position of the mobile projector device based on position information about the source device, control the driving unit to move the mobile projector device to the projection position, and control the projector to project the image when the mobile projector is at the projection position.

[0029] The above instructions, when executed by the at least one processor, may cause the mobile projector device to identify an area adjacent to the source device when the mobile projector is at the projection position, and control the projector to project the image to the identified area.

[0030] The above mobile projector device may further include a short-focus projection lens and a long-focus projection lens, and the mobile projector device may be configured to project the image by at least one of a first projection method using the short-focus projection lens and a second projection method using the long-focus projection lens.

[0031] The above mobile projector device may further include an inertial sensor and a distance sensor, and the instructions, when executed by the at least one processor, may further cause the mobile projector device to determine, by the inertial sensor, whether the projection position has a change in height.

[0032] The above instructions, when executed by the at least one processor, may further cause the mobile projector device to determine, by the distance sensor, whether the height change causes the mobile projector device to fall based on determining that the projection position has a height change, and to control the projector to project the image using the first projection method based on determining that the height change causes the mobile projector device to fall.

[0033] The instructions, when executed by the at least one processor, may further cause the mobile projector device to determine, by the distance sensor, whether the height change causes the mobile projector device to fall based on determining that the projection position has a height change, perform a face recognition operation based on determining that the height change causes the mobile projector device to fall, determine whether the user's face is recognized by the face recognition operation, and control the projector to project the image using the first projection method based on determining that the user's face is not recognized.

[0034] The instructions, when executed by the at least one processor, may further cause the mobile projector device to determine, by the distance sensor, whether the height change causes the mobile projector device to fall based on determining that the projection position has a height change, perform a face recognition operation based on determining that the height change causes the mobile projector device to fall, determine whether a user's face is recognized by the face recognition operation, determine, by at least one camera in a direction in which the user's face is recognized, whether a gesture of the user is detected based on the recognition of the user's face, and identify, based on the detection of the user's gesture, whether the user's gesture is a first gesture or a second gesture, control the projector to project the image using the first projection method based on the user's gesture being the first gesture, control the projector to project the image using the second projection method based on the user's gesture being the second gesture, and control the projector to project the image using the second projection method based on the user's gesture not being detected.

[0035] The above instructions, when executed by the at least one processor, may further cause the mobile projector device to: control the projector to project the image using the second projection method based on determining that the projection position does not have a height change.

[0036] The above-described aspects and other aspects, features and advantages will become more apparent from the following description with reference to the accompanying drawings.

[0037] FIG. 1A is a diagram illustrating a mobile projector according to one or more embodiments.

[0038] FIG. 1b is a block diagram of a mobile projector according to one or more embodiments.

[0039] FIG. 2 is a diagram illustrating a mobile projector and a source device according to one or more embodiments.

[0040] FIG. 3 is a flowchart illustrating the operation of a mobile projector according to one or more embodiments.

[0041] FIG. 4A is a flowchart illustrating an operation of accepting an operation request for a mobile projector to operate as an extended display device according to one or more embodiments.

[0042] FIG. 4b is a flowchart of a mobile projector and a source device for operation of an extended display device according to one or more embodiments.

[0043] FIG. 4c is a flowchart of a mobile projector and a source device for operation of an extended display device according to one or more embodiments.

[0044] FIG. 4D is a diagram illustrating a multi-view setup screen that may be displayed on a source device when a mobile projector according to one or more embodiments can operate as an extended display device of the source device.

[0045] FIG. 5A is a signal flow diagram illustrating an operation of a mobile projector according to one or more embodiments to reject an operation request to operate as an extended display device.

[0046] FIG. 5b is a diagram illustrating a multi-view setting screen that may be displayed on a source device when a mobile projector according to one or more embodiments cannot operate as an extended display device of the source device.

[0047] FIG. 6A is a diagram illustrating a multiview displayed on a source device according to one or more embodiments.

[0048] FIG. 6b is a diagram illustrating a multi-view of a source device displaying a mobile projector icon according to one or more embodiments.

[0049] FIG. 6c is a diagram illustrating a multiview based on selection of a movable projector icon according to one or more embodiments.

[0050] FIG. 7A is a flowchart illustrating operations of a source device and a mobile terminal for displaying a multiview associated with a mobile terminal according to one or more embodiments.

[0051] FIG. 7b is a diagram illustrating a multi-view of a source device including an image of a mobile terminal according to one or more embodiments.

[0052] FIG. 8A is a flowchart illustrating the operation of a mobile terminal, a source device, or a mobile projector for displaying a multiview associated with a mobile terminal according to one or more embodiments.

[0053] FIG. 8B is a flowchart illustrating another operation of a mobile terminal, source device, or mobile projector for displaying a multiview associated with a mobile terminal according to one or more embodiments.

[0054] FIG. 8c is a diagram illustrating an example in which an image of a mobile terminal constituting a multi-view according to one or more embodiments is projected onto an area adjacent to a source device.

[0055] FIG. 8d is a diagram illustrating an image of a mobile terminal configuring a multi-view according to one or more embodiments being projected onto an area adjacent to a user.

[0056] FIG. 9 is a diagram illustrating projectible areas based on the location of a source device according to one or more embodiments.

[0057] FIGS. 10A and 10B are diagrams illustrating sources for multi-view according to one or more embodiments.

[0058] FIG. 11 is a flowchart illustrating an operation of a mobile projector according to one or more embodiments to determine a projection position.

[0059] FIG. 12 is a diagram illustrating a map that may be used in a mobile projector according to one or more embodiments.

[0060] FIG. 13A is a flowchart illustrating an operation of a mobile projector according to one or more embodiments to determine a projection method based on a change in height and the presence of a falling section.

[0061] FIG. 13b is a flowchart illustrating an operation of a mobile projector according to one or more embodiments to determine a projection method based on the results of a user's face recognition and gesture detection.

[0062] FIG. 14a and FIG. 14b are diagrams illustrating a situation in which the height of a mobile projector changes according to one or more embodiments.

[0063] FIG. 15a is a drawing for explaining a long-focus projection method according to one or more embodiments.

[0064] FIG. 15b is a drawing for explaining a single-focus projection method according to one or more embodiments.

[0065] FIG. 16 is a flowchart illustrating an operation of a mobile projector according to one or more embodiments to project an image by taking into account the direction in which a user's face is recognized.

[0066] FIG. 17A is a diagram illustrating a mobile projector according to one or more embodiments projecting an image from the opposite direction of a user's face.

[0067] FIG. 17b is a diagram illustrating a mobile projector according to one or more embodiments projecting an image in a direction in which a user's face is recognized.

[0068] FIG. 18 is a flowchart illustrating an operation of a mobile projector according to one or more embodiments to project an image based on an obstacle detection result.

[0069] FIG. 19A is a diagram illustrating an operation of a mobile projector according to one or more embodiments to change a projection angle for long-focus projection.

[0070] FIG. 19b is a diagram illustrating an operation of a mobile projector according to one or more embodiments to change a projection angle for short-focus projection.

[0071] FIG. 20 is a flowchart illustrating an operation of a mobile projector according to one or more embodiments to perform a projection operation based on a user's face or gaze.

[0072] FIG. 21 is a drawing for explaining an operation of a mobile projector according to one or more embodiments to determine a projection angle.

[0073] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0074] One or more embodiments of the present disclosure may provide a mobile projector and a method of operating a mobile projector.

[0075] One or more embodiments of the present disclosure may provide a mobile projector usable as an extension monitor (or extension display) and a method of operating the mobile projector.

[0076] One or more embodiments of the present disclosure may provide a mobile projector capable of configuring a source device and a multi-view, and a method of operating the mobile projector.

[0077] One or more embodiments of the present disclosure may provide a mobile projector and a method of operating the mobile projector that projects an image taking into account the location of a user or a location of a source device.

[0078] One or more embodiments of the present disclosure may provide a mobile projector and a method of operating the mobile projector for projecting an image at a location corresponding to a user's line of sight.

[0079] One or more embodiments of the present disclosure may provide a mobile projector and a method of operating the mobile projector that determines a projection position or projection method based on a change in the situation of the mobile projector, information about the surrounding environment, or information about the user.

[0080] One or more embodiments of the present disclosure may provide a mobile projector and a method of operating the mobile projector that determines a projection position or projection method based on a user's face position or gesture.

[0081] One or more embodiments of the present disclosure may provide a mobile projector that projects an image while avoiding obstacles and a method of operating the mobile projector.

[0082] One or more embodiments of the present disclosure may provide a mobile projector and a method of operating the mobile projector to determine a projection position or projection method to provide content to a user more efficiently.

[0083] FIG. 1A is a diagram illustrating a mobile projector according to one or more embodiments.

[0084] Referring to FIG. 1A, a mobile projector (100) may include a main body (101), a drive assembly (112), a projection unit (130), and a camera assembly (150). According to one or more embodiments, the mobile projector (100) may correspond to a mobile robot including a projector function.

[0085] The main body (101) can form the exterior of the portable projector (100). For example, the main body (101) can form a cylindrical or spherical exterior, but is not limited thereto and can also form an exterior of another shape, such as a hexahedron. The main body (101) can be configured to be rotatable. For example, the main body (101) can be rotated by a motor and can be rotated based on a preset angle or direction. The motors can be provided on each side of the main body (101).

[0086] The driving unit (112) may include at least one wheel used to move the main body (101) and at least one wheel motor configured to rotate the at least one wheel. For example, the driving unit (112) may include a first wheel (102) and a second wheel (103) installed on both lower sides of the main body (101), and two wheel motors that respectively drive the rotation of the first wheel (102) and the second wheel (103). The first wheel (102) and the second wheel (103) may have corresponding configurations and may be installed in a symmetrical form with respect to the main body (101).

[0087] The driving unit (112) may further include at least one auxiliary wheel to assist the first wheel (102) and the second wheel (103). The at least one auxiliary wheel may be installed at the lower front or rear of the main body (101), but is not limited thereto.

[0088] The projection unit (130) may be configured to be included in the front of the main body (101) and project an image. The projection unit (130) may also be referred to as a projection module, a projection module, or a projector. The projection unit (130) may project an image onto a first area (e.g., a wall or a screen) or a second area (e.g., a floor) based on the rotation angle or rotation direction of the main body (101).

[0089] The camera assembly (150) may include at least one camera included in the front of the main body (101). For example, the at least one camera may include, but is not limited to, an RGB camera (151) and / or a depth camera (152) positioned adjacent to the projection unit (130).

[0090] The detailed configuration of the mobile projector (100) illustrated in FIG. 1a below is described with reference to FIG. 1b.

[0091] FIG. 1b is a block diagram of a mobile projector according to one or more embodiments.

[0092] Referring to FIG. 1B, the mobile projector (100) may include a driver (111), a driving unit (112), a transceiver (120), a projector (130), a sensor assembly (140), a camera unit (150), a memory (160), a user interface (UI) (170), a power supply unit (180), and a processor (190). According to one or more embodiments, the mobile projector (100) may include additional components (e.g., a display) other than the illustrated components, or may omit at least one of the illustrated components.

[0093] The driving unit (111) may include at least one motor for rotating the main body (101). The at least one motor may be driven based on a control signal provided by the processor (190). The at least one motor may be driven to control the rotation angle or direction of the main body (101).

[0094] The driving unit (112) may include at least one wheel (e.g., the first wheel (102) and the second wheel (103) of FIG. 1) used for movement or driving of the mobile projector (100) and at least one wheel motor. At least one wheel motor may be connected to at least one wheel and may be driven to rotate at least one wheel.

[0095] The communication unit (120) can support the establishment of a wired communication channel or a wireless communication channel between the mobile projector (100) and a source device or an external electronic device (e.g., a home appliance, a mobile terminal, or a server), and the performance of communication through the established communication channel.

[0096] According to one or more embodiments, the communication unit (120) may include a wired communication module and / or a wireless communication module. For example, the wired communication module may include a Local Area Network (LAN) communication module or a power line communication module, and the wireless communication module may include a short-range communication module and / or a mobile communication module.

[0097] The short-range communication module may include at least one of a Near Field Communication (NFC) communication module, a Radio-Frequency Identification (RFID) communication module, a Zigbee communication module, a Wireless Fidelity (Wi-Fi) communication module, a Wi-Fi Direct communication module, a Bluetooth communication module, a Bluetooth Low Energy (BLE) communication module, a Wireless Local Area Network (WLAN) communication module, an Ant+ communication module, an Ultra Wideband (UWB) communication module, an Infrared Data Association (IrDA) communication module, or a Microwave (uWave) communication module.

[0098] The mobile communication module may include at least one communication module capable of communicating with an external electronic device via a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a long-distance communication network such as a LAN or a Wide Area Network (WAN)).

[0099] The communication unit (120) may include an antenna module used to transmit or receive signals or power to or from an external source (e.g., a source device or an external electronic device), or may be connected to an antenna module. The antenna module may include one or more antennas (e.g., an array antenna).

[0100] The projection unit (130) may be configured to project an image. According to one or more embodiments, the image may include at least one of a still image, a moving image, a real-time streaming image, a broadcast image, a simulation image, an interactive image, or an animated image.

[0101] The projection unit (130) may include at least two projection lenses. For example, the at least two projection lenses may include a first projection lens (131) and a second projection lens (132). The first projection lens (131) and the second projection lens (132) may project images based on different focal lengths or different projection distances. The first projection lens (131) and the second projection lens (132) may project images based on different projection directions or different projection angles.

[0102] According to one or more embodiments, the first projection lens (131) may represent a projection lens having a shorter focal length (or projection distance) than the second projection lens (132). The second projection lens (132) may represent a projection lens having a longer focal length (or projection distance) than the first projection lens (131). The first projection lens (131) may be referred to as a short-focus projection lens, and the second projection lens (132) may be referred to as a long-focus projection lens.

[0103] According to one or more embodiments, an image projected through the first projection lens (131) may be projected on a first area (e.g., a floor) that is a first distance away from the mobile projector (100) in a first direction. An image projected through the second projection lens (132) may be projected on a second area (e.g., a wall) that is a second distance away from the mobile projector (100) in a second direction. The first distance may be shorter than the second distance, and the first direction and the second direction may be different. The first direction and / or the second direction may be determined based on a rotation angle of the main body (101) of the mobile projector (100).

[0104] The first projection lens (131) or the second projection lens (132) can be selectively used for the projection operation. Either the first projection lens (131) or the second projection lens (132) can be selected based on the rotation angle of the main body (101). For example, either the first projection lens (131) or the second projection lens (132) can be selected for the projection operation based on an angle rotated clockwise (e.g., 90 degrees) from a basic angle (or initial setting angle) of the main body (101) (e.g., 0 degrees on an axis parallel to the floor), or an angle rotated counterclockwise (e.g., -90 degrees) from the basic angle.

[0105] The sensor assembly (140) may include a plurality of sensors configured to detect information about the surrounding environment of the mobile projector (100). For example, the sensor assembly (140) may include an inertial sensor (141) and / or a distance sensor (142).

[0106] The inertial sensor (141) can measure the motion state of the mobile projector (100). For example, the inertial sensor (141) may be an IMU (Inertial Measurement Unit) sensor including an accelerometer that measures the acceleration of the mobile projector (100) and a gyroscope that measures the angular velocity of the mobile projector (100).

[0107] According to one or more embodiments, when the mobile projector (100) rises or falls, a change in height (or altitude) of the mobile projector (100) can be detected based on the acceleration measured by the accelerometer in the rising or falling direction.

[0108] According to one or more embodiments, the direction and speed of the height change of the mobile projector (100) can be estimated based on the rotation speed of the mobile projector (100) measured by the gyroscope.

[0109] Measurement results from an accelerometer and a gyroscope can be combined to more accurately detect changes in the height of the mobile projector (100).

[0110] The distance sensor (142) may be a sensor that measures the distance to an object (e.g., a person, an object, an animal, a structure, or an obstacle). For example, the distance sensor (142) may include at least one of an ultrasonic sensor that measures the distance to an object by measuring the reflection time using an ultrasonic signal, an infrared sensor that measures the distance to an object by detecting an infrared signal reflected from the object, and a ToF (Time-of-Flight) sensor that measures the distance to an object using light or a laser pulse.

[0111] The sensor assembly (140) may further include a sensor capable of measuring the position and distance of an object, such as a LiDAR (Laser imaging Detection And Ranging) sensor or a RADAR (RAdio Detection And Ranging) sensor.

[0112] The camera assembly (150) may include at least one camera. The at least one camera may capture still or moving images, or function as an image sensor. For example, the camera assembly (150) may be used to detect the surrounding environment of the mobile projector (100), detect a user, identify a user's face, pupil (iris), or gestures, or recognize an object.

[0113] According to one or more embodiments, the camera assembly (150) may include an RGB camera (151), a depth camera (152), or a wheel camera (153).

[0114] An RGB camera (151) can be used to analyze and compare colors. The RGB camera (151) can include a sensor that recognizes the colors red, green, or blue.

[0115] The depth camera (152) can measure the distance between the movable projector (100) and an object in a three-dimensional space. The depth camera (152) can measure the distance to an object based on the ToF method, and can be used to estimate the motion of an object or recognize a user's gesture.

[0116] The wheel camera (153) may refer to a camera that can be mounted on at least one wheel included in the driving unit (112). The wheel camera (153) may be used to detect lines or patterns on the ground to estimate the position or direction of the mobile projector (100), or to identify the driving path of the mobile projector (100).

[0117] The memory (160) can store various data that can be used to control the operation of each component of the mobile projector (100). The memory (160) can store, for example, a plurality of application programs used in the mobile projector (100), data for controlling the operation of the mobile projector (100), and commands. At least some of the application programs stored in the memory (160) can be downloaded from an external source (e.g., a server) via wireless communication. At least some of the application programs stored in the memory (160) can be stored in the memory (160) from the time of shipment for the basic functions of the mobile projector (100).

[0118] The memory (160) can store at least one program for processing and controlling the processor (190), and can store input and / or output data. The memory (160) can also store an artificial intelligence (AI) model. The memory (160) can include at least one of a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., Secure Digital (SD) or eXtreme Digital (XD) memory), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read-Only Memory (ROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Programmable Read-Only Memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. According to one or more embodiments, a web storage or a cloud server that performs a storage function on the Internet may be operated by the mobile projector (100).

[0119] The UI (170) may include various types of input devices for receiving commands or data to be used in components of the mobile projector (100) (e.g., processor (190)) from an external source (e.g., a user) of the mobile projector (100). For example, the UI (170) may include at least one of a microphone, a microphone array, a key pad, a touch pad, a trackball, a jog switch, or a touch screen that receives touch input and displays a graphical user interface (GUI).

[0120] The UI (170) may include various types of output devices for displaying or outputting various information related to the operation of the mobile projector (100). For example, the UI (170) may include a display for visually displaying information, an audio output device for outputting information as a voice or audio signal (e.g., a warning sound, an alert sound, or music), or a vibration mechanism for generating mechanical vibration for notification or warning.

[0121] The power supply unit (180) may include a device or circuit that supplies power to the mobile projector (100). The power supply unit (180) may include an interface that can receive power from an external power supply unit and a battery that stores the power received from the external power supply unit. Instead of the interface and the battery, the power supply unit (180) may also include a rechargeable battery charged with power for the operation of the mobile projector (100).

[0122] The processor (190) can control the overall operation of the mobile projector (100) and perform the operations of the mobile projector (100) to be presented below. The processor (190) can execute calculations or data processing related to control and / or communication of at least one other component of the mobile projector (100). For example, the processor (190) is electrically connected to the driving unit (111), the driving unit (112), the communication unit (120), the projection unit (130), the sensor assembly (140), the camera assembly (150), the memory (160), the UI (170), and the power unit (180), and can execute instructions of a program stored in the memory (160). The processor (190) can include a processing circuit that executes instructions of a program stored in the memory (160). The processor (190) may include at least one of a CPU (Central Processing Unit), an NPU (Neural Processing Unit), a GPU (Graphics Processing Unit), an MPU (Micro Processing Unit), an MCU (Micro Controller Unit), an AP (Application Processor), a CP (Communication Processor), a SoC (System on Chip), or an IC (Integrated Circuit) sensor hub, a Supplementary Processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array), and may have multiple cores.

[0123] FIG. 2 is a diagram illustrating a mobile projector and a source device according to one or more embodiments.

[0124] Referring to FIG. 2, the source device (200) may be an external electronic device capable of providing a video source or video information (e.g., content information and / or video frames). The source device (200) may be, for example, any one of a mobile terminal such as a smartphone or tablet, a wearable device, a laptop, a personal computer (PC), or a home appliance including a display such as a television (TV).

[0125] The mobile projector (100) may be set as an extension monitor (or extension display) as a sink device corresponding to the source device (200). The settings of the extension display device may be settings for expanding the display area of ​​the source device (200). When the mobile projector (100) is set as an extension display device, the mobile projector (100) may receive an image from the source device (200) and perform a display operation or a projection operation. When the mobile projector (100) is set as an extension display device, the mobile projector (100) may be connected to the source device (200) to provide a wider display space. Depending on the settings of the extension display device, the display space of the image may move away from the display of the source device (200) to another space (e.g., the projection space of the mobile projector (100). For example, an image displayed on a source device (200) can be transmitted to a mobile projector (100) and projected, and an image projected by the mobile projector (100) can be displayed again on the source device (200).

[0126] The mobile projector (100) and the source device (200) can be wirelessly connected based on wireless communication within a local network without using a cable (e.g., HDMI (High-Definition Multimedia Interface) cable), adapter, or Display Port.

[0127] Wireless connections can be implemented for wireless display. Wireless display can refer to a display based on a wireless connection between at least two devices. For example, wireless display can include a display associated with Wi-Fi display, screen mirroring, or screen sharing. Wireless display can be implemented based on various standards, such as Miracast, Wi-Fi Direct, Chromecast, Intel WiDi (Wireless Display), or AirPlay.

[0128] Wireless connections can be peer-to-peer (P2P) and may not require additional infrastructure such as servers, routers, or access points.

[0129] According to one or more embodiments, the mobile projector (100) may obtain image information from the source device (200) based on being set as an extended display device. For example, the image information may include information about content that is being displayed or will be displayed on the source device (200), or configuration information of audio and / or video frames, text, or graphics associated with the content. According to one or more embodiments, the mobile projector (100) may perform a projection operation based on the image information obtained from the source device (200). For example, the mobile projector (100) may project an image that is the same as or different from an image displayed on the source device (200) based on the image information obtained from the source device (200).

[0130] According to one or more embodiments, the mobile projector (100) may configure a multi-view with the source device (200) based on being set as an extended display device. The operation of the mobile projector (100) for multi-view may be included in the operation of the extended display device.

[0131] A multiview may include two or more views. The two or more views may include screens associated with two or more sources (or multiview sources) (e.g., images). The two or more sources may be related to images provided from at least one of an application installed on the source device (200), a broadcast channel available on the source device (200), or an external electronic device (e.g., a mobile terminal or a USB camera) connected to the source device (200). The two or more sources may be selected by a user.

[0132] If the mobile projector (100) is not set as an extended display device, two or more views may be displayed on the source device (200). For example, if the mobile projector (100) is not set as an extended display device and the multi-view includes a first view and a second view, the first view and the second view may be displayed on the source device (200).

[0133] When the mobile projector (100) is set as an extended display device, at least one of the two or more views can be displayed on the source device (200), and at least one other of the two or more views can be projected (or displayed) by the mobile projector (100). For example, when the mobile projector (100) is set as an extended display device and the multi-view includes a first view and a second view, the first view can be displayed on the source device (200), and the second view can be projected by the mobile projector (100).

[0134] According to one or more embodiments, the first view may include a screen of a broadcast program received by the source device (200), and the second view may include a screen of an application executed on the source device (200). When the mobile projector (100) is set as an extended display device, a user may use the application executed on the source device (200) while watching the broadcast program based on the first view displayed on the source device (200) and the second view projected by the mobile projector (100).

[0135] According to one or more embodiments, the first view may include a screen of a broadcast program received by the source device (200), and the second view may include a screen of an application executed on the mobile terminal. When the mobile projector (100) is set as an extended display device, the user may use various functions (e.g., executing an application, making a voice or video call, viewing a captured photo or video, or a search or translation function) via the mobile terminal while watching the broadcast program based on the first view displayed on the source device (200) and the second view projected by the mobile projector (100).

[0136] FIG. 3 is a flowchart illustrating the operation of a mobile projector according to one or more embodiments.

[0137] Referring to FIG. 3, in operation 302, the mobile projector (100) may determine whether to set the mobile projector (100) as an extended display device associated with the source device (200). According to an example, the mobile projector (100) may include operation state information indicating an operation state of the mobile projector (100). The operation state information may be updated based on a change in the operation state of the mobile projector (100). The operation state information may include, for example, any one of information indicating that the mobile projector (100) is moving, information indicating that the mobile projector (100) is stopped (or has stopped moving), information indicating that the mobile projector (100) is projecting, or information indicating that the mobile projector (100) is stopped and not projecting.

[0138] The mobile projector (100) can receive a request message requesting the setting of an extended display device from the source device (200), and determine whether to set the mobile projector (100) as an extended display device of the source device (200) based on the operating status information.

[0139] According to one or more embodiments, the mobile projector (100) may determine to be set as an extended display device if the operating state information includes any one of information indicating that the mobile projector (100) is moving, information indicating that the mobile projector (100) is stopped (or has stopped moving), or information indicating that the mobile projector (100) is stopped and not projecting.

[0140] According to one or more embodiments, the mobile projector (100) may determine not to set itself as an extended display device if the operational status information includes information indicating that it is projecting.

[0141] In operation 304, the mobile projector (100) may receive information about an image to be projected from the source device (200) based on the mobile projector (100) being set as an extended display device of the source device (200). According to one or more embodiments, the information about the image to be projected may include information about at least one of an image received in real time by the source device (e.g., a broadcast image or a real-time streaming image), an image associated with an application executed in the source device (200), an image associated with a camera (e.g., a USB camera) connected to the source device (200), or an image provided from a mobile terminal connected to the source device (200).

[0142] In operation 306, the mobile projector (100) can determine a projection position based on the location information of the source device (200). According to one or more embodiments, the projection position may indicate a location or position of the mobile projector (100) for projecting an image based on the acquired information, a location or position of the mobile projector (100) for determining a projection area on which to project an image, or a location adjacent to the source device (200). According to one or more embodiments, the location information of the source device (200) can be identified from a map (or map view) stored in the mobile projector (100). Once the projection position is determined, the mobile projector (100) can move to the determined projection position.

[0143] In operation 308, the mobile projector (100) can project an image based on the received information while at the determined projection position. According to one or more embodiments, the mobile projector (100) can determine a projection area while at the determined projection position. The mobile projector (100) can identify the source device (200) while at the projection position and scan the space adjacent to the source device (200) using at least one camera. The mobile projector (100) can determine the projection area based on the scan result. For example, the mobile projector (100) can determine an area in the space adjacent to the source device (200) free of users or obstacles as the projection area and project an image on the determined projection area. In other words, the mobile projector (100) can identify a projection position to which it must move in order to project the image to be projected. It will be apparent to those skilled in the art from the teachings of this disclosure that the mobile projector (100) may perform various judgments while the mobile projector (100) is at a determined projection position, while moving to the determined projection position, and before moving to the determined projection position, as described herein. Accordingly, the phrase "at a projection position" may indicate that an operation is performed while the mobile projector (100) is at a projection position or while moving to a projection position.

[0144] The mobile projector (100) may not know the location information of the source device (200). In this case, the mobile projector (100) may perform a user discovery (or search) operation without performing operations 306 and 308. For example, the mobile projector (100) may perform an operation to discover a user by driving. User discovery may be performed based on user account verification using a mobile terminal, facial recognition, iris recognition, or voice recognition techniques. When the user is discovered, the mobile projector (100) may determine a projection area at the location where the user is discovered, and project an image into the determined projection area. For example, the mobile projector (100) may determine an area free of obstacles adjacent to the location where the user is discovered, an area facing the user's face, or an area corresponding to the user's line of sight as the projection area, and project an image into the determined projection area.

[0145] The operations illustrated in FIG. 3 are not limited to the illustrated order and may be performed in various orders. For example, the operation of receiving information about the image to be projected in operation 304 may be performed in operation 302 or operation 306. According to one or more embodiments, at least some of the operations illustrated in FIG. 3 may be omitted, or more operations may be performed than those illustrated in FIG. 3.

[0146] FIG. 4A is a signal flow diagram illustrating an operation of a mobile projector accepting an operation request for an extended display device according to one or more embodiments.

[0147] According to one or more embodiments, the operations illustrated in FIG. 4A may be operations associated with operation 302 of FIG. 3.

[0148] Referring to FIG. 4A, in operation 402, the source device (200) may search for a connectable display device. According to one or more embodiments, the source device (200) may search for a connectable display device based on wireless communication (e.g., Wi-Fi or Wi-Fi Direct communication). When one or more display devices are searched by the source device (200), the one or more searched display devices may be displayed in the form of icons on the display of the source device (200). The display of the source device (200) may also display, for example, a name, a MAC (Media Access Control) address, or an identifier (e.g., a serial number or product code) corresponding to the one or more searched display devices.

[0149] In operation 404, the source device (200) may register the mobile projector (100) among one or more searched display devices as an extended display device of the source device (200). According to one or more embodiments, when the mobile projector (100) is selected by an external input (e.g., a user input) among one or more display devices registered to the source device (200), the source device (200) may select the mobile projector (100) as an extended display device of the source device (200). The extended display device may refer to a display that displays an image provided from the source device (200) or a projector that projects an image provided from the source device (200).

[0150] In operation 406, the source device (200) may transmit a connection request message requesting a wireless connection to the mobile projector (100) based on the mobile projector (100) being selected as the extended display device of the source device (200). According to one or more embodiments, the connection request message may include information indicating that the mobile projector (100) has been selected as the extended display device of the source device (200) and / or information requesting the mobile projector (100) to set up the extended display device (or operate the extended display device).

[0151] In operation 408, the mobile projector (100) may receive a connection request message and determine to operate as an extended display device of the source device (200) based on the operating status information. For example, the mobile projector (100) may determine to operate as an extended display device of the source device (200) if the operating status information includes any one of information indicating that the mobile projector (100) is moving, information indicating that the mobile projector (100) is stopped (or has stopped moving), or information indicating that the mobile projector (100) is stopped and not projecting.

[0152] At operation 410, the mobile projector (100) may transmit a connection acceptance message to the source device (200) indicating that it will accept the wireless connection based on its decision to operate as an extended display device of the source device (200).

[0153] In operation 412, the mobile projector (100) can establish a wireless connection with the source device (200) based on transmitting a connection acceptance message to the source device (200).

[0154] The source device (200) may receive a connection acceptance message from the mobile projector (100), and, based on the reception of the connection acceptance message, determine that the mobile projector (100) will be set (or operated) as an extended display device. Accordingly, the source device (200) may establish a wireless connection with the mobile projector (100) in response to the reception of the connection acceptance message.

[0155] Establishing a wireless connection may include, for example, an action of setting up a channel for wireless communication between the mobile projector (100) and the source device (200), an action of exchanging device capability information between the mobile projector (100) and the source device (200), or an action of maintaining a wireless communication session between the mobile projector (100) and the source device (200).

[0156] According to one or more embodiments, when a wireless connection is established between the mobile projector (100) and the source device (200), the operations illustrated in FIG. 4b or FIG. 4c may be performed.

[0157] FIG. 4b is a signal flow diagram of a mobile projector and a source device for operation of an extended display device according to one or more embodiments.

[0158] Referring to FIG. 4B, at operation 414, the source device (200) may activate a mobile projector icon based on completion of wireless connection setup between the mobile projector (100) and the source device (200). According to one or more embodiments, the mobile projector icon may be an icon that can be selected to use the mobile projector (100) as an extended display device and may be displayed on the display of the source device (200).

[0159] In one example, the source device (200) may visually and / or functionally activate (e.g., increase saturation, enhance brightness, add highlighting, or make selectable) the mobile projector icon to activate the mobile projector icon. A visually and / or functionally activated mobile projector icon may indicate that a function associated with the mobile projector (100) is available for use (e.g., an extended display device function).

[0160] At action 416, the source device (200) can identify that an activated mobile projector icon is selected.

[0161] In operation 418, the source device (200) may transmit image information to the mobile projector (100) based on the selection of an activated mobile projector icon. The image information transmitted to the mobile projector (100) may correspond to an image selected by the user or an image that the user has designated to be projected through the mobile projector (100).

[0162] A mobile projector (100) can obtain information about an image to be projected by receiving image information from a source device (200).

[0163] In operation 420, the mobile projector (100) can identify the location of the source device (200) based on the received image information and move to the identified location. For example, the mobile projector (100) can identify the location of the source device (200) based on a pre-stored map or location information of the source device (200).

[0164] In operation 422, the mobile projector (100) can identify the source device (200) at the location to which the mobile projector has moved and scan the space adjacent to the source device (200). The mobile projector (100) can perform operation 422 while at the identified location, while moving to the identified location, or before moving to the identified location. According to one or more embodiments, the mobile projector (100) can scan the space adjacent to the source device (200) using at least one camera included in the camera assembly (150).

[0165] In operation 424, the mobile projector (100) can determine a projection area based on the scan results and project an image on the determined projection area. For example, the mobile projector (100) can determine an area in a space adjacent to the source device (200) where there are no obstacles or users as a projection area and project an image on the determined projection area.

[0166] FIG. 4c is another signal flow diagram of a mobile projector and a source device for operation of an extended display device according to one or more embodiments.

[0167] Since operations 434 to 438 of FIG. 4c correspond to operations 414 to 418 of FIG. 4b, a detailed description thereof is omitted.

[0168] Referring to FIG. 4c, in operation 440, the mobile projector (100) may determine that the location of the source device (200) is not identified. For example, if there is no pre-stored map or location information of the source device (200), the mobile projector (100) may determine that the location of the source device (200) is not identified.

[0169] The mobile projector (100) can start moving to find the user if the location of the source device (200) is not identified.

[0170] In operation 442, the mobile projector (100) can discover the user through movement. For example, the mobile projector (100) can discover the user based on facial recognition, iris recognition, or voice recognition techniques.

[0171] In operation 444, the mobile projector (100) can determine a projection area based on the location where the user was found, and project an image onto the determined projection area. For example, the mobile projector (100) can determine an area free of obstacles adjacent to the location where the user was found, an area facing the user's face, or an area corresponding to the user's line of sight as the projection area, and project an image onto the determined projection area. The method for determining the projection area will be described in detail later.

[0172] FIG. 4D is a diagram illustrating a multi-view setup screen that may be displayed on a source device when a mobile projector according to one or more embodiments can operate as an extended display device of the source device.

[0173] According to one or more embodiments, the mobile projector (100) may operate as an extended display device of the source device (200) based on the operations illustrated in FIG. 4a. In this case, the source device (200) may display a multi-view setting screen as illustrated in FIG. 4d on the display.

[0174] Referring to FIG. 4d, the multiview setting screen may be output when a UI (or multiview UI) for starting or executing multiview is selected in the source device (200). The multiview setting screen may include a multiview setting UI (450).

[0175] The multiview setting UI (450) may include a GUI such as an icon, a button, or a window. For example, the multiview setting UI (450) may include a recently used multiview icon (452), a saved multiview icon (454), multiple source icons (456)(458)(460)(462) indicating multiview sources, or a source device icon (464) and a portable projector icon (466) for selecting a multiview output method.

[0176] The recently used multiview icon (452) may indicate configuration information of a recently used multiview. A recently used multiview may include the most recently used multiview, or at least one multiview used within a predetermined time range (e.g., within the last week).

[0177] The configuration information of the recently used multiview may include at least one of information about recently used multiview sources (e.g., information about applications A and B running on the source device (200), information about recently used multiview layout (e.g., information about multiview areas of predetermined sizes arranged left and right), or information about a device (e.g., source device (200)) from which each of the recently used multiview sources is output (displayed or projected).

[0178] When the recently used multi-view icon (452) is selected, the source device (200) can provide the user with a multi-view screen based on the configuration information of the recently used multi-view. For example, screens for applications A and B running on the source device (200) can be displayed in multi-view areas arranged on the left and right of the source device (200).

[0179] The saved multiview icon (454) can indicate configuration information of the saved multiview.

[0180] The configuration information of the saved multi-view may include configuration information of at least one multi-view stored in the source device (200). The configuration information of at least one multi-view stored in the source device (200) may be stored by the user.

[0181] For example, the configuration information of the stored multi-view may include at least one of information about multi-view sources stored in the source device (200) (e.g., information about a broadcast channel used in the source device (200) and information about an application running in the source device (200), multi-view layout information stored in the source device (200) (e.g., information about an area corresponding to the entire screen of the source device (200) and information about a projection area on the right side of the source device (200), or information about a device from which each of the multi-view sources stored in the source device (200) is output (e.g., the source device (200) or a mobile projector (100)).

[0182] When a saved multi-view icon (454) is selected, the source device (200) can provide the user with a multi-view screen based on the configuration information of the saved multi-view. For example, the screen of a broadcast program C received by the source device (200) through a broadcast channel can be displayed in an area corresponding to the entire screen of the source device (200), and the screen of application A can be projected by a mobile projector (100) in a projection area on the right side of the source device (200).

[0183] According to one or more embodiments, at least two of the multiple source icons (456)(458)(460)(462), a source device icon (464) or a portable projector icon (466) may be selected by the user without selecting a recently used multiview icon (452) or a saved multiview icon (454).

[0184] Multiple source icons (456)(458)(460)(462) can indicate different types of sources that can be used as multi-view sources.

[0185] For example, the plurality of source icons (456)(458)(460)(462) may include at least one application icon (456) running on the source device (200) (e.g., application A and / or B icons), a broadcast channel icon (458) indicating a broadcast channel that can be received or used on the source device (200), a mobile terminal icon (460) indicating a mobile terminal connected to the source device (200), or a camera icon (462) indicating a camera connected to the source device (200). According to one or more embodiments, if there is no mobile terminal or camera connected to the source device (200), the mobile terminal icon (460) or the camera icon (462) may be deactivated or not displayed.

[0186] The source device icon (464) and the mobile projector icon (466) can be used to select a multi-view output method. The source device icon (464) can be selected to display the multi-view based on the source device (200), and the mobile projector icon (466) can be selected to display the multi-view based on the source device (200) and the mobile projector (100).

[0187] According to one or more embodiments, the mobile projector icon (466) may be selectively displayed. For example, the mobile projector icon (466) may be displayed when the mobile projector (100) is registered as an extended display device of the source device (200), or may be displayed when the mobile projector (100) is in an operational state in which it can be used as an extended display device. The operational state in which the mobile projector (100) can be used as an extended display device may include a state in which the mobile projector (100) registered as an extended display device is not performing a projection operation, a state in which the mobile projector (100) is in a moving state, or a state in which the mobile projector is not performing a projection operation. FIG. 5A is a signal flow diagram illustrating an operation in which the mobile projector rejects an operation request for an extended display device according to one or more embodiments.

[0188] Referring to FIG. 5A, in operation 502, the source device (200) may search for connectable devices. According to one or more embodiments, the source device (200) may search for connectable display devices based on wireless communication (e.g., Wi-Fi or Wi-Fi Direct communication). When one or more display devices are searched by the source device (200), the one or more searched display devices may be displayed in the form of icons on the display of the source device (200). The display of the source device (200) may also display, for example, a name, a MAC address, or an identifier (e.g., a serial number or product code) corresponding to the one or more searched display devices.

[0189] In operation 504, the source device (200) may register a mobile projector (100) among one or more searched display devices as an extended display device of the source device (200). According to one or more embodiments, when the mobile projector (100) is selected by an external input (e.g., a user input) among one or more searched display devices registered to the source device (200), the source device (200) may select the mobile projector (100) as an extended display device of the source device (200).

[0190] In operation 506, the source device (200) may transmit a connection request message requesting a wireless connection to the mobile projector (100) based on the mobile projector (100) being selected as the extended display device. According to one or more embodiments, the connection request message may include information indicating that the mobile projector (100) has been selected as the extended display device of the source device (200) and / or information requesting the mobile projector (100) to set up (or operate) the extended display device.

[0191] In operation 508, the mobile projector (100) may, in response to the connection request message, receive the connection request message and determine not to operate as an extended display device of the source device (200) based on the operating state information. For example, the mobile projector (100) may determine not to operate as an extended display device of the source device (200) if the operating state information includes information indicating that the mobile projector (100) is projecting.

[0192] In operation 510, the mobile projector (100) may transmit a connection rejection message to the source device (200) indicating that the wireless connection is to be rejected based on the mobile projector (100) determining that it will not operate as an extended display device of the source device (200). According to one or more embodiments, the connection rejection message may include information indicating that the mobile projector (100) is not set up as an extended display device or will not perform the operation of an extended display device and / or information indicating that the mobile projector (100) rejects a setup request (or operation request) of the extended display device.

[0193] The source device (200) may determine that the mobile projector (100) will not be set up (or operated) as an extended display device based on receiving a connection rejection message from the mobile projector (100).

[0194] In operation 512, the source device (200) may, in response to receiving a connection rejection message, disable the mobile projector icon. For example, the source device (200) may visually and / or functionally disable the mobile projector icon (e.g., by reducing saturation, reducing brightness, removing highlighting, or rendering it unselectable) to indicate that the mobile projector (100) cannot be used or selected as an extended display device.

[0195] In operation 514, the source device (220) may reselect an extended display device from one or more searched display devices or terminate the operation.

[0196] FIG. 5b is a diagram illustrating a multi-view setting screen that may be displayed on a source device when a mobile projector according to one or more embodiments cannot operate as an extended display device of the source device.

[0197] According to one or more embodiments, the mobile projector (100) cannot operate as an extended display device of the source device (200) based on operations 502 to 510 of FIG. 5A. In this case, the source device (200) can display a multi-view setting screen as illustrated in FIG. 5B on the display.

[0198] Referring to FIG. 5B, the multi-view settings screen may include a multi-view settings UI (550). The multi-view settings UI (550) may include a disabled mobile projector icon (566). The disabled mobile projector icon (566) may indicate that the mobile projector (100) cannot be used or selected as an extended display device. The disabled mobile projector icon (566) may be an icon that is visually and / or functionally disabled (e.g., desaturated, deblurred, de-highlighted, or rendered unselectable).

[0199] The recently used multi-view icon (552), the saved multi-view icon (554), the multiple source icons (556)(558)(560)(562), or the source device icon (564) included in the multi-view setting UI (550) correspond to the recently used multi-view icon (452), the saved multi-view icon (454), the multiple source icons (456)(458)(460)(462) indicating multi-view sources, or the source device icon (464) for selecting a multi-view output method included in the multi-view setting UI (450) of FIG. 4d, and thus a detailed description thereof will be omitted.

[0200] FIG. 6A is a diagram illustrating a multiview displayed on a source device according to one or more embodiments.

[0201] Referring to FIG. 6A, a recently used multiview icon (552) or a saved multiview icon (554) can be selected in the multiview setting UI (550) illustrated in FIG. 5D. When a recently used multiview icon (552) or a saved multiview icon (554) is selected, multiview configuration information (e.g., multiview sources, multiview layout, or output device for each multiview source) is predetermined, so faster multiview provision may be possible.

[0202] For example, the source device (200) can identify, based on predetermined multi-view configuration information, that the multi-view sources include broadcast programs received through a broadcast channel from the source device (200) and applications executed in the source device (200), that the multi-view layout indicates that two areas of the same size are arranged side by side on the left and right, and that the output device of the multi-view sources is the source device (200). Based on the identification result, the source device (200) can output a screen as illustrated in FIG. 6A.

[0203] According to one or more embodiments, a broadcast program screen (e.g., a soccer game) may be displayed on a first area (610) of the display of the source device (200), and a screen of an application (e.g., a game application) running on the source device (200) may be displayed on a second area (620) of the display of the source device (200). Based on this, a user may play a game while watching a soccer game through the source device (200).

[0204] FIG. 6b is a diagram illustrating a multi-view of a source device displaying a mobile projector icon according to one or more embodiments.

[0205] According to one or more embodiments, the mobile projector (100) may be set as an extended display device of the source device (200) before or while the screen illustrated in FIG. 6A is displayed on the source device (200). In this case, the source device (200) may display a mobile projector icon to indicate that the mobile projector (100) is available as an extended display device.

[0206] According to one example, the source device (200) may display a multiview based on predetermined multiview configuration information, and may display a first movable projector icon (612) and a second movable projector icon (622) in a first area (610) and a second area (620) of the display, respectively. The first movable projector icon (612) and the second movable projector icon (622) may be displayed at a predetermined location (e.g., a lower left corner of the area). The first movable projector icon (612) and the second movable projector icon (622) may be displayed visually highlighted.

[0207] The first mobile projector icon (612) or the second mobile projector icon (622) can be selected to change the image output device to the mobile projector (100). When the first mobile projector icon (612) or the second mobile projector icon (622) is selected, the image displayed in the area including the selected icon can be transmitted to the mobile projector (100). For example, when the second mobile projector icon (622) is selected, image information for a game application displayed in the second area (620) can be transmitted to the mobile projector (100). The mobile projector (100) can receive image information from the source device (200) and project the game application based on the received image information.

[0208] FIG. 6c is a diagram illustrating a multiview based on selection of a movable projector icon according to one or more embodiments.

[0209] Based on the selection of the second mobile projector icon (622) in FIG. 6b, a multi-view based on the source device (200) and the mobile projector (100) can be provided.

[0210] Referring to FIG. 6c, the source device (200) can transmit image information (e.g., image information for a game application displayed in the second area (620) of FIG. 6b) to the mobile projector (100) based on the selection of the second mobile projector icon (622).

[0211] The source device (200) can reconfigure the screen based on the transmitted image information. For example, the source device (200) can release the first area (610) and the second area (620) (i.e., initiate a process for changing the way the first area (610) and the second area (620) are provided) and create a third area (630) of a predetermined size (e.g., a size corresponding to the entire screen of the display) as an area for multi-view. The source device (200) can display the broadcast program screen that was displayed in the first area (610) in the third area (630).

[0212] The mobile projector (100) can receive image information about a game application running on the source device (200). The mobile projector (100) can perform a projection operation based on the received image information. For example, the mobile projector (100) can identify the location of the source device (200) and move to the identified location. The mobile projector (100) can identify the source device (200) at the location to which the mobile projector (100) has moved and project an image to an area adjacent to the source device (200) (e.g., an area to the left of the source device (200) where no obstacles or users are present) (640). If the mobile projector (100) fails to identify the location of the source device (200), the mobile projector (100) can also project an image to a location adjacent to the user based on a user search operation through movement.

[0213] According to one or more embodiments, the source device (200) may utilize the video of a mobile terminal as a multi-view source. The operation of the source device (200) and the mobile terminal in this regard will be described with reference to FIG. 7a.

[0214] FIG. 7A is a signal flow diagram illustrating operations of a source device and a mobile terminal for displaying a multiview associated with a mobile terminal according to one or more embodiments.

[0215] Referring to FIG. 7A, in operation 702, the source device (200) may establish a wireless connection with the mobile terminal (300). According to one or more embodiments, the source device (200) may establish a wireless connection with the mobile terminal (300) based on a request from the mobile terminal (300). For example, the source device (200) may establish a wireless connection based on Wi-Fi communication with the mobile terminal (300).

[0216] In operation 704, the source device (200) can receive a mirroring request message for screen mirroring from the mobile terminal (300).

[0217] In operation 706, the source device (200) may transmit a mirroring response message to the mobile terminal (300) in response to the mirroring request message, accepting mirroring of the screen of the mobile terminal (300) on the source device (200). According to one or more embodiments, the source device (200) may transmit the mirroring response message to the mobile terminal (300) based on a user's selection.

[0218] In operation 708, the source device (200) may receive image information from the mobile terminal (300) in response to transmitting the mirroring response message. According to one or more embodiments, the source device (200) may receive image information from the mobile terminal (300) in real time while the wireless connection is maintained. The image information may include information about a screen being displayed on the mobile terminal (300). The image information may also be stored in a buffer (e.g., a graphics buffer) of the source device (200).

[0219] In operation 710, the source device (200) can identify that a mobile terminal icon (e.g., a mobile terminal icon (460) of FIG. 4d or a mobile terminal icon (560) of FIG. 5b) is selected for multi-view source selection, and a source device icon (e.g., a source device icon (464) of FIG. 4d or a source device icon (564) of FIG. 5b) is selected to indicate a multi-view output method.

[0220] In operation 712, the source device (200) may output an image to a predetermined area within the display of the source device (200) based on the selection of the mobile terminal icon and the source device icon. For example, the source device (200) may output a screen as illustrated in FIG. 7b to the display.

[0221] FIG. 7b is a diagram illustrating a multi-view of a source device including an image of a mobile terminal according to one or more embodiments.

[0222] Referring to FIG. 7B, the source device (200) may display a multi-view on the display of the source device (200) based on the selection of the source device icon. The multi-view may be displayed based on multiple areas (e.g., a first area (730) and a second area (740)) within the display of the source device (200).

[0223] For example, if the multi-view sources include streaming video received in real time from the source device (200) and video from a mobile terminal (e.g., the mobile terminal (300) of FIG. 7A), the source device (200) may display the streaming video in a first area (730) and the video from the mobile terminal in a second area (740). The area where each video is displayed may be predetermined or designated by the user. The area where each video is displayed may also be changed according to the user's selection after the multi-view display.

[0224] FIG. 8A is a signal flow diagram illustrating the operation of a mobile terminal, a source device, or a mobile projector to display a multiview associated with a mobile terminal according to one or more embodiments.

[0225] Since operations 802 to 808 of FIG. 8a correspond to operations 702 to 708 of FIG. 7a, a detailed description thereof will be omitted.

[0226] Referring to FIG. 8A, in operation 810, the source device (200) can identify that a mobile terminal icon (e.g., a mobile terminal icon (460) of FIG. 4D) is selected for multi-view source selection and a mobile projector icon (e.g., a mobile projector icon (466) of FIG. 4D) is selected to indicate a multi-view output method.

[0227] In operation 812, the source device (200) can transmit image information received from the mobile terminal (300) to the mobile projector (100) based on the selection of the mobile terminal icon and the mobile projector icon.

[0228] In operation 814, the mobile projector (100) can identify the location of the source device and move to the identified location. For example, the mobile projector (100) can identify the location of the source device (200) based on a pre-stored map or location information of the source device (200).

[0229] In operation 816, the mobile projector (100) can identify the source device at the location to which the mobile projector (100) has moved and scan the space adjacent to the source device. According to one or more embodiments, the mobile projector (100) can scan the space adjacent to the source device (200) using at least one camera included in the camera assembly (150).

[0230] In operation 818, the mobile projector (100) can determine a projection area based on the scan results and project an image (an image of the mobile terminal (300) based on the received image information) on the determined projection area. For example, the mobile projector (100) can determine an area in a space adjacent to the source device (200) where no obstacles or users exist as a projection area and project an image on the determined projection area.

[0231] FIG. 8B is a signal flow diagram illustrating another operation of a mobile terminal, a source device, or a mobile projector for displaying a multiview associated with a mobile terminal according to one or more embodiments.

[0232] Since operations 822 to 830 of FIG. 8b correspond to operations 802 to 810 of FIG. 8a, a detailed description thereof will be omitted.

[0233] Referring to FIG. 8B, in operation 834, the mobile projector (100) may determine that the location of the source device (200) is not identified. For example, if there is no pre-stored map or location information of the source device (200), the mobile projector (100) may not be able to identify the location of the source device (200), and thus may determine that the location of the source device (200) is not identified.

[0234] The mobile projector (100) can start moving to find the user if the location of the source device (200) is not identified.

[0235] In operation 836, the mobile projector (100) can discover the user through movement. For example, the mobile projector (100) can discover the user based on facial recognition, iris recognition, or voice recognition techniques.

[0236] In operation 838, the mobile projector (100) can determine a projection area based on the location where the user was found, and project an image (an image of the mobile terminal (300) based on the received image information) on the determined projection area. For example, the mobile projector (100) can determine an area free of obstacles adjacent to the location where the user was found, an area facing the user's face, or an area corresponding to the user's line of sight as the projection area, and project an image on the determined projection area.

[0237] FIG. 8c is a diagram illustrating an example in which an image of a mobile terminal constituting a multi-view according to one or more embodiments is projected onto an area adjacent to a source device.

[0238] Referring to FIG. 8c, based on the operations illustrated in FIG. 8a, the mobile projector (100) can project an image of a mobile terminal (e.g., a mirrored screen of the mobile terminal) onto an area (840) adjacent to the source device (200). The image of the mobile terminal can form a multi-view together with an image displayed on a display area (830) of the source device (200) (e.g., a real-time streaming image of the source device (200).

[0239] FIG. 8d is a diagram illustrating an image of a mobile terminal configuring a multi-view according to one or more embodiments being projected onto an area adjacent to a user.

[0240] Referring to FIG. 8d, based on the operations illustrated in FIG. 8b, the mobile projector (100) can project an image of the mobile terminal (e.g., a mirrored screen of the mobile terminal) to an area (860) adjacent to the user (850). Since the mobile projector (100) does not know the location of the source device (200), it can perform a search operation for the user (850). The mobile projector (100) determines that the user (850) is located adjacent to the source device (200) and can perform a user search operation. When the user (850) is searched, the mobile projector (100) can project the image of the mobile terminal to an area adjacent to the user (850) (e.g., an area without obstacles adjacent to the user (850), an area facing the user's face, or an area corresponding to the user's line of sight). The video of the mobile terminal can form a multi-view together with the video displayed in the display area (870) of the source device (200) (e.g., real-time streaming video of the source device (200)).

[0241] FIG. 9 is a diagram illustrating projectible areas based on the location of a source device according to one or more embodiments.

[0242] Referring to FIG. 9, the mobile projector (100) can perform projection operations for multi-view at various locations. According to one or more embodiments, when the location of the source device (200) is identified based on a stored map (or when the location of the source device (200) is known), the mobile projector (100) can move to the location of the source device (200) (i.e., the mobile projector (100) can move to a location adjacent to the source device (200). The mobile projector (100) can determine a projection location (or projection area) at the location to which the mobile projector (100) has moved. The mobile projector (100) can identify the source device (200) at the location to which the mobile projector (100) has moved, and use at least one area adjacent to the source device (200) as a projection area for multi-view.

[0243] According to one or more embodiments, the mobile projector (100) may scan the surrounding space of the source device (200) from the position to which the mobile projector (100) has moved, and determine an area within the scanned space where no obstacles or users exist as a projection area. For example, the mobile projector (100) may use any one of the first projectible area (902) located above the source device (200), the second projectible area (904) located on the right side of the source device (200), the third projectible area (906) located below the source device (200), or the fourth projectible area (908) located on the left side of the source device (200) as a projectible area for multi-view. However, this is only one example, and the projectible area may be changed in various ways, such as a diagonal position of the source device (200).

[0244] FIGS. 10A and 10B are diagrams illustrating sources for multi-view according to one or more embodiments.

[0245] Referring to FIGS. 10a and 10b, various sources can be used for multi-view.

[0246] Referring to FIG. 10A, the multi-view source may include an image provided from a mobile terminal (300). The mobile terminal (300) may be wirelessly connected to a source device (200) and may transmit an image to the source device (200) via a wireless connection (operation 1012).

[0247] The source device (200) can transmit the image received from the mobile terminal (300) to the mobile projector (100) so that the image provided from the mobile terminal (300) can be projected through the mobile projector (100) (operation 1014). In this way, the image of the mobile terminal (300) can be transmitted to the mobile projector (100) through the source device (200).

[0248] Referring to FIG. 10b, the multi-view source may include an application (1010) installed (or executed) on the source device (200), or an image associated with a USB camera (1020) connected to the source device (200) via USB.

[0249] The source device (200) can transmit an image associated with the application (1010) or the USB camera (1020) to the mobile projector (100) so that the image associated with the application (1010) or the USB camera (1020) can be projected through the mobile projector (100) (operation 1030). In this way, the source device (200) can transmit an image associated with a device built into or directly (or wired) connected to the source device (200) to the mobile projector (100).

[0250] FIG. 11 is a flowchart illustrating an operation of a mobile projector according to one or more embodiments to determine a projection position.

[0251] According to one or more embodiments, the operations illustrated in FIG. 11 may be operations associated with operation 306 of FIG. 3.

[0252] Referring to FIG. 11, in operation 1102, the mobile projector (100) can determine whether location information of the source device (200) is stored. According to one or more embodiments, the location information of the source device (200) can be identified from a map (or map view) of an indoor space. The map can include a structure of the indoor space (e.g., the arrangement or number of rooms, living rooms, kitchens, bathrooms, windows, verandas, multipurpose rooms, entrances, and exits), or arrangement information or location information for each device. The map can be a two-dimensional (2D) or three-dimensional (3D) map. The map can be generated by the mobile projector (100) based on SLAM (Simultaneous Localization and Mapping) technology, or can be provided from an external source (e.g., a server or an external electronic device).

[0253] The mobile projector (100) can identify the location information (e.g., coordinate information within a home) of the source device (200) from the stored map when a map is stored. Based on this, the mobile projector (100) can determine that the location information of the source device (200) is stored when a map is stored.

[0254] If it is determined that the location information of the source device (200) is stored, the mobile projector (100) can move based on the location information of the source device (200) in operation 1104. For example, the mobile projector (100) can move to a location corresponding to the location information of the source device (200).

[0255] In operation 1106, the mobile projector (100) can determine the identified location to which the mobile projector (100) has moved as a projection location (or a location to determine a projection area).

[0256] If the mobile projector (100) determines that the location information of the source device (200) is not stored, it can perform a drive to find the user in operation 1108.

[0257] In operation 1110, the mobile projector (100) may determine whether it has discovered a user while driving. According to one or more embodiments, the mobile projector (100) may determine whether it has discovered a user based on user account verification using a mobile terminal, facial recognition, iris recognition, or voice recognition techniques.

[0258] In operation 1112, the mobile projector (100) can determine the location where the user was found as the projection location based on the location where the user was found while performing the driving.

[0259] In operation 1114, the mobile projector (100) may output a user discovery failure notification based on not finding the user while performing the driving. According to one or more embodiments, the mobile projector (100) may output the user discovery failure notification visually through a projection operation using the first projection lens (131) or the second projection lens (132), output it as a voice or acoustic signal, or output it by generating vibration.

[0260] FIG. 12 is a diagram illustrating a map that may be used in a mobile projector according to one or more embodiments.

[0261] Referring to FIG. 12, a map (1200) that can be used in a mobile projector (100) may include a virtual map that represents the structure of an indoor space and the arrangement of devices within the indoor space. For example, in the map (1200), the location information of the source device (200) may be indicated by a first icon (or a first indicator) corresponding to the coordinate information of the source device (200), and the location information of the mobile projector (100) may be indicated by a second icon (or a second indicator) corresponding to the coordinate information of the mobile projector (100).

[0262] According to one or more embodiments, the mobile projector (100) may receive a map (1200) from a server or load a map (1200) previously created and stored by the mobile projector (100) based on being registered or selected as an extended display device of the source device (200).

[0263] When a map (1200) is acquired, the mobile projector (100) can identify the location information of the source device (200) based on the acquired map (1200). The mobile projector (100) can also identify the location information of the mobile projector (100) based on the acquired map (1200).

[0264] The mobile projector (100) may move from its current location (e.g., room 2) to a location (e.g., room 1) where the source device (200) is located based on the identified location information. According to one or more embodiments, the mobile projector (100) may determine that a user will be present in a location adjacent to the source device (200) and perform a user discovery operation after moving the location.

[0265] Referring to FIGS. 13A and 13B below, an operation of a mobile projector (100) determining a projection method for operation of an extended display device will be described. According to one or more embodiments, the operations illustrated in FIGS. 13A and 13B may be operations associated with operation 308 of FIG. 3, and may be operations performed based on the determination of a projection position.

[0266] For example, when the mobile projector (100) is set as an extended display device, projection methods that can be used may include a first projection method and / or a second projection method. The first projection method may include a method using a first projection lens (e.g., a short-focus projection lens) (131), and the second projection method may include a method using a second projection lens (e.g., a long-focus projection lens) (132). The first projection method may have a shorter projection distance or focal length than the second projection method. For convenience of explanation, the first projection method will hereinafter be referred to as a short-focus projection method, and the second projection method will hereinafter be referred to as a long-focus projection method.

[0267] FIG. 13A is a flowchart illustrating an operation of a mobile projector according to one or more embodiments to determine a projection method based on a change in height and the presence of a falling section.

[0268] Referring to FIG. 13A, in operation 1302, the mobile projector (100) may periodically or aperiodically measure a height (or altitude) of the mobile projector (100) from a specific point (e.g., the floor) using the inertial sensor (141) at the determined projection location. According to one or more embodiments, the determined projection location may be the projection location determined in 306 of FIG. 3 (e.g., a location determined based on location information of the source device (200), the projection location determined in operation 1106 of FIG. 11 (e.g., a location where the source device (200) is present), or the projection location determined in operation 1112 (e.g., a location where the user is found).

[0269] In operation 1304, the mobile projector (100) can determine whether there is a change in height (or altitude change) of the mobile projector (100) based on the height measurement result. The change in height of the mobile projector (100) can be caused by the user at the projection position. For example, the change in height of the mobile projector (100) can be caused based on the user lifting the mobile projector (100) at the projection position, or the user placing the mobile projector (100) on the floor on a table (or desk).

[0270] The mobile projector (100) can project an image based on a long-focus projection method at a determined projection position of the mobile projector (100) in operation 1312 based on the absence of a change in the height of the mobile projector (100).

[0271] The mobile projector (100) can detect the surrounding environment of the mobile projector (100) using the distance sensor (142) and / or at least one camera (e.g., RGB camera (151) and / or depth camera (152)) included in the camera assembly (150) based on a change in height of the mobile projector (100) in operation 1306.

[0272] In operation 1308, the mobile projector (100) may determine whether a fall zone exists based on the detection results of the surrounding environment. According to one or more embodiments, the fall zone may represent a zone or point where the mobile projector (100) may fall. For example, the fall zone may include stairs, steps, or points with abrupt elevation changes.

[0273] The mobile projector (100) can project an image based on a single-focus projection method at the determined projection position of the mobile projector (100) in operation 1310 based on the absence of a falling section.

[0274] The mobile projector (100) can perform operation 1314 of FIG. 13b, which is connected by symbol B, based on the existence of a fall section.

[0275] FIG. 13b is a flowchart illustrating an operation of a mobile projector according to one or more embodiments to determine a projection method based on the results of a user's face recognition and gesture detection.

[0276] Referring to FIG. 13b, the mobile projector (100) can perform a face recognition operation using at least one camera at operation 1314 based on the existence of a fall section.

[0277] In operation 1316, the mobile projector (100) may determine whether the user's face is recognized based on a facial recognition operation. According to one or more embodiments, the mobile projector (100) may determine whether the user's face is recognized based on previously stored user facial information.

[0278] The mobile projector (100) can project an image based on a single-focus projection method at a determined projection position in operation 1326 based on the user's face not being recognized.

[0279] The mobile projector (100) can perform a gesture detection operation in the direction in which the user's face is recognized, based on the recognition of the user's face, in operation 1318.

[0280] In operation 1320, the mobile projector (100) can determine whether a user's gesture is detected based on the gesture detection operation.

[0281] The mobile projector (100) can project an image based on a long-focus projection method at a determined projection position in operation 1328 based on the user's gesture not being detected.

[0282] The mobile projector (100) can identify the type of the detected gesture at operation 1322 based on the detected user's gesture.

[0283] In operation 1324, the mobile projector (100) may determine whether the type of identified gesture is a first gesture or a second gesture. According to one or more embodiments, the first gesture may include a gesture in which the user points to the floor with a hand (or finger), and the second gesture may include a gesture in which the user points to a wall with a hand (or finger).

[0284] The mobile projector (100) can project an image based on a long-focus projection method at the determined projection position of the mobile projector (100) at operation 1326 based on the type of the identified gesture being the first gesture.

[0285] The mobile projector (100) can project an image based on a long-focus projection method at the determined projection position of the mobile projector (100) at operation 1328 based on the type of the identified gesture being the second gesture.

[0286] The mobile projector (100) may perform operation 1302 of FIG. 13a again, connected by symbol A, after performing operation 1326 or 1328.

[0287] According to one or more embodiments, the mobile projector (100) may receive user input (or commands) while projecting an image. For example, the mobile projector (100) may receive user input through the UI (170).

[0288] The mobile projector (100) may stop projecting the image and deactivate the setting of the mobile projector (100) as an extended display device if the control signal generated based on the interaction with the UI (170) includes information instructing to project an image different from the image being projected. Alternatively, the mobile projector (100) may ignore the control signal, maintain the setting of the extended display device, and continue projecting the image, depending on the user's settings.

[0289] The mobile projector (100) can stop the projection operation and disable the setting of the mobile projector (100) as an extended display device or stop the operation of the extended display device if the control signal generated based on the interaction with the UI (170) includes information instructing movement.

[0290] The mobile projector (100) may perform other operations than those described above depending on the operation priority.

[0291] According to one or more embodiments, the conditional projection scheme based on the operations illustrated in FIGS. 13a and 13b may be as shown in the following [Table 1].

[0292]

[0293] Referring to [Table 1], the mobile projector (100) can use a long-focus projection method based on the fact that there is no change in the height of the mobile projector (100).

[0294] The mobile projector (100) can utilize a single-focus projection method based on the fact that the height of the mobile projector (100) changes and there is no falling section.

[0295] The mobile projector (100) can use a single-focus projection method based on the fact that the mobile projector (100) has a change in height, a falling section, and the user's face is not recognized.

[0296] The mobile projector (100) can utilize a long-focus projection method based on the change in height of the mobile projector (100), the presence of a falling section, recognition of the user's face, and non-detection of the user's gesture.

[0297] The mobile projector (100) can utilize a single-focus projection method based on the change in height of the mobile projector (100), the presence of a falling section, recognition of the user's face, detection of the user's gesture, and identification of the detected gesture as the first gesture.

[0298] The mobile projector (100) can utilize a long-focus projection method based on the change in height of the mobile projector (100), the presence of a falling section, recognition of the user's face, detection of the user's gesture, and identification of the detected gesture as a second gesture.

[0299] According to one or more embodiments, the conditions that can be used to determine the projection method (e.g., whether the height of the mobile projector (100) changes, whether a fall zone exists, whether a user's face is recognized, whether a gesture is detected, or the type of gesture detected) may be used individually or in combination of at least two. For example, the projection method may be determined by a combination of whether the height of the mobile projector (100) changes and whether a fall zone exists. In this case, the conditions of whether a user's face is recognized or the type of gesture detected may not be used. When the combination of whether the height of the mobile projector (100) changes and whether a fall zone exists is used, the mobile projector (100) may use a long-focus projection method based on the presence of a height change of the mobile projector (100) and the presence of a fall zone.

[0300] FIG. 14a and FIG. 14b are diagrams illustrating a situation in which the height of a mobile projector changes according to one or more embodiments.

[0301] Referring to FIGS. 14A and 14B, the height of the mobile projector (100) from its position can be changed by the user (1402). For example, as shown in FIG. 14A, if the user (1402) lifts the mobile projector (100), the height of the mobile projector (100) can be changed. As another example, as shown in FIG. 14B, if the user places the mobile projector (100) located on the floor (1404) on a table (1406), the height of the mobile projector (100) can be changed. The mobile projector (100) can detect a change in the height of the mobile projector (100) using an inertial sensor (141).

[0302] FIG. 15a is a drawing for explaining a long-focus projection method according to one or more embodiments.

[0303] Referring to FIG. 15A, the mobile projector (100) can project images based on a long-focus projection method. The mobile projector (100) can utilize a second projection lens (e.g., a long-focus projection lens) (132) for the long-focus projection method. The mobile projector (100) can control the main body (101) so that an angle or direction for the long-focus projection method is indicated.

[0304] According to one or more embodiments, an angle or direction corresponding to a long-focus projection method may be used as an initial setting angle or initial setting direction (or basic angle or basic direction, or default angle or default direction) of the mobile projector (100).

[0305] According to one or more embodiments, the mobile projector (100) can project an image on an area (or flat surface) located at a distance from the mobile projector (100), such as a wall or screen (1502), based on a long-focus projection method.

[0306] FIG. 15b is a drawing for explaining a single-focus projection method according to one or more embodiments.

[0307] Referring to FIG. 15b, the mobile projector (100) can project images based on a single-focus projection method. The mobile projector (100) can utilize a first projection lens (e.g., a single-focus projection lens) (131) for the single-focus projection method. The mobile projector (100) can control the main body (101) so that an angle or direction for the single-focus projection method is indicated.

[0308] According to one or more embodiments, the mobile projector (100) can set an angle or direction for a single-focus projection method by rotating the main body (101) by a preset angle (e.g., 45 degrees) in a preset direction (e.g., -y-axis direction or counterclockwise) from an initial set angle (e.g., 0 degrees on an axis parallel to the floor).

[0309] According to one or more embodiments, the mobile projector (100) can project an image onto an area (or plane) adjacent to the mobile projector (100), such as a floor surface (1504), based on a short-focus projection method.

[0310] For example, the projection distance corresponding to a short-focus projection method may be shorter than the projection distance corresponding to a long-focus projection method.

[0311] FIG. 16 is a flowchart illustrating an operation of a mobile projector according to one or more embodiments to project an image by taking into account the direction in which a user's face is recognized.

[0312] According to one or more embodiments, the operations illustrated in FIG. 16 may be performed based on the projection method set for the mobile projector (100) and may be performed for the operation of the extended display device. For example, the set projection method may correspond to a projection method determined based on the operations of FIG. 13a and / or FIG. 13b.

[0313] Referring to FIG. 16, in operation 1602, the mobile projector (100) can perform a face recognition operation using at least one camera.

[0314] In operation 1604, the mobile projector (100) can determine whether the user's face has been recognized based on the result of performing the face recognition operation.

[0315] The mobile projector (100) can terminate operation or project an image based on a set projection method based on whether the user's face is recognized.

[0316] The mobile projector (100) can identify the set projection method in operation 1606 based on the recognition of the user's face.

[0317] In operation 1608, the mobile projector (100) can determine whether the identified projection method is a long-focus projection method or a short-focus projection method.

[0318] In operation 1610, the mobile projector (100) can project an image from the opposite direction of the user's face based on the identified projection method being a long-focus projection method.

[0319] In operation 1612, the mobile projector (100) can project an image in the direction in which the user's face is recognized based on the identified projection method being a long-focus projection method.

[0320] FIG. 17A is a schematic diagram illustrating a mobile projector according to one or more embodiments projecting an image in an opposite direction of a user's face.

[0321] FIG. 17A illustrates an operation of a mobile projector (100) that can be performed in operation 1610 of FIG. 16. When the mobile projector (100) uses a long-focus projection method, the mobile projector (100) can project an image from the opposite direction of the user's face (1702) so that the image is not projected onto the user's face (1702). For example, the mobile projector (100) can project an image onto an area (1700) located in the opposite direction of the user's face (1702) so that the image is projected in the direction toward which the user's gaze is directed. The mobile projector (100) can project an image by controlling the main body (101) to have an angle corresponding to the long-focus projection method (e.g., an initial setting angle).

[0322] FIG. 17b is a schematic diagram illustrating a mobile projector according to one or more embodiments projecting an image in a direction in which a user's face is recognized.

[0323] FIG. 17b illustrates an operation of a mobile projector (100) that can be performed in operation 1612 of FIG. 16. When the mobile projector (100) uses a single-focus projection method, the mobile projector (100) can project an image from a direction in which a user's face (1702) is recognized. The mobile projector (100) can project an image by controlling the main body (101) to have an angle corresponding to the single-focus projection method (e.g., an angle rotated 45 degrees in the -y-axis direction from the initial setting angle). Since the projection angle corresponding to the single-focus projection method may be directed toward the floor (1704), the image may not be projected on the user's face (1706).

[0324] FIG. 18 is a flowchart illustrating an operation of a mobile projector according to one or more embodiments to project an image based on an obstacle detection result.

[0325] According to one or more embodiments, the operations illustrated in FIG. 18 may be performed based on the projection method set for the mobile projector (100) and may be performed for the operation of the extended display device. For example, the set projection method may correspond to a projection method determined based on the operations of FIG. 13a and / or FIG. 13b.

[0326] In operation 1802, the mobile projector (100) can identify a direction corresponding to a set projection method. For example, the mobile projector (100) can identify a first direction corresponding to a long-focus projection method, which can correspond to a first angle corresponding to the long-focus projection method, and a second direction corresponding to a short-focus projection method, which can correspond to a second angle corresponding to the short-focus projection method. The first angle and / or the second angle can be determined based on an initial setting angle and / or a rotation angle of the main body (101).

[0327] In operation 1804, the mobile projector (100) may perform an obstacle detection operation in the identified direction. According to one or more embodiments, the mobile projector (100) may perform an obstacle detection operation to analyze a projection area (or projection surface) corresponding to the identified direction using at least one camera. For example, the mobile projector (100) may perform an obstacle detection operation to analyze the projection area based on a plane detection operation or an object (obstacle) detection operation using a computer vision algorithm.

[0328] In operation 1806, the mobile projector (100) can determine whether an obstacle is detected in the projection area based on the result of performing the obstacle detection operation. For example, the obstacle may include a person, an object, an animal, or a structure.

[0329] The mobile projector (100) can change direction by adjusting the projection angle in operation 1808 based on the detection of an obstacle in the projection area. According to one or more embodiments, the projection angle can be adjusted based on the rotation of the main body (101).

[0330] In action 1810, the mobile projector (100) can project an image in a changed direction.

[0331] The mobile projector (100) can project an image in an identified direction in operation 1812 based on the fact that no obstacles are detected in the projection area.

[0332] FIG. 19A is a diagram illustrating an operation of a mobile projector according to one or more embodiments to change a projection angle for long-focus projection.

[0333] Referring to Fig. 19a, the mobile projector (100) can perform long-focus projection using a long-focus projection method at an initially set angle. When the mobile projector (100) detects the presence of a user or the user's face or an obstacle in the projection direction (or projection area), the mobile projector (100) can rotate the main body (101) to change the projection angle.

[0334] According to one or more embodiments, as shown in (a) of FIG. 19A, the mobile projector (100) can rotate the main body (101) in the positive direction of an axis perpendicular to the floor surface (e.g., the y-axis) by a preset angle (e.g., an angle between 0 and 90 degrees) (1402).

[0335] According to one or more embodiments, as shown in (b) of FIG. 19a, the position of the projection unit (130) may be changed based on the rotation of the main body (101), and the projection direction or angle may be changed based on the changed position of the projection unit (130).

[0336] FIG. 19b is a diagram illustrating an operation of a mobile projector according to one or more embodiments to change a projection angle for short-focus projection.

[0337] Referring to FIG. 19b, the mobile projector (100) can perform long-focus projection at an initially set angle and then rotate the main body (101) to perform short-focus projection.

[0338] According to one or more embodiments, as shown in (a) of FIG. 19b, the mobile projector (100) can rotate the main body (101) in the negative direction of an axis perpendicular to the floor surface (e.g., the y-axis) by a preset angle (e.g., an angle between 0 and 45 degrees) (1904).

[0339] According to one or more embodiments, as shown in (b) of FIG. 19b, the position of the projection unit (130) may be changed based on the rotation of the main body (101), and the projection direction or angle may be changed to a projection direction or angle for single-focus projection based on the changed position of the projection unit (130).

[0340] According to one or more embodiments, the mobile projector (100) may rotate the main body (101) to change the projection angle when it detects that a user or the user's face is present in the projection direction (or projection area) for short-focus projection, or that an obstacle is present. For example, the mobile projector (100) may rotate the main body (101) in the positive or negative direction about an axis perpendicular to the floor (e.g., the y-axis) by a preset angle (e.g., an angle between 0 and 10 degrees).

[0341] FIG. 20 is a flowchart illustrating an operation of a mobile projector according to one or more embodiments to perform a projection operation based on a user's face or gaze.

[0342] According to one or more embodiments, the operations illustrated in FIG. 20 may be performed based on what the user perceives by the mobile projector (100) and may be performed for the operation of the extended display device. According to one or more embodiments, the mobile projector (100) may know in advance the height of the mobile projector (100), the position of the projection unit (130), or the second projection lens (132).

[0343] Referring to FIG. 20, in operation 2002, the mobile projector (100) can recognize the user's face or pupils using at least one camera included in the camera assembly (150).

[0344] In operation 2004, the mobile projector (100) can predict the height of the user's face or pupil. For example, the mobile projector (100) can predict the height of the user's face or pupil based on the distance sensor (142) and / or at least one camera.

[0345] In operation 2006, the mobile projector (100) can measure the distance to the projection area (or projection surface). For example, the mobile projector (100) can measure the distance between the mobile projector (100) and the projection area using a distance sensor (142).

[0346] In Action 2008, the mobile projector (100) can determine the projection angle based on the predicted height and measured distance.

[0347] In operation 2010, the mobile projector (100) can perform a projection operation based on a determined projection angle. For example, the mobile projector (100) can rotate the main body (101) to correspond to the projection angle and then project an image onto a projection area.

[0348] FIG. 21 is a drawing for explaining an operation of a mobile projector according to one or more embodiments to determine a projection angle.

[0349] Referring to Fig. 21, when the mobile projector (100) uses a long-focus projection method, the projection area may be a wall surface (2102). The mobile projector (100) can measure the distance (A) to the wall surface (2102).

[0350] A mobile projector (100) can recognize a user's face or pupils. The mobile projector (100) can measure the distance (B) from a position on a wall (2102) located a distance (A) from the mobile projector (100) to a position (2104) corresponding to the user's face or pupils.

[0351] The mobile projector (100) can determine the projection angle (a) using the inverse trigonometric function based on the distance (A) and height (B). For example, the mobile projector (100) can determine the projection angle (a) using the following [Mathematical Formula 1].

[0352]

[0353] A mobile projector (100) can project an image onto a wall (2102) by rotating the main body (101) based on a determined projection angle (a). The image projected onto the wall (2102) can correspond to the user's line of sight.

[0354] According to various embodiments of the present document, a mobile projector (100) can be used as an extended display device in an indoor space (e.g., inside a house), thereby providing a more effective user experience. For example, the mobile projector (100) can move to a location where a user or a source device (200) is located, regardless of where the mobile projector (100) is located in the indoor space, and project images of various contents. The mobile projector (100) can determine a projection location by recognizing the current location of the mobile projector (100), changes in the situation of the mobile projector (100) (e.g., changes in altitude or height), surrounding environment information (e.g., whether it is in a place where a fall is possible), or the user's face or gestures, thereby improving the user experience of the extended display device.

[0355] When an expression such as "at least one of" is used in this disclosure before a list of elements, it modifies the entire list of elements without modifying individual elements of the list. For example, the expression "at least one of a, b, and c" should be understood to include a only, b only, c only, both a and b, both a and c, both b and c, or all of a, b, and c. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish the corresponding element from other corresponding elements and do not limit the corresponding elements in any other respect (e.g., importance or order). When an (e.g., a first) element is referred to as "coupled" or "connected" to another (e.g., a second) element, with or without the terms "functionally" or "communicatively," it means that the corresponding element can be connected to the corresponding other element directly (e.g., wired), wirelessly, or via a third element. In contrast, when an element is said to be "directly over," "directly above," "directly on," "directly below," "directly under," "directly beneath," "directly beneath," "directly connected to," or "directly coupled to" another element or layer, there are no intervening elements or layers.

[0356] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0357] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In a mobile projector device, A drive assembly including at least one wheel; transceiver; projector; memory for storing commands; and Contains at least one processor, The above instructions, when executed by the at least one processor, cause the mobile projector device to: Determine whether the above mobile projector device is set as an extended display device associated with the source device, Based on determining that the above mobile projector device is set as the extended display device, information about the image is received from the source device through the communication unit, Determine the projection position to project the image based on the location information of the source device, A mobile projector device that causes the projection unit to be controlled to project the image at the projection position.

2. In paragraph 1, The above instructions, when executed by the at least one processor, cause the mobile projector device to: Receiving a request message requesting to set the above mobile projector device as the extended display device from the source device through the communication unit, Based on receipt of the above request message, determine whether a projection operation by the projection unit is being performed, Based on the determination that the above projection operation is being performed, the communication unit is controlled to determine not to set the mobile projector device as the extended display device and to transmit a rejection message to the source device. A mobile projector device, wherein, based on the determination that the projection operation is not being performed, the mobile projector device is determined to be set as the extended display device, and further causes the communication unit to control the communication unit to transmit an acceptance message to the source device.

3. In paragraph 1, The above instructions, when executed by the at least one processor, cause the mobile projector device to: A mobile projector device further causing the driving unit to be controlled to move the mobile projector device to the projection position.

4. In paragraph 1 or paragraph 3, The above instructions, when executed by the at least one processor, cause the mobile projector device, while at the projection position: Identify an area adjacent to the source device, A mobile projector device further causing the projector to be controlled to project the image on the identified area.

5. In paragraph 1, It further includes a user interface (UI), The above instructions, when executed by the at least one processor, cause the mobile projector device to: Identify control signals based on interactions with the above UI, Based on the identified control signal, further causing the projection of the image to be stopped and the setting of the mobile projector device as the extended display device to be released, A mobile projector device, wherein the above-identified control signal includes information instructing to project another image or information instructing to move the mobile projector device.

6. In paragraph 1, Information about the above video is: A mobile projector device comprising information about at least one of an image received in real time by the source device, an image associated with an application running on the source device, an image associated with a camera connected to the source device, and an image provided from a mobile terminal connected to the source device.

7. In paragraph 1, entity; drive unit; multiple sensors; and Includes at least one more camera, The above instructions, when executed by the at least one processor, cause the mobile projector device to: The presence or absence of an obstacle in the direction in which the image is projected is determined by at least one camera and / or the plurality of sensors, A mobile projector device, further causing the driving unit to be controlled so that the main body of the mobile projector device rotates by a predetermined angle based on determining the presence of the obstacle.

8. In paragraph 1, multiple sensors; and Includes at least one more camera, The above instructions, when executed by the at least one processor, cause the mobile projector device to: The presence or absence of a change in height at the above projection position is determined by a first sensor among the plurality of sensors, Based on determining the presence of the above height change: The presence or absence of a fall zone adjacent to the mobile projector device is determined by at least one camera or at least one second sensor among the plurality of sensors, and the fall zone indicates an area into which the mobile projector device may fall. Based on determining the absence of the above-mentioned fall section, the projection unit is controlled to project the image using the first projection method, Based on the determination of the absence of the above height change: Further causing the projector to be controlled to project the image using a second projection method, A mobile projector device in which the first projection method and the second projection method have different projection distances or projection directions.

9. In paragraph 8, The above instructions, when executed by the at least one processor, cause the mobile projector device to: Based on determining the existence of the above fall zone: Performing a facial recognition operation by at least one camera, Based on the above facial recognition operation, determine whether the user's face is recognized, A mobile projector device further causing the projector to be controlled to project the image using the first projection method based on the user's face not being recognized.

10. In paragraph 8, The above instructions, when executed by the at least one processor, cause the mobile projector device to: Based on determining the existence of the above fall zone: Performing a facial recognition operation by at least one camera, Based on the above facial recognition operation, determine whether the user's face is recognized, Based on the recognition of the user's face: Determining whether the user's gesture is detected by at least one camera from the direction in which the user's face is recognized, Based on the detected user gestures: Identify whether the user's gesture is a first gesture or a second gesture, Controlling the projection unit to project the image using the first projection method based on the user's gesture being the first gesture; Based on the user's gesture being the second gesture, the projector is controlled to project the image using the second projection method, Based on the above user's gesture not being detected: A mobile projector device that causes the projection unit to be controlled to project the image using the second projection method.

11. In the method of operating a mobile projector device, An action to determine whether the above-mentioned mobile projector device is to be set as an extended display device associated with a source device; An operation of receiving information about an image from the source device based on determining that the mobile projector device is set as the extended display device; An operation of determining a projection position to project the image based on the location information of the source device; and A method comprising an action of projecting the image at the projection position.

12. In paragraph 11, The action for determining whether to set the above mobile projector device as the above extended display device is: An action of receiving, from the source device, a request message requesting that the mobile projector device be set as the extended display device; An action for determining whether a projection operation is being performed based on receipt of the above request message; An operation of determining not to set the mobile projector device as the extended display device based on the judgment that the projection operation is being performed, and transmitting a rejection message to the source device; and A method comprising: determining to set the mobile projector device as the extended display device based on determining that the projection operation is not being performed, and transmitting an acceptance message to the source device.

13. In paragraph 11, A method further comprising an action of controlling a driving unit of the mobile projector device to move the mobile projector device to the projection position.

14. In paragraph 11 or 13, The action of projecting the above image is: An operation of identifying an area adjacent to the source device; and A method comprising an action of projecting the image onto the identified area.

15. In paragraph 11, An operation for identifying a control signal generated based on interaction with a user interface (UI) of the above mobile projector device; and Further comprising an action of stopping projection of the image and releasing the setting of the mobile projector device as the extended display device based on the identified control signal; A method wherein the identified control signal includes information instructing to project another image or information instructing to move the mobile projector device.

Images