Robot and image projection method thereof

Abstract

A robot is disclosed. The robot comprises a projector, a camera, a driving unit, and one or more processors configured to: project a first calibration image in which a marker is moved by using the projector; obtain an image by photographing, using the camera, a first calibration image projected by the robot and a first calibration image projected by another robot; on the basis of the obtained image, identify the positions of respective markers of the first calibration image projected by the robot and the first calibration image projected by the other robot; identify a delay time for adjusting a reproduction time point of an image to be projected by the robot on the basis of the identified positions; and while the robot is at a position corresponding to a projection surface onto which an image is to be projected, control the projector to project the image onto the projection surface on the basis of the delay time.

Description

Robot and its image projection method

[0001] The present disclosure relates to a robot that projects an image and a method for projecting the image thereof.

[0002] In addition to simple repetitive functions, robots can sense their surroundings in real time using sensors, cameras, and other tools, collect information, and drive autonomously.

[0003] These robots are currently being used in a wide range of fields, providing users with a variety of services. For example, robots equipped with projectors can project images onto walls, ceilings, and other surfaces within the space where they are located, providing users with a variety of video content.

[0004] A robot according to an embodiment of the present disclosure includes a projector, a camera, a driving unit, and one or more processors. The one or more processors project a first calibration image in which a marker is moved using the projector. The one or more processors capture the first calibration image projected by the robot and the first calibration image projected by another robot using the camera to acquire images. The one or more processors identify the positions of markers in each of the first calibration image projected by the robot and the first calibration image projected by the other robot based on the acquired images. The one or more processors identify a delay time for adjusting the playback timing of an image to be projected by the robot based on the identified positions. The one or more processors control the projector to project the image on the projection surface based on the delay time while the robot is at a position corresponding to the projection surface on which the image is to be projected.

[0005] Additionally, the first calibration image projected by the other robot may be an image of the marker moving.

[0006] In addition, the one or more processors can identify a playback point of the first calibration image projected by the robot based on a position of a marker in the first calibration image projected by the robot, identify a playback point of the first calibration image projected by the other robot based on a position of a marker in the first calibration image projected by the other robot, and identify a difference between the identified playback points as the delay time.

[0007] In addition, the one or more processors may identify a playback point corresponding to a position of a marker in a first calibration image projected by the robot among a plurality of playback points corresponding to a plurality of movement degrees of the marker, identify the identified playback point as a playback point of the first calibration image projected by the robot, and identify a playback point corresponding to a position of a marker in a first calibration image projected by the other robot among a plurality of playback points corresponding to a plurality of movement degrees of the marker, and identify the identified playback point as a playback point of the first calibration image projected by the other robot.

[0008] Additionally, the markers included in the first calibration image projected by each of the robot and the other robot may include a QR code.

[0009] In addition, the one or more processors may project a second calibration image using the projector while the robot is at a position corresponding to a projection surface on which the image is to be projected, capture the second calibration image projected by the robot and the second calibration image projected by the other robot using the camera to obtain an image, control the movement of the robot using the driving unit based on the obtained image so that the second calibration image projected by the robot and the second calibration image projected by the other robot are aligned with each other, and control the projector to project the image based on the delay time after the movement of the robot is controlled.

[0010] In addition, the one or more processors can identify a target area corresponding to a marker of a second calibration image projected by the robot based on the acquired image, and control the movement of the robot using the driving unit so that the marker of the second calibration image projected by the other robot is positioned in the target area in the acquired image.

[0011] Additionally, the markers included in the second calibration images projected by each of the robot and the other robot may include QR codes positioned at fixed locations.

[0012] Additionally, the position corresponding to the projection surface can be determined based on at least one of the position of the projection surface set according to user input, the size of the image, and the aspect ratio of the image.

[0013] An image projection method of a robot including a projector and a camera according to an embodiment of the present disclosure includes the steps of: projecting a first calibration image in which a marker is moved using the projector; capturing the first calibration image projected by the robot and the first calibration image projected by another robot using the camera to acquire images; identifying positions of markers of each of the first calibration image projected by the robot and the first calibration image projected by the other robot based on the acquired images; identifying a delay time for adjusting a playback time of an image to be projected by the robot based on the identified positions; and controlling the projector to project the image on the projection surface based on the delay time while the robot is at a position corresponding to the projection surface on which the image is to be projected.

[0014] A non-transitory computer-readable medium storing computer instructions that, when executed by one or more processors of a robot including a projector and a camera according to an embodiment of the present disclosure, cause the robot to perform an operation, the operation includes: projecting a first calibration image in which a marker is moved using the projector; capturing the first calibration image projected by the robot and the first calibration image projected by another robot using the camera to acquire images; identifying a position of a marker in each of the first calibration image projected by the robot and the first calibration image projected by the other robot based on the acquired images; identifying a delay time for adjusting a playback time of an image to be projected by the robot based on the identified positions; and controlling the projector to project the image onto the projection surface based on the delay time while the robot is at a position corresponding to the projection surface on which the image is to be projected.

[0015] FIG. 1 is a drawing for explaining the operation of a robot according to an embodiment of the present disclosure.

[0016] FIG. 2a is a block diagram illustrating the configuration of a robot according to an embodiment of the present disclosure.

[0017] FIG. 2b is a block diagram illustrating the configuration of a robot according to an embodiment of the present disclosure.

[0018] FIG. 3 is a flowchart illustrating an operation of a robot performing time synchronization according to an embodiment of the present disclosure.

[0019] FIG. 4 is a drawing for explaining an example of a first calibration image according to an embodiment of the present disclosure.

[0020] FIG. 5 is a drawing for explaining an example of a robot taking first calibration images according to an embodiment of the present disclosure.

[0021] FIG. 6 is a diagram illustrating a method for a robot to identify the positions of markers in first calibration images according to an embodiment of the present disclosure.

[0022] FIG. 7 is a drawing for explaining an example of a method for a user to set a projection surface using a mobile device according to an embodiment of the present disclosure.

[0023] FIG. 8 is a drawing for explaining an example of images projected by a robot and another robot according to an embodiment of the present disclosure.

[0024] FIG. 9 is a flowchart illustrating an operation of a robot performing fine adjustment according to an embodiment of the present disclosure.

[0025] FIG. 10 is a drawing for explaining an example of a robot taking second calibration images according to an embodiment of the present disclosure.

[0026] FIGS. 11, 12a, and 12b are drawings for explaining an example of a method for controlling the movement of a robot by capturing images of second calibration images according to an embodiment of the present disclosure.

[0027] FIG. 13 is a drawing for explaining an example of images projected by a robot and another robot according to an embodiment of the present disclosure.

[0028] Hereinafter, terms used in this specification will be briefly described, and the present disclosure will be described in detail. In this disclosure, the expression “at least one of a, b, or c” can refer to “a,” “b,” “c,” “a and b,” “a and c,” “b and c,” “all of a, b, and c,” or variations thereof.

[0029] The terms used in this disclosure are selected from widely used, common terms, taking into account the functions of the disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, in which case their meanings will be described in detail in the relevant description. Therefore, the terms used in this disclosure should not be defined simply as names, but rather based on the meanings of the terms and the overall content of the disclosure.

[0030] Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art described herein. Furthermore, terms containing ordinal numbers, such as "first" or "second," used herein may be used to describe various components, but such components should not be limited by such terms. Such terms are used solely to distinguish one component from another.

[0031] When a part of the specification is said to "include" a component, unless otherwise specifically stated, this does not exclude other components but rather implies the inclusion of other components. Furthermore, terms such as "part" and "module" used in the specification refer to a unit that processes at least one function or operation, which may be implemented in hardware, software, or a combination of hardware and software.

[0032] The term "and / or" includes any combination of a plurality of related described elements or any one of a plurality of related described elements.

[0033] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0034] The present disclosure will be described below with reference to the attached drawings.

[0035] FIG. 1 is a drawing for explaining the operation of a robot according to an embodiment of the present disclosure.

[0036] In the present disclosure, the robot (100-1, 100-2) may be a mobile robot (e.g., a mobile robot). The robot (100-1, 100-2) may also be referred to as an autonomous driving device or a mobile device.

[0037] The movement of the robot (100-1, 100-2) may include exploring its surroundings, detecting its position and obstacles, and autonomously navigating within the space using the detected information. The term "movement" may be replaced with expressions such as "driving," for example.

[0038] The space may include various indoor spaces where the robot (100-1, 100-2) can move, such as a home, office, hotel, factory, store, supermarket, restaurant, etc.

[0039] In the present disclosure, the robots (100-1, 100-2) may be implemented as various types of robots. For example, the robots (100-1, 100-2) may be implemented as a robot vacuum cleaner that moves within a space and performs cleaning, a guide robot that guides a user along a path within a space or provides various information related to services provided within the space, a delivery robot or serving robot that transports loaded products to a specific location within a space, or a mobile projection device for projecting images while moving within a location.

[0040] In one embodiment, the robots (100-1, 100-2) can project images together to provide images to the user.

[0041] For example, the first robot (100-1) and the second robot (100-2) may project an image together, such that the first robot (100-1) projects a portion (11) of the image (10) and the second robot projects the remaining portion (12) of the image (10), thereby implementing a single entire image (10) through the images projected by the first robot (100-1) and the second robot (100-2). Accordingly, a larger image can be provided to the user than when the image is projected by a single robot.

[0042] For convenience of explanation, in the following, one of the first robot (100-1) and the second robot (100-2) for providing a large-screen image is referred to as robot (100), and the other robot is referred to as another robot.

[0043] In order for the robot (100) to provide images to the user together with other robots, the specific operations for projecting images will be described in more detail through the drawings and descriptions thereof described below.

[0044] FIG. 2a is a block diagram illustrating the configuration of a robot according to an embodiment of the present disclosure.

[0045] Referring to FIG. 2a, the robot (100) may include a projector (110), a camera (120), a driving unit (130), and one or more processors (140).

[0046] A projector (110) can project images under the control of one or more processors (140). The images can include still images and video. The video can include various visual information representing the movement of an object using a plurality of consecutive still images. Each of the plurality of still images included in the video can mean a frame (or video frame).

[0047] The projector (110) can project an image onto a projection surface using light emitted from a light source. For example, the projector (110) can project an image using a CRT (Cathode-Ray Tube) method, an LCD (Liquid Crystal Display) method, a DLP (Digital Light Processing) method, or an LCoS (Liquid Crystal on Silicon) method. The projection surface may include a separately provided screen, various wall surfaces within a space where the robot (100) moves, or one side of an object.

[0048] The projector (110) can project images in various aspect ratios. For example, the aspect ratios can include various ratios such as 4:3, 5:4, or 16:9.

[0049] The camera (120) can perform photography under the control of one or more processors (140). For example, the camera (120) may include an RGB camera. The camera (120) can capture the front of the robot (100) and generate an image. The image may include a plurality of pixels.

[0050] The driving unit (130) can control the movement of the robot (100) under the control of one or more processors (140).

[0051] According to one example, the driving unit (130) can move and stop the robot (100) and control the moving direction and / or moving speed of the robot (100).

[0052] For example, the driving unit (130) may include a plurality of wheels and at least one wheel motor. The wheel motor may control the direction of rotation and speed of the wheels, thereby controlling the direction and speed of movement of the robot (100). For example, if the robot (100) includes two wheels (e.g., a left wheel and a right wheel), the wheel motor may include a left wheel motor for controlling the direction of rotation and speed of movement of the left wheel, and a right wheel motor for controlling the direction of rotation and speed of movement of the right wheel.

[0053] According to an example, the driving unit (130) can rotate the body of the robot (100). For example, the driving unit (130) can rotate the body of the robot (100) in an upward or downward direction. For example, the driving unit (130) may include a motor and / or an actuator.

[0054] One or more processors (140) can control the overall operations of the robot (100). For example, one or more processors (140) can control the overall operations of the robot (100) to project an image together with another robot and provide a single image to a user by executing one or more instructions of a program stored in the memory of the robot (100).

[0055] The one or more processors (140) may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a hardware accelerator, or a machine learning accelerator. The one or more processors (140) may control one or any combination of other components of the robot (100) and may perform operations or data processing related to communication. The one or more processors (140) may execute one or more programs or instructions stored in the memory of the robot (100). For example, the one or more processors (140) may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in the memory of the robot (100).

[0056] When a method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0057] One or more processors (140) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors (140) are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include an internal processor memory, such as a cache memory or an on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0058] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0059] In embodiments of the present disclosure, a processor may mean a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but embodiments of the present disclosure are not limited thereto.

[0060] FIG. 2b is a block diagram illustrating the configuration of a robot according to an embodiment of the present disclosure.

[0061] Referring to FIG. 2b, the robot (100) may include a projector (110), a camera (120), a driving unit (130), one or more processors (140), a memory (150), a sensor unit (160), a communication interface (170), an input interface (180), and an output interface (190). However, such a configuration is exemplary, and it is obvious that new configurations may be added or some configurations may be omitted in addition to such configurations when implementing the present disclosure. Meanwhile, a detailed description of configurations that overlap with the configurations illustrated in FIG. 2a among the configurations illustrated in FIG. 2b will be omitted.

[0062] Memory (150) may store instructions, data structures, and program codes. Operations performed by one or more processors (140) may be implemented by executing instructions or codes of a program stored in memory (150).

[0063] The memory (150) may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), and may include a non-volatile memory including at least one of a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk, and a volatile memory such as a RAM (Random Access Memory) or an SRAM (Static Random Access Memory).

[0064] The memory (150) can store one or more instructions and / or programs that cause the robot (100) to perform an operation to project an image together with another robot to provide a single image to a user.

[0065] The sensor unit (160) can detect spatial structures or objects. Objects may include walls, furniture, home appliances, etc. within the space. Additionally, information acquired by the sensor unit (160) can be used to create a map of the space.

[0066] The sensor unit (160) may include at least one of a 3D camera sensor (e.g., a depth camera), a light detection and ranging (LiDAR) sensor, an obstacle detection sensor, and a driving detection sensor.

[0067] A 3D camera sensor can capture images of the surroundings of a robot (100) and generate 3D spatial information related to the surroundings of the robot (100). For example, the 3D camera sensor can detect the distance to objects around the robot (100) and generate an image (e.g., a depth image) containing 3D distance information.

[0068] A lidar sensor outputs a laser in a 360-degree direction, and when a laser reflected from an object is received, the sensor analyzes the time difference taken for the laser to reflect from the object and return, the signal intensity of the received laser, etc., to obtain geometry information about the space. The geometry information may include the position, distance, direction, etc. of the object. The lidar sensor may provide the obtained geometry information to one or more processors (140).

[0069] The obstacle detection sensor can detect objects around the robot (100). For example, the obstacle detection sensor can include at least one of an ultrasonic sensor, an infrared sensor, an RF (radio frequency) sensor, a geomagnetic sensor, and a PSD (Position Sensitive Device) sensor. The obstacle detection sensor can detect objects present in front, behind, to the side, or on the movement path of the robot (100). The obstacle detection sensor can provide information about the detected objects to one or more processors (140).

[0070] The driving detection sensor can detect the driving of the robot (100). For example, the driving detection sensor can include at least one of a gyro sensor, a wheel encoder, and an acceleration sensor. The gyro sensor can detect the rotation direction and rotation angle of the robot (100). The wheel encoder can detect the number of rotations of the wheels of the robot (100). The acceleration sensor can detect changes in the speed of the robot (100). The driving detection sensor can provide detected driving information to one or more processors (140).

[0071] The communication interface (170) can perform data communication with electronic devices under the control of one or more processors (140). The electronic devices may include servers, home appliances, mobile devices (e.g., smartphones, tablet PCs, wearable devices, etc.).

[0072] For example, the communication interface (170) may include a communication circuit that can perform data communication between the robot (100) and the electronic device using at least one of data communication methods including wired LAN, wireless LAN, Wi-Fi, Wi-Fi Direct, Bluetooth, ZigBee, Wi-Fi Direct (WFD), infrared Data Association (IrDA), Bluetooth Low Energy (BLE), Near Field Communication (NFC), Wireless Broadband Internet (Wibro), World Interoperability for Microwave Access (WiMAX), Shared Wireless Access Protocol (SWAP), Wireless Gigabit Alliances (WiGig), and RF communication.

[0073] The input interface (180) includes circuitry. The input interface (180) can receive user input and transmit the user input to one or more processors (140). For example, the input interface (180) can receive various user inputs for setting or selecting various functions supported by the robot (100).

[0074] The input interface (180) may include various types of input devices.

[0075] In one example, the input interface (180) may include a physical button. The physical button may include a function key or a dial button. The physical button may also be implemented as one or more keys.

[0076] In one example, the input interface (180) can receive user input using a touch method. For example, the input interface (180) can be implemented as a touch screen capable of performing the function of a display (191).

[0077] For example, the input interface (180) may receive a user's voice using a microphone. One or more processors (140) may perform a function corresponding to the user's voice using voice recognition. For example, one or more processors (140) may convert the user's voice into text data using a STT (Speech To Text) function, obtain control command data based on the text data, and perform a function corresponding to the user's voice based on the control command data. Depending on the embodiment, the STT function may be performed by an external server.

[0078] The output interface (190) may include a display (191) and a speaker (192).

[0079] The display (191) can display various screens. One or more processors (140) can display various notifications, messages, information, etc. related to the operation of the robot (100) on the display (191).

[0080] The display (191) may be implemented as a display including a self-luminous element or a display including a non-luminous element and a backlight. For example, the display (191) may be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes) display, a micro LED display, a Mini LED display, a QLED (Quantum dot light-emitting diodes) display, etc.

[0081] The speaker (192) can output audio signals. One or more processors (140) can output warning sounds, notification messages, response messages corresponding to user input, etc. related to the operation of the robot (100) through the speaker (192).

[0082] One or more processors (140) can generate a map of a space using information acquired through the sensor unit (160). The map can be generated during an initial exploration of the space. For example, one or more processors (140) can explore the space using a lidar sensor, acquire topographic information about the space, and generate a map of the space using the topographic information.

[0083] For example, the map may include a grid map. A grid map is a map that divides space into cells of a certain size and expresses it. For example, a grid map may be a map that divides space into a plurality of cells of a predetermined size and indicates the presence or absence of an object in each cell. The plurality of cells may be divided into cells where no objects exist (e.g., cells where a robot (100) can drive) (free space) and cells where objects exist (occupied space). A line connecting cells occupied by objects may represent a boundary of the space (e.g., an object such as a wall). However, the present invention is not limited thereto, and the map may be of various types.

[0084] One or more processors (140) can identify the location of the robot (100) on the map using Simultaneous Localization and Mapping (SLAM). For example, one or more processors (140) can obtain spatial topographic information using a lidar sensor, compare the obtained topographic information with pre-stored topographic information, or compare the obtained topographic information to identify the location of the robot (100) on the map. However, the present invention is not limited thereto, and one or more processors (140) can identify the location of the robot (100) on the map using SLAM using a camera (120).

[0085] One or more processors (140) can control the movement of the robot (100) using information obtained through the sensor unit (160).

[0086] For example, one or more processors (140) can control the driving unit (130) to allow the robot (100) to move through space using a map stored in the memory (150). In addition, one or more processors (140) can obtain information using the sensor unit (160) while the robot (100) moves through space, and can detect objects around the robot (100) using the obtained information. When an object is detected, one or more processors (140) can control the driving unit (130) to move while avoiding the object.

[0087] For convenience of explanation, one or more processors (140) are referred to as processors (140) below.

[0088] In one embodiment, the robot (100) can project images together with other robots to provide images to the user. Since the robot (100) and the other robots each project a portion of the overall image, synchronization between the robots may be necessary to provide the user with a natural, large-screen image.

[0089] Synchronization between robots may include time synchronization. Time synchronization may include adjusting the playback timing of images projected by the robot (100) and / or another robot so that images from the same playback timing are projected onto the projection surface by the robot (100) and another robot.

[0090] FIG. 3 is a flowchart illustrating an operation of a robot performing time synchronization according to an embodiment of the present disclosure.

[0091] Hereinafter, a calibration image for time synchronization projected by the robot (100) and another robot is described as a first calibration image.

[0092] For example, the processor (140) can receive user input for time synchronization.

[0093] In this case, user input can be entered through a mobile device. The mobile device can communicate with the robot (100) and other robots. For example, a user can connect to a server by executing an application installed on the mobile device, create a user account, and communicate with the server based on the logged-in user account to register the robot (100) and other robots. The server can register the robot (100) and other robots to the user account by listing identification information (e.g., serial number or MAC address) of the robot (100) and other robots in the user account.

[0094] In addition, the user can control the robot (100) and other robots using an application installed on the mobile device. For example, when the user logs into a user account using an application installed on the mobile device, a user interface (UI) screen related to the robot (100) and other robots registered to the user account may be displayed on the mobile device. The user can input user input for controlling the robot (100) and other robots on the UI screen displayed on the mobile device. The server can transmit control commands corresponding to the user input entered on the mobile device to the robot (100) and other robots.

[0095] However, the mobile device may be directly connected to the robot (100) and other robots, without limitation. For example, the mobile device may communicate with the robot (100) and other robots using a short-range wireless network (e.g., Bluetooth, Wi-Fi Direct, etc.). In addition, the mobile device may transmit control commands corresponding to user inputs entered into the mobile device to the robot (100) and other robots.

[0096] When a control command for time synchronization is received, the processor (140) can perform an operation for time synchronization with another robot.

[0097] The operation for time synchronization may include the robot (100) identifying a delay time between images based on the positions of markers in the first calibration images projected by the robot (100) and another robot. In this case, the delay time may be used to adjust the playback timing of the images to be projected by the robot (100).

[0098] Below, the method by which the robot (100) performs operations for time synchronization is described in more detail.

[0099] In operations S310 and S320, the processor (140) can project a first calibration image using a projector (110) and capture the first calibration image projected by the robot (100) and the first calibration image projected by another robot using a camera (120) to obtain an image.

[0100] For example, the robot (100) and another robot may be positioned adjacent to each other. The robots being adjacent to each other may include the robots being positioned within a predetermined distance from each other such that one robot can capture images projected by the other robots using a camera.

[0101] According to one example, when a user input for time synchronization is received, the processor (140) may control the driving unit (130) to cause the robot (100) to move to a specific location within the map based on the location of the robot (100). In addition, the processor (140) may play a first calibration image and project the first calibration image onto a projection surface using a projector (110). In addition, another robot may move to a specific location within the map according to the user input for time synchronization. Accordingly, the robot (100) and the other robot may be positioned adjacent to each other. In addition, the other robot may play the first calibration image and project the first calibration image onto the projection surface.

[0102] In one example, a user may place a robot (100) and another robot adjacent to each other. In this case, the robot (100) and the other robot may each project a first calibration image onto the projection surface when a user input for time synchronization is received.

[0103] The first calibration image may be an image of a marker moving. The marker may move in one direction within the first calibration image. The markers in the first calibration images projected by the robot (100) and other robots may have the same size and the same movement speed.

[0104] For example, referring to FIG. 4, if the width (or horizontal) of the first calibration image (410) corresponds to the x-axis and the height (or vertical) of the first calibration image corresponds to the y-axis, the marker (420) can move in the x-axis direction over time.

[0105] The marker may include a visual graphic element that can be recognized by the robot (100) and other robots.

[0106] For example, the marker could be a QR (Quick Response) code.

[0107] For example, a marker may be a graphic element in the shape of a rectangle with a different color from the background area. The background area may include the remaining area of ​​the first calibration image excluding the marker. However, this is not limited to this, and the shape of the marker may vary.

[0108] The processor (140) can acquire images by photographing the first calibration images projected by the robot (100) and another robot using the camera (120). For example, referring to FIG. 5, the robot (100) can photograph the first calibration image (510) projected by the robot (100) and the first calibration image (520) projected by the other robot (100-2), thereby acquiring images in which the first calibration images (510, 520) are photographed.

[0109] In operation S330, the processor (140) can identify the positions of markers of each of the first calibration image projected by the robot (100) and the first calibration image projected by another robot based on the image acquired using the camera (120).

[0110] For example, the processor (140) can identify areas corresponding to the first calibration images in an image acquired using the camera (120) and identify markers included in the areas.

[0111] The area corresponding to the first calibration image in the image acquired using the camera (120) may be an area including the first calibration image in the image acquired using the camera (120).

[0112] According to one example, pixels corresponding to four vertices of the first calibration image projected by the robot (100) and another robot, respectively, may include QR codes. The processor (140) may recognize the four QR codes in the image acquired using the camera (120) and identify the area defined by the four QR codes as the area corresponding to the first calibration image.

[0113] In one example, the processor (140) can identify an area in the image corresponding to the first calibration image based on the color values ​​(e.g., RGB values, etc.) of the image. The color of the first calibration image may be different from the color of the projection surface. For example, the processor (140) can identify the color of the projection surface on which the first calibration image is to be projected using the camera (120) and project the first calibration image having a color different from the color of the projection surface onto the projection surface. Another robot can also use the same method to project the first calibration image having a color different from the color of the projection surface onto the projection surface. In this case, the processor (140) can capture the first calibration images projected by the robot (100) and other robots using the camera (120), and identify an area in the image having a color that is distinct from other areas based on the color values ​​of the captured images as an area corresponding to the first calibration image.

[0114] Meanwhile, the processor (140) can identify two areas corresponding to the first calibration image in the image captured by the camera (120). In this case, the processor (140) can distinguish between the area corresponding to the first calibration image projected by the robot (100) and the area corresponding to the first calibration image projected by another robot among the two areas.

[0115] In one example, the processor (140) can identify the direction of another robot by searching the surroundings using the camera (120) before projecting the first calibration image. Then, based on the direction of the other robot, the processor (140) can identify, among two areas, an area corresponding to the first calibration image projected by the robot (100) and an area corresponding to the first calibration image projected by the other robot.

[0116] For example, when another robot is located to the right of the robot (100), the processor (140) can identify that the left area among the two areas corresponds to the first calibration image projected by the robot (100), and that the right area among the two areas corresponds to the first calibration image projected by the other robot.

[0117] For example, when another robot is located to the left of the robot (100), the processor (140) can identify that among the two regions, the left region corresponds to the first calibration image projected by the other robot, and among the two regions, the right region corresponds to the first calibration image projected by the robot (100).

[0118] For example, a QR code may include identification information of a robot. The processor (140) may obtain identification information from an image captured by a camera (120) through QR code recognition, and based on the identification information, may identify an area corresponding to a first calibration image projected by the robot (100) and an area corresponding to a first calibration image projected by another robot among two areas.

[0119] For example, the marker may be a rectangular graphic element. In this case, the colors of the markers projected by the robot (100) and the other robot may be different. The processor (140) identifies the color value of the marker based on the image captured by the camera (120), and based on the color value of the marker, identifies the area corresponding to the first calibration image projected by the robot (100) and the area corresponding to the first calibration image projected by the other robot among two areas.

[0120] The processor (140) can identify markers in areas corresponding to the first calibration images and identify the locations of the markers.

[0121] For example, the marker may be a QR code. The processor (140) can identify the marker in an area corresponding to the first calibration image through QR code recognition.

[0122] For example, a marker may include a rectangular graphic element. The processor (140) may recognize an area corresponding to the first calibration image that has a different color from the background area and has a rectangular shape as a marker.

[0123] The position of the marker may include the degree to which the marker has moved in the first calibration image.

[0124] For example, referring to FIG. 6, the x,y coordinate values ​​of the pixel (611) corresponding to the upper left vertex of the area (610) corresponding to the first calibration image projected by the robot (100) in the image (600) acquired using the camera (120) are (x a1 , y a1 ), and the x,y coordinate values ​​of the pixel (612) corresponding to the upper right corner of the area (610) are (x a2 , y a1 ) is. And, the x,y coordinate values ​​of the pixel (631) corresponding to the upper left corner of the marker (630) are (x a3 , y a3 )am.

[0125] In this case, the processor (140) positions the marker (630) at (x a3 -x a1 ) / (x a2 -x a1 ) can be calculated as follows.

[0126] Also, referring to FIG. 6, the x,y coordinate values ​​of the pixel (621) corresponding to the upper left vertex of the area (620) corresponding to the first calibration image projected by another robot in the image (600) acquired using the camera (120) are (x a4 , y a4), and the x,y coordinate values ​​of the pixel (622) corresponding to the upper right corner of the area (620) are (x a5 , y a4 ) is. And, the x,y coordinate values ​​of the pixel (641) corresponding to the upper left corner of the marker (640) are (x a6 , y a6 )am.

[0127] In this case, the processor (140) positions the marker (640) at (x a6 -x a4 ) / (x a5 -x a4 ) can be calculated as follows.

[0128] In operation S340, the processor (140) can identify the delay time based on the positions of the markers.

[0129] The delay time may include the difference between the playback time of an image projected from a robot (100) and the playback time of an image projected from another robot.

[0130] For example, even if a robot (100) and another robot start playing an image at the same time, the playing times of the images played by the robots may be different from each other due to factors such as the specifications of the robots, the system load, or the communication environment. If the playing times of the images projected by the robots are different, it becomes impossible to implement a single, natural, large-screen image using the images projected by the robots. In the present disclosure, in order to match the playing times of the images projected by the robots, the delay time is identified using the first calibration images, and the playing time of the image to be projected by the robot (100) is adjusted based on the identified delay time.

[0131] For example, the processor (140) can identify the playback time of the first calibration image played by the robot (100) based on the position of the marker in the first calibration image projected by the robot (100). In addition, the processor (140) can identify the playback time of the first calibration image played by another robot based on the position of the marker in the first calibration image projected by the other robot.

[0132] In one example, information about the playback point in time according to the position of the marker can be stored in the memory (150).

[0133] Since the marker in the first calibration image moves over time, the degree to which the marker moves in the first calibration image can correspond to the playback time of the first calibration image.

[0134] For example, information about multiple playback points of the first calibration image corresponding to multiple degrees of movement of the marker can be stored in the memory (150).

[0135] The processor (140) can identify a playback point corresponding to a position of a marker in a first calibration image projected by the robot (100) among a plurality of playback points corresponding to a plurality of movement degrees of the marker, and identify the identified playback point as a playback point of the first calibration image projected by the robot (100). In addition, the processor (140) can identify a playback point corresponding to a position of a marker in a first calibration image projected by another robot among a plurality of playback points corresponding to a plurality of movement degrees of the marker, and identify the identified playback point as a playback point of the first calibration image projected by the other robot.

[0136] The processor (140) can identify the difference between the identified playback points as a delay time.

[0137] As an example, let us assume that the playback time of the first calibration image projected by a robot (100) is n seconds and the playback time of the first calibration image projected by another robot is m seconds. In this case, n <m이고, n,m은 자연수이다. 이 경우, 프로세서(140)는 로봇(100)에 의해 투사된 제1 캘리브레이션 영상의 재생 시점이 다른 로봇에 의해 투사된 캘리브레이션 영상의 제1 재생 시점보다 m-n초 느리며, 딜레이 시간은 m-n초인 것으로 식별할 수 있다.

[0138] As an example, let us assume that the playback time of the first calibration image projected by a robot (100) is m seconds and the playback time of the first calibration image projected by another robot is n seconds. In this case, n <m이고, n,m은 자연수이다. 이 경우, 프로세서(140)는 로봇(100)에 의해 투사된 제1 캘리브레이션 영상의 재생 시점이 다른 로봇에 의해 투사된 제1 캘리브레이션 영상의 재생 시점보다 m-n초 빠르며, 딜레이 시간은 m-n초인 것으로 식별할 수 있다.

[0139] In operation S350, the processor (140) can project an image based on a delay time.

[0140] In this case, the video may include various video contents such as television programs, movies, or dramas. In addition, the video may include various information such as weather, time, etc., or various information and advertising videos related to services provided within the space where the robot (100) is located.

[0141] In one example, the processor (140) can receive user input for displaying an image.

[0142] User input can be entered via a mobile device. The user can input user input for displaying an image on the mobile device using an application installed on the mobile device. In this case, the user input for displaying the image may include user input for setting at least one of the position of the projection surface, the size of the image, and the aspect ratio of the image.

[0143] For example, referring to FIG. 7, the mobile device (200) can display a UI screen (700) for setting a projection surface using an application.

[0144] The UI screen (700) may include a GUI (graphical user interface) for setting the position of the projection surface, the size of the image, and the aspect ratio of the image.

[0145] For example, a GUI (710) for setting the position of the projection surface may include a map of the space where the robot (100) is located. A user may input a user input for selecting a wall on the map to set the position of the projection surface on which the image will be projected.

[0146] Additionally, the GUI (720) for setting the size of the image may include multiple selection items. The user may input a user input for selecting one of the multiple selection items to set the size of the image (e.g., 65 inches in FIG. 7).

[0147] Additionally, the GUI (730) for setting the aspect ratio of the image may include multiple selection items. The user may input a user input to select one of the multiple selection items to set the aspect ratio of the image (e.g., 32:9 in FIG. 7).

[0148] The server can transmit control commands to the robot (100) and other robots to display images based on user input entered into the mobile device. Additionally, if the mobile device communicates directly with the robot (100) and other robots, the mobile device can directly transmit control commands to the robot (100) and other robots to display images based on user input.

[0149] The processor (140) can control the driving unit (130) to move the robot (100) to a preset location within the map based on the control command. Other robots can also move to a preset location within the map based on the control command.

[0150] The preset position may be a position corresponding to the projection surface on which the image is to be projected. The position corresponding to the projection surface on which the image is to be projected may be determined based on at least one of the position of the projection surface, the size of the image, and the aspect ratio of the image, set according to user input.

[0151] For example, the memory (150) may store information about a location within a map corresponding to the location of the projection surface, the size of the image, and the aspect ratio of the image. In this case, the location may be expressed by a coordinate value.

[0152] For example, assume that a robot (100) projects an image onto a wall based on an aspect ratio set according to user input. In this case, the size of the image projected onto the wall may vary depending on the distance between the wall and the robot (100). For example, the closer the distance between the robot (100) and the wall, the smaller the size of the image projected onto the wall by the robot (100), and the farther the distance between the robot (100) and the wall, the larger the size of the image projected onto the wall by the robot (100).

[0153] Taking these points into account, a suitable position for the robot (100) to project an image onto the projection surface can be preset based on the position of the projection surface, the size of the image, and the aspect ratio of the image. Information regarding the position within this map can be stored in the memory (150).

[0154] The processor (140) can control the driving unit (130) to move the robot (100) to a position corresponding to the projection surface based on the position information stored in the memory (150). Other robots can also move to a position corresponding to the projection surface within the map using the same method. When the robot (100) and the other robot move to the corresponding positions, the robot (100) and the other robot can be in adjacent positions.

[0155] The processor (140) can control the projector (110) to project an image onto the projection surface based on a delay time while the robot (100) is at a position corresponding to the projection surface on which the image is to be projected.

[0156] For example, the processor (140) can adjust the playback time of an image to be projected by the robot (100) based on the delay time, and project the image with the adjusted playback time onto a projection surface using the projector (110).

[0157] According to one example, if the processor (140) identifies that the playback time of the calibration image projected by the robot (100) is mn seconds slower than the playback time of the calibration image projected by another robot, the processor (140) can adjust the playback time of the image to be projected by the robot (100) to be mn seconds faster. For example, if the playback time of the image to be projected by the robot (100) is t seconds, the processor (140) can adjust the playback time of the image to t+(mn) seconds.

[0158] According to one example, if the processor (140) identifies that the playback time of the calibration image projected by the robot (100) is mn seconds earlier than the playback time of the calibration image projected by another robot, the processor (140) may adjust the playback time of the image to be projected by the robot (100) to be mn seconds later. For example, if the processor (140) identifies that the playback time of the image to be projected by the robot (100) is t seconds, the processor (140) may adjust the playback time of the calibration image to t-(mn) seconds.

[0159] Meanwhile, the images projected by the robot (100) and the other robots may be portions of the entire image. In addition, the aspect ratios of the images projected by the robot (100) and the other robots may be determined based on the aspect ratios of the images set according to user input. For example, assume that the aspect ratio of the images set according to user input is a:b. In this case, the processor (140) may control the projector (110) to project the images at an aspect ratio of a / 2:b.

[0160] Accordingly, referring to FIG. 8, the playback time of images (810, 820) projected by the robot (100) and another robot (100-2) is made to match, and a natural large-screen image can be provided to the user.

[0161] Meanwhile, in the above-described example, it was described that the robot (100) projects the first calibration image using a projector (110), but it is not limited thereto.

[0162] For example, the robot (100) may not project the first calibration image. In this case, the robot (100) may use the camera (120) to capture the first calibration image projected by another robot, identify the position of the marker in the first calibration image, and identify the playback time of the first calibration image based on the position of the marker. In addition, the robot (100) may play the first calibration image according to a user input for time synchronization, and identify the playback time of the first calibration image played by the robot (100) based on the time at which the first calibration image projected by the other robot was captured using the camera (120). In addition, the robot (100) may identify the delay time based on the identified playback time points.

[0163] In addition, although the aforementioned example described that the position where the robot (100) projects the first calibration image and the position where the image is projected are different, the present invention is not limited thereto. For example, when a user input for displaying an image is received, the robot (100) and another robot may move to a position corresponding to the projection surface and project the first calibration image, respectively. The robot (100) may identify a playback point in time using the first calibration images projected by the robot (100) and another robot, and may project an image based on the playback point in time.

[0164] As described above, the robot (100) and other robots can each move to a position corresponding to the projection surface on which the image is to be projected in order to project the image.

[0165] In this case, the robots can explore their surroundings to identify their own locations and move based on the identified locations. If an error occurs, the robot (100) and / or other robots may not be able to move to the correct location. Furthermore, if the robots project images onto the projection surface at inaccurate locations, the projected images may not be aligned with each other. For example, the projected images may be of different sizes, or may overlap or be spaced apart. In this case, it becomes impossible to create a single, natural, large-screen image using the images projected by the robots.

[0166] Therefore, in the present disclosure, the robot (100) can perform fine tuning while the robot (100) is at a position corresponding to the projection surface on which the image is to be projected, and project the image onto the projection surface based on the delay time.

[0167] Fine tuning may include controlling the movement of the robot (100) so that images projected by the robot (100) and other robots can be aligned with each other on the projection surface. In this case, the operation of controlling the movement of the robot (100) may include moving the robot (100) or moving the body of the robot (100). When the body of the robot (100) rotates, the projector (110) and camera (120) provided on the body may also rotate together.

[0168] FIG. 9 is a flowchart illustrating an operation of a robot performing fine adjustment according to an embodiment of the present disclosure.

[0169] Hereinafter, the calibration images for fine adjustment projected by the robot (100) and other robots are described as second calibration images.

[0170] In steps S910 and S920, the processor (140) can project a second calibration image using a projector (110). Another robot can also project a second calibration image. In addition, the processor (140) can capture the second calibration image projected by the robot (100) and the second calibration image projected by another robot using a camera (120) to acquire an image.

[0171] The second calibration image may include markers. The markers may not move in the second calibration image and may be positioned at fixed locations. The sizes of the markers in the second calibration images projected by the robot (100) and other robots may be identical.

[0172] In the second calibration image, the position of the marker can be determined based on the position of the robot (100).

[0173] For example, a robot (100) may be positioned to the left of another robot. At this time, in the second calibration image projected by the robot (100), the marker may be positioned in the central region of the right side of the second calibration image. That is, the right side of the marker may be positioned on the right side of the second calibration image, and the center of the right side of the marker may be positioned at the center of the right side of the second calibration image.

[0174] In this case, in the second calibration image projected by another robot, the marker may be located in the central region of the left part of the second calibration image. The left side of the marker may be located on the left side of the second calibration image, and the center of the left side of the marker may be located at the center of the left side of the second calibration image.

[0175] For example, a robot (100) may be positioned to the right of another robot. At this time, in the second calibration image projected by the robot (100), the marker may be positioned in the central region of the left part of the second calibration image. That is, the left side of the marker may be positioned on the left side of the second calibration image, and the center of the left side of the marker may be positioned at the center of the left side of the second calibration image.

[0176] In this case, in the second calibration image projected by another robot, the marker may be located in the central region of the right side of the second calibration image. The right side of the marker may be located on the right side of the second calibration image, and the center of the right side of the marker may be located at the center of the right side of the second calibration image.

[0177] The marker may include a visual graphic element that can be recognized by the robot (100) and other robots.

[0178] For example, the marker could be a QR code.

[0179] For example, a marker may be a graphic element in the shape of a rectangle with a different color from the background area. The background area may include the remaining area of ​​the second calibration image excluding the marker. However, this is not limited to this, and the shape of the marker may vary.

[0180] Additionally, the aspect ratio of the second calibration image projected by the robot (100) and the other robots can be determined based on the aspect ratio of the image set according to the user input. For example, assume that the aspect ratio of the image set according to the user input is a:b. In this case, the processor (140) can control the projector (110) to project the second calibration image with an aspect ratio of a / 2:b. The other robots can also project the second calibration image with an aspect ratio of a / 2:b.

[0181] For example, referring to FIG. 10, the robot (100) and another robot can each project second calibration images (1010, 1020). The robot (100) can capture the second calibration image (1010) projected by the robot (100) and the second calibration image (1020) projected by the other robot (100-2), thereby obtaining images in which the second calibration images (1010, 1020) are captured.

[0182] Thereafter, the processor (140) can control the movement of the robot using the driving unit (130) based on the image acquired using the camera (120) so that the second calibration image projected by the robot (100) and the second calibration image projected by another robot are aligned with each other.

[0183] In this case, the alignment of the second calibration images projected by the robot (100) and another robot may include the sizes of the second calibration images being the same on the projection surface, the second calibration images being arranged side by side without overlapping or being spaced apart from each other.

[0184] For example, the processor (140) can control the movement of the robot (100) using image-based visual servoing.

[0185] Image-based visual servoing may include a control method that controls the robot's movements to reduce the error between a specific location in the image and a target location, thereby ensuring that the specific location is positioned at the target location. This will be described in more detail in operations S930 and S940.

[0186] In operation S930, the processor (140) can identify a target area corresponding to a marker of a second calibration image projected by the robot (100) based on an image acquired using the camera (120).

[0187] For example, the processor (140) can identify a marker of a second calibration image projected by the robot (100) in an image acquired using the camera (120).

[0188] The method for identifying an area corresponding to the second calibration image in an image acquired using a camera (120) and the method for identifying a marker in an area corresponding to the second calibration image are the same as those described for the first calibration image.

[0189] And, the processor (140) can identify a symmetrical area of ​​the marker as a target area based on one side of the marker.

[0190] One side of the marker may be a side located on one side of the second calibration image among the plurality of sides of the marker. The processor (140) may identify the x, y coordinate values ​​of the pixels corresponding to the plurality of vertices of the target area based on the x, y coordinate values ​​of the pixels corresponding to the plurality of vertices of the marker of the second calibration image projected by the robot (100), thereby identifying the target area.

[0191] For example, referring to FIG. 11, the x,y coordinate values ​​of the pixels corresponding to the four vertices of the marker (1120) in the area (1110) corresponding to the second calibration image projected by the robot (100) in the image (1100) acquired using the camera (120) are (x b1 , y b1 ), (x b2 , y b1 ), (x b3 , y b3 ), (x b4 , y b3 ) is. In this case, the symmetrical area (1140) of the marker (1120) based on one side (1130) of the marker (1120) may be the target area. The x, y coordinate values ​​of the pixels corresponding to the four vertices of the target area (1140) are (x b5 , y b1 ), (x b6 , yb1 ), (x b7 , y b3 ), (x b8 , y b3 ) is. Here, x b6 =x b2 +(x b2 -x b1 ), x b8 =x b4 +(x b4 -x b3 )am.

[0192] In operation S940, the processor (140) can control the movement of the robot (100) using the driving unit (130) so that the marker of the second calibration image projected by another robot is located in the target area in the image acquired using the camera (120).

[0193] Positioning a marker in a second calibration image projected by another robot in a target area may include positioning vertices of the marker at corresponding vertices of the target area.

[0194] For example, when the robot (100) moves toward the projection surface (e.g., when the robot (100) moves forward), the size of the marker of the second calibration image projected by another robot in the image captured by the camera (120) may increase, and when the robot (100) moves to the opposite side of the projection surface (e.g., when the robot (100) moves backward), the size of the marker of the second calibration image projected by another robot in the image captured by the camera (120) may decrease.

[0195] Additionally, when the robot (100) moves to the left, the marker of the second calibration image projected by another robot in the image captured by the camera (120) may move to the right, and when the robot (100) moves to the right, the marker of the second calibration image projected by another robot in the image captured by the camera (120) may move to the left.

[0196] In addition, when the body of the robot (100) rotates, the projector (110) and camera (120) provided on the body may also rotate together. In this case, when the body of the robot (100) rotates upward, the marker of the second calibration image projected by another robot in the image captured by the camera (120) may move downward, and when the body of the robot (100) rotates downward, the marker of the second calibration image projected by another robot in the image captured by the camera (120) may move upward.

[0197] Taking this into consideration, the processor (140) can control the movement of the robot (100) using the driving unit (130) so that the marker of the second calibration image projected by another robot is positioned in the target area in the image acquired using the camera (120).

[0198] For example, the processor (140) can control the driving unit (120) to move the robot (100) forward or backward so that the size of the marker in the second calibration image projected by another robot in the image captured by the camera (120) becomes the same as the size of the target area.

[0199] In addition, the processor (140) can control the driving unit (120) to move the robot (100) to the left or right, or rotate the body of the robot (100) in an upward or downward direction so that the marker of the second calibration image projected by another robot in the image captured by the camera (120) does not overlap or is spaced apart from the marker of the second calibration image projected by the robot (100) and is positioned in the target area.

[0200] The processor (140) can repeatedly control the movement of the robot (100) using the driving unit (120) so as to reduce the error between the position of the marker of the second calibration image projected by another robot in the image captured by the camera (120) and the position of the target area. Accordingly, the marker of the second calibration image projected by another robot in the image captured by the camera (120) can be positioned in the target area.

[0201] For example, referring to FIGS. 12A and 12B, the robot (100) can control the movement of the robot (100) to position a marker (1160) of an area (1150) corresponding to a second calibration image projected by another robot in an image (1100) acquired using a camera (120) in a target area (1140). In this case, the upper left vertex, the upper right vertex, the lower left vertex, and the lower right vertex of the marker (1160) can be positioned at the upper left vertex, the upper right vertex, the lower left vertex, and the lower right vertex of the target area (1140), respectively.

[0202] Accordingly, the second calibration images (1210, 1220) projected by the robot (100) and another robot (100-2) have the same size on the projection surface and can be arranged side by side without overlapping or being spaced apart from each other.

[0203] In operation S950, the processor (140) can control the projector (110) to project an image based on a delay time after the movement of the robot (100) is controlled.

[0204] The method of projecting an image based on the delay time is the same as described in the first calibration image.

[0205] Accordingly, referring to FIG. 13, images (1310, 1320) projected onto the projection surface by the robot (100) and another robot (100-2) have the same size and can be arranged in parallel. In addition, the playback timings of the images (1310, 1320) can be synchronized. Accordingly, a single, natural, large-screen image can be provided to the user.

[0206] Various embodiments of the present disclosure may be implemented in a computer-readable recording medium using software, hardware, or a combination thereof, or a computer or similar device. In some cases, the embodiments described herein may be implemented by the processor itself. In a software implementation, embodiments, such as the procedures and functions described herein, may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

[0207] Meanwhile, computer instructions for performing processing operations of an electronic device according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. When the computer instructions stored in such a non-transitory computer-readable medium are executed by a processor of a specific device, the computer instructions cause the specific device to perform processing operations in a robot (100) according to various embodiments described above.

[0208] A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of non-transitory computer-readable media include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0209] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In robots, projector; camera; driving unit; and Using the above projector, a first calibration image of the marker moving is projected, Using the above camera, an image is acquired by capturing a first calibration image projected by the robot and a first calibration image projected by another robot, Based on the acquired image, the positions of markers of each of the first calibration image projected by the robot and the first calibration image projected by the other robot are identified, Identifying a delay time for adjusting the playback time of an image to be projected by the robot based on the identified position; A robot comprising one or more processors that control the projector to project the image onto the projection surface based on the delay time while the robot is at a position corresponding to the projection surface on which the image is to be projected.

2. In paragraph 1, The first calibration image projected by the other robot is an image of the marker moving.

3. In paragraph 1, One or more of the above processors, Identifying the playback point of the first calibration image projected by the robot based on the position of the marker of the first calibration image projected by the robot, Identifying the playback point of the first calibration image projected by the other robot based on the position of the marker of the first calibration image projected by the other robot, A robot that identifies the difference between the identified playback points as the delay time.

4. In paragraph 3, One or more of the above processors, Among the plurality of playback points corresponding to the plurality of movement degrees of the marker, a playback point corresponding to the position of the marker of the first calibration image projected by the robot is identified, and the identified playback point is identified as the playback point of the first calibration image played back by the robot. A robot that identifies a playback point corresponding to a position of a marker in a first calibration image projected by the other robot among a plurality of playback points corresponding to a plurality of movement degrees of the marker, and identifies the identified playback point as a playback point of the first calibration image played back by the other robot.

5. In paragraph 1, A marker included in a first calibration image projected by each of the above robot and the other robot includes a QR code.

6. In paragraph 1, One or more of the above processors, While the robot is at a position corresponding to the projection surface on which the image is to be projected, the second calibration image is projected using the projector, Using the above camera, an image is acquired by capturing a second calibration image projected by the robot and a second calibration image projected by the other robot, Controlling the movement of the robot using the driving unit based on the acquired image so that the second calibration image projected by the robot and the second calibration image projected by the other robot are aligned with each other, A robot that controls the projector to project the image based on the delay time after the movement of the robot is controlled.

7. In paragraph 6, One or more of the above processors, Identifying a target area corresponding to a marker of a second calibration image projected by the robot based on the acquired image, A robot that controls the movement of the robot using the driving unit so that the marker of the second calibration image projected by the other robot in the acquired image is positioned in the target area.

8. In paragraph 7, A robot in which a marker included in a second calibration image projected by each of the above robot and the other robot includes a QR code positioned at a fixed position.

9. In paragraph 1, The position corresponding to the above projection surface is, A robot whose position is determined based on at least one of the position of the projection surface, the size of the image, and the aspect ratio of the image, set according to user input.

10. A method for projecting an image of a robot including a projector and a camera, A step of projecting a first calibration image in which a marker is moved using the above projector; A step of obtaining an image by capturing a first calibration image projected by the robot and a first calibration image projected by another robot using the above camera; A step of identifying the positions of markers of each of the first calibration image projected by the robot and the first calibration image projected by the other robot based on the acquired image; A step of identifying a delay time for adjusting the playback time of an image to be projected by the robot based on the identified position; and An image projection method comprising: a step of controlling the projector to project the image onto the projection surface based on the delay time while the robot is at a position corresponding to the projection surface on which the image is to be projected; 11. In paragraph 10, A method of projecting an image in which the first calibration image projected by the other robot is an image in which the marker moves.

12. In paragraph 10, The step of identifying the above delay time is: A step of identifying a playback point in time of a first calibration image projected by the robot based on a position of a marker in the first calibration image projected by the robot; A step of identifying a playback point of a first calibration image projected by the other robot based on a position of a marker of the first calibration image projected by the other robot; and An image projection method comprising: a step of identifying the difference between the identified playback points as the delay time; 13. In paragraph 12, The step of identifying the playback point of the first calibration image projected by the above robot is: A step of identifying a playback point corresponding to the position of the marker of the first calibration image projected by the robot among a plurality of playback points corresponding to a plurality of movement degrees of the marker; and A step of identifying the identified playback point as the playback point of the first calibration image played by the robot; The step of identifying the playback point of the first calibration image projected by the other robot is, A step of identifying a playback point corresponding to the position of the marker of the first calibration image projected by the other robot among a plurality of playback points corresponding to a plurality of movement degrees of the marker; and An image projection method comprising: a step of identifying the identified playback point as a playback point of a first calibration image played back by the other robot.

14. In paragraph 10, A method for projecting an image, wherein a marker included in a first calibration image projected by each of the above robot and the other robot includes a QR code.

15. In paragraph 10, A step of projecting a second calibration image using the projector while the robot is at a position corresponding to the projection surface on which the image is to be projected; A step of obtaining an image by capturing a second calibration image projected by the robot and a second calibration image projected by the other robot using the camera; A step of controlling the movement of the robot based on the acquired image so that the second calibration image projected by the robot and the second calibration image projected by the other robot are aligned with each other; and An image projection method comprising: a step of controlling the projector to project the image based on the delay time after the movement of the robot is controlled;

Images