Mobile device and mobile device control method

Abstract

A mobile device for simulating installation of a real image device according to one embodiment includes a display, a camera for photographing an image of a space in which the image device can be set, a user interface for receiving a user input, and a control unit that identifies a plane from the image of the space photographed by the camera, controls the display to display a grid on a virtual plane corresponding to the plane, adjusts the grid based on the user input, and controls the display to display a virtual image of the image device on the grid.

Description

Technical Field

[0001] This disclosure relates to a mobile device capable of using augmented reality technology to simulate the installation of an imaging device. This disclosure also relates to a method for controlling the mobile device. Background Technology

[0002] Mobile devices are devices that provide various functions based on mobility and are widely used in various fields. Mobile devices include various functional modules that provide a range of functions. For example, a mobile device may include a camera module that provides image acquisition capabilities.

[0003] Augmented reality (AR) technology exists as a technology that utilizes the camera modules of mobile devices. AR technology refers to a graphics technique in which virtual objects or information are overlaid on the real environment, making the objects appear to exist in the real environment.

[0004] Unlike existing virtual reality technologies that realize virtual spaces and virtual objects, augmented reality (AR) overlays virtual objects onto real space. Therefore, it can supplement and provide additional information that is difficult to obtain independently in the real world. AR can be applied to a variety of real-world environments and is specifically viewed as a next-generation display technology applicable to ubiquitous environments. Summary of the Invention

[0005] Other aspects will be set forth in part in the description which follows, and in part will be apparent from the description, or may be learned by practice of exemplary embodiments.

[0006] A mobile device according to one or more examples of this disclosure includes a display, a camera configured to capture an image of a space where an imaging device will be mounted, a user interface, and a controller configured to recognize a plane from the image captured by the camera, control the display to display a grid on a virtual plane corresponding to the plane, adjust the grid based on user input received via the user interface, and control the display to display a virtual image of the imaging device on the grid.

[0007] The controller can adjust the grid by moving the grid based on user input received via the user interface.

[0008] The controller can adjust the grid by rotating the grid based on user input received via the user interface.

[0009] The controller can control the display to show both the guide lines and spatial images captured by the camera, and identify the plane based on the guide lines.

[0010] The controller can control the display to show the guide lines as either base guide lines or corner guide lines.

[0011] The controller can control the display to show a virtual image of the imaging device in a designated area of ​​the grid based on user input received via the user interface to select a designated area of ​​the grid.

[0012] The controller can control the display to rotate the virtual image of the image device on a virtual plane based on user input received via the user interface.

[0013] The controller can control the display to further display text, which includes at least one of the following: the model name of the image device corresponding to the virtual image of the image device, its size, resolution, and distance from the floor.

[0014] The controller can display a menu bar that allows selection of at least one of the image device's category, model, and size, and controls the display to show a virtual image of the image device based on user input received via the user interface from the menu bar.

[0015] When multiple virtual images from multiple image devices are displayed, the controller can completely or partially delete multiple virtual images from multiple image devices based on user input received via the user interface to delete one or more of the multiple virtual images from multiple image devices.

[0016] The controller can control the display to show the content on the area of ​​the virtual image displayed on the image display device based on user input of the selected content received via the user interface.

[0017] The controller can store simulation results related to the installation of the imaging device as at least one of video and screenshot images.

[0018] A method for controlling a mobile device according to an example of this disclosure includes: capturing an image of a space in which an imaging device is permitted to be installed via a camera; identifying a plane from the image captured by the camera; displaying a grid on a virtual plane corresponding to the plane via a display; adjusting the grid based on user input received via a user interface; and displaying a virtual image of the imaging device on the grid.

[0019] In this method, adjusting the grid may include moving the grid based on user input received via a user interface.

[0020] In this method, adjusting the mesh may include rotating the mesh orientation based on user input received via a user interface.

[0021] The method may also include displaying the guide line and a spatial image captured by the camera together on a display, wherein identifying the plane may include identifying the plane based on the guide line.

[0022] Displaying guide lines together with spatial images captured by the camera can include displaying guide lines as base guide lines or corner guide lines.

[0023] The virtual image displayed by the imaging device may include displaying the virtual image of the imaging device in the designated area of ​​the grid based on user input received via the user interface.

[0024] The method may further include receiving user input via a user interface to rotate a virtual image of the image device, rotating the virtual image of the image device on a virtual plane based on the user input received via the user interface, and displaying the rotated virtual image of the image device on a grid.

[0025] The method may also include displaying text that includes at least one of the following: the model name of the image device corresponding to the virtual image of the image device, its size, resolution, and distance from the floor.

[0026] The method may also include displaying a menu bar that enables selection of at least one of the category, model, and size of the imaging device, wherein displaying a virtual image of the imaging device may include displaying a virtual image of the imaging device based on user input received via a user interface from the menu bar.

[0027] The method may also include, when displaying multiple virtual images of multiple image devices, deleting multiple virtual images of multiple image devices completely or partially based on user input received via a user interface to delete one or more of the multiple virtual images of multiple image devices.

[0028] The method may also include displaying the selected content on an area of ​​a virtual image on a displaying image device based on user input received via a user interface.

[0029] The method may also include storing simulation results related to the installation of the imaging device as at least one of video and screenshot images. Attached Figure Description

[0030] The above and other aspects, features, and advantages of exemplary embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0031] Figure 1 This is a control block diagram for a mobile device.

[0032] Figure 2 yes Figure 1 The diagram shows a detailed block diagram of the controller.

[0033] Figures 3 to 5 This is a flowchart illustrating a method for controlling a mobile device.

[0034] Figure 6 A method for initiating a simulation related to the installation of an imaging device is shown.

[0035] Figures 7 to 9 A method is shown for identifying planes and displaying grids from images of spaces where imaging devices can be mounted.

[0036] Figures 10 to 12 The method for adjusting the grid is shown.

[0037] Figure 13 The pattern of changing the grid is shown.

[0038] Figures 14 to 16 An implementation of displaying a virtual image of an image device on a grid is shown.

[0039] Figure 17 A virtual image of a rotating and displaying image device is shown.

[0040] Figure 18 Multiple virtual images are shown on the display image device.

[0041] Figures 19 to 23 An implementation of a virtual image display device is shown.

[0042] Figure 24 This shows how to show or hide the grid.

[0043] Figure 25 and Figure 26 A virtual image of the image device is shown.

[0044] Figures 27 to 29 This illustrates playing content on an area of ​​a virtual image displayed on an image display device. Detailed Implementation

[0045] The embodiments disclosed herein and the components shown in the accompanying drawings are merely examples, and various modifications can be made to replace the embodiments and drawings of this disclosure. The terminology used herein is for illustrative purposes and is not intended to limit or restrict this disclosure.

[0046] As used herein, terms such as “unit” can be implemented as software or hardware. Depending on the implementation, multiple units may be implemented by a single unit or component, or a single unit may include multiple components. Furthermore, terms described herein, such as “unit,” “block,” “component,” and “module,” can refer to a unit that performs at least one function or operation. For example, the term can refer to hardware (e.g., circuitry), software stored in memory, or a program processed by a processor.

[0047] As used herein, singular expressions may include plural expressions unless otherwise stated in the context. Furthermore, terms such as “first” and “second” as used herein are used to distinguish one part from another, and unless otherwise stated, they do not imply sequential representation.

[0048] Terms such as “comprising” and “having” are intended to indicate the presence of features, figures, steps, operations, components, parts or combinations thereof disclosed in the specification, and are not intended to exclude the possibility that one or more other features, figures, steps, operations, components, parts or combinations thereof may be present or may be added.

[0049] In addition, terms such as first, second, etc., are used to distinguish one component from another, and these components are not limited by these terms.

[0050] When an item is described using the connective term “and / or”, the description should be understood to include any and all combinations of one or more of the related listed items. That is, the term “and / or” includes multiple combinations of related items or any one of multiple related items. For example, the scope of the expression or phrase “A and / or” includes all of the following: (1) item “A”, (2) item “B”, and (3) a combination of items “A and B”.

[0051] Furthermore, the scope of the expression or phrase “at least one of A and B” is intended to include all of the following cases: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, the scope of the expression or phrase “at least one of A, B and C” is intended to include all of the following cases: (1) at least one A, (2) at least one B, (3) at least one C, (4) at least one A and at least one B, (5) at least one A and at least one C, (6) at least one B and at least one C, and (7) at least one A, at least one B and at least one C.

[0052] When an element is described in this disclosure as being “connected to” or “coupled to” another element, the statement includes examples of direct connection or coupling as well as connections or couplings in which another element is inserted.

[0053] In the following, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. The same reference numerals or symbols given in the drawings denote parts or components that perform substantially the same function.

[0054] The examples disclosed herein are intended to provide a mobile device and a method for controlling the mobile device, which enables a user to realistically simulate the installation of an imaging device by displaying a virtual image corresponding to the actual size of the imaging device along with an image of the real space in which the imaging device will be installed.

[0055] Based on the mobile device and the method for controlling the mobile device based on the examples disclosed herein, a user can realistically simulate the installation of an imaging device by displaying a virtual image corresponding to the actual size of the imaging device, along with an image of the real space in which the imaging device will be installed.

[0056] Furthermore, based on the mobile device and the method for controlling the mobile device based on the examples disclosed herein, a more realistic simulation can be performed by playing content on an area displaying a virtual image corresponding to the actual size of the image device.

[0057] Figure 1 This is a control block diagram of a mobile device according to an implementation method, and Figure 2 yes Figure 1 The diagram shows a detailed block diagram of the controller.

[0058] First, according to one embodiment, the mobile device 1 is a portable device and includes a device capable of providing augmented reality. For example, the mobile device 1 can be implemented as a device capable of providing augmented reality by including a camera, such as a mobile phone, portable multimedia player (PMP), digital radio player, personal digital assistant (PDA), music file player (e.g., MP3 player), portable gaming terminal, tablet PC, and smartphone. In the following description, for ease of description, it is assumed that the mobile device 1 is a smartphone.

[0059] Reference Figure 1 The mobile device 1 according to the embodiment may include a camera 100, a user interface 210, a microphone 220, an output unit 300, a communicator 400, a storage device 500, and a controller 600.

[0060] Camera 100 can capture images of a space where an imaging device can be mounted. Camera 100 can be a camera module mounted on a mobile device 1 and can include a lens, a charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) image sensor, and an analog-to-digital converter. Camera 100 converts the image obtained by capturing an object into a digital signal and sends the digital signal to controller 600. Controller 600, described later, can perform processing on the converted digital image signal.

[0061] User interface 210 receives user input and sends the received input to controller 600. User interface 210 may include various user input devices, such as keyboard, dome switch, touchpad (static / electrostatic), jog wheel, jog switch, etc.

[0062] Furthermore, the user interface 210 may also include a display 310 with a touch panel. The user interface 210 can be treated in the same manner as the display 310, which senses user input, such as touch input, and simultaneously displays the processing results of the user input as an image. The user interface 210 may include user interface (UI) elements displayed on the display 310 and capable of controlling the mobile device 1. Meanwhile, user input may include touch input and / or gesture input.

[0063] Microphone 220 can receive audio signals, convert the audio signals into electrical signals, and send the electrical signals to controller 600. Microphone 220 can receive user voice commands and send the received voice commands to controller 600, and controller 600 can control mobile device 1 in response to user voice commands.

[0064] The output unit 300 can be a device capable of outputting image signals or audio signals, and can correspond to various devices. For example, the output unit 300 may include at least one of a display 310 and a speaker 320. The speaker 320 can output audio signals associated with various functions of the mobile device 1.

[0065] The display 310 can display information input by the user or information provided to the user on various screens. The display 310 displays information processed by the controller 600. For example, the display 310 can display an image of space captured by the camera 100, and can also display augmented reality images in which virtual images are overlaid on the image of space. The display 310 can display a graphical user interface (GUI) associated with various functions of the mobile device 1.

[0066] Furthermore, the display 310 can also be used as an input device including a touch panel. The display 310 can be implemented as a liquid crystal display, a thin-film transistor liquid crystal display, an organic light-emitting diode (OLED), a flexible display, a three-dimensional (3D) display, etc.

[0067] The communicator 400 connects the mobile device 1 to external devices, servers, etc., via a network or communication channel, such as via wired and / or wireless methods. The communicator 400 may include various communication modules. For example, the communicator 400 may include communication modules that can connect to various communication networks (e.g., Bluetooth, Zigbee, wireless LAN, home radio frequency (RF), ultra-wideband (UWB), and the Internet).

[0068] The storage device 500 can store programs and data used to control the operation of the mobile device 1. Furthermore, the storage device 500 can store input / output data (e.g., text, photos, messages, still images, videos, etc.).

[0069] The storage 500 may correspond to a memory and may include volatile memory such as static random access memory (S-RAM) and dynamic RAM (D-RAM) and non-volatile memory such as read-only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) for long-term storage of data.

[0070] The controller 600 may include a processor 601, which generates control signals for controlling the operation of the mobile device 1 using programs and data stored in the storage 500 and / or memory 602. The processor 601 may include logic and operational circuitry to process data based on programs provided from the storage 500 and / or memory 602, and to generate control signals based on the processing results. The controller 600 may be implemented as hardware, such as circuitry.

[0071] Furthermore, the processor 601 and memory 602 can be implemented as separate chips, or as a single chip. The controller 600 may also include multiple processors 601 and multiple memories 602.

[0072] Reference Figure 2 The controller 600 may include a spatial recognizer 610, a virtual image generator 620, and an augmented reality processor 630. The spatial recognizer 610, the virtual image generator 620, and the augmented reality processor 630 may be implemented as separate processors or as a single processor.

[0073] The space recognizer 610 can recognize planes from an image of the space where the imaging device is to be installed, captured by the camera 100, in response to user input. For example, the space recognizer 610 can distinguish and recognize wall surfaces W, floor surfaces F, and ceiling surfaces included in an image of the real space. The space recognizer 610 can recognize planes based on guide lines (12a, 12b, and / or 12c), which will be described later.

[0074] Imaging devices can have flat or curved shapes, but for ease of description, we will assume that the imaging device has a flat shape. Furthermore, imaging devices can be mounted in various locations, but are typically mounted on a wall surface.

[0075] An image device is an electronic device capable of displaying images. An image device can be a standalone device, such as a light-emitting diode (LED) TV or an OLED TV. Alternatively, an image device can be a display module. A display module can be an LED module comprising at least one LED pixel or an LED cabinet in which multiple LED modules are connected. Multiple display modules can be physically connected to form a screen. In other words, an image device can include equipment for connecting and assembling multiple display modules.

[0076] Furthermore, imaging devices can be categorized in various ways. For example, they can be classified into categories such as digital signage, stand-alone displays, modular displays, video walls, outdoor equipment, interactive equipment, and professional equipment. These categories can be determined based on product characteristics, market demands, and other factors. Additionally, each category can include multiple product models.

[0077] Furthermore, the space recognizer 610 can store plane-related data identified from an image of a specific space in the storage 500 / memory 602, and can retrieve plane-related data from the storage 500 / memory 602. When the image of a space captured by the camera 100 is the same as the image of the space stored in the storage 500 / memory 602, the space recognizer 610 can retrieve plane-related data included in the captured space from the storage 500 / memory 602.

[0078] As described above, since the mobile device 1 according to the embodiment can automatically identify the space or plane in which the image device can be installed, it is easier to simulate the installation of the image device.

[0079] The virtual image generator 620 can generate virtual images of objects. The generated virtual images of objects can interact with user input, and the user can select, move, and change the virtual images of objects.

[0080] For example, the virtual image generator 620 can generate a grid 20, which will be displayed on a virtual plane VP corresponding to a plane identified from an image in space. The grid 20 is an image of a lattice pattern and has multiple cells, each cell having a predetermined size. The size of the cells 21 of the grid 20 can be changed based on user input, and the virtual image generator 620 can generate a grid 20 with cells 21 of varying sizes based on user input for changing the size of the cells 21. Furthermore, the pattern of the grid 20 can also be modified.

[0081] Furthermore, the virtual image generator 620 can adjust the grid 20 based on user input. Adjusting the grid 20 is different from changing the size of the cell 21. That is, adjusting the grid 20 means changing at least one of the grid 20's position and orientation.

[0082] For example, the virtual image generator 620 can move the grid 20 based on user input. The virtual image generator 620 can match the starting line of the grid 20 with the boundary line of the identified plane based on user input for adjusting the grid 20. User input for moving the grid 20 could be prolonged touching and dragging of the grid 20 on the display 310.

[0083] Furthermore, the virtual image generator 620 can rotate the orientation of the grid 20 based on user input for adjusting the grid 20. The user's input for rotating the orientation of the grid 20 can be achieved by simultaneously touching two points of the grid 20 on the display 310 and dragging one of the two points.

[0084] Furthermore, the virtual image generator 620 can generate a virtual image 30 for the image device and a virtual content image VC that can be displayed around the virtual image for the image device. The content includes various types of data, such as photos, videos, audio, and text, and the virtual content image VC is a virtual image that includes this content information. Moreover, the content corresponding to the virtual content image VC can be content stored in the storage 500 / memory 602, or it can be content obtained from an external server, etc., via the communicator 400.

[0085] The augmented reality processor 630 can display images of real space and virtual images together on the display 310. For example, the augmented reality processor 630 can generate an augmented reality image that overlays images of real space and virtual images onto each other, and display the generated augmented reality image on the display 310.

[0086] The augmented reality processor 630 can control the display 310 to simultaneously display an image of the space along with guide lines (12a, 12b, and / or 12c). The augmented reality processor 630 can display a grid 20 generated by the virtual image generator 620 on a virtual plane VP corresponding to a plane identified in the image of the space captured by the camera 100, based on user input, and control the display 310 to simultaneously display the grid 20 and the virtual image 30 of the imaging device on the grid 20. Furthermore, the augmented reality processor 630 can control the display 310 to simultaneously display an image of the real space, the virtual image 30 of the imaging device, and the virtual content image VC.

[0087] As described above, by displaying a virtual image corresponding to the actual size of the imaging device and an image of the real space in which the imaging device would be installed, users can realistically simulate the installation of the imaging device.

[0088] Figures 3 to 5 This is a flowchart illustrating a method for controlling a mobile device.

[0089] Figure 3 A virtual image of a display grid and image device is shown schematically.

[0090] Reference Figure 3The camera 100 of the mobile device 1 acquires an image by capturing a real space in which an imaging device can be installed (701). The controller 600 identifies a plane from the image of the space captured by the camera 100 (702). The controller 600 can control the display 310 to display a grid 20 on a virtual plane VP corresponding to the identified plane (703). For example, the controller 600 can display a grid 20 comprising a plurality of cells 21, each cell 21 having a predetermined size, on the virtual plane VP corresponding to the plane identified from the image of the captured space. The controller 600 can control the display 310 to display a virtual image 30 of the imaging device on the grid 20 displayed on the virtual plane VP based on user input (704).

[0091] Figure 4 The display content is further shown and the simulation results are stored.

[0092] Reference Figure 4 The camera 100 of the mobile device 1 acquires an image by photographing the actual space in which an image device can be installed (801). The controller 600 identifies a plane from the image of the space captured by the camera 100 based on guide lines 12a, 12b, and 12c (12) (802). The controller 600 can control the display 310 to display the image of the space together with the guide lines 12a, 12b, and / or 12c. Furthermore, the controller 600 can control the display 310 to display the guide lines as either the base guide line 12a or the corner guide lines 12b and 12c.

[0093] The controller 600 controls the display 310 to display a grid 20 on a virtual plane VP corresponding to the identified plane (803). For example, the controller 600 may display a grid 20 comprising a plurality of cells 21, each cell 21 having a predetermined size, on the virtual plane VP corresponding to a plane identified from an image of the shooting space. The controller 600 may adjust the grid based on user input (804). The controller 600 controls the display 310 to display a virtual image 30 of the imaging device on the grid 20 displayed on the virtual plane VP based on user input (805).

[0094] Subsequently, the controller 600 can control the display 310 to display content on the area where the virtual image 30 of the image device is displayed, based on user input (806). For example, the controller 600 can generate at least one virtual content image VC around the virtual image 30 of the image device based on user input for selecting the virtual image 30 of the image device via the user interface 210, and control the display 310 to display one or more virtual content images VC. The controller 600 can also control the display 310 to display content corresponding to the selected virtual content image on the area where the virtual image 30 of the image device is displayed, based on user input for selecting one of the one or more virtual content images VC.

[0095] Furthermore, the controller 600 can store simulation results related to the installation of the imaging device in storage 500 and / or memory 602 (807). The controller 600 can store simulation results related to the installation of the imaging device as at least one of video and screenshot images. The controller 600 can store the simulation process and / or simulation results as at least one of video and screenshot images based on commands entered via recording icon 70 and / or screenshot icon 80, which will be described later.

[0096] Figure 5 A method for displaying virtual images on an imaging device is shown in more detail.

[0097] Reference Figure 5 The camera 100 of the mobile device 1 acquires an image by capturing a real space in which an image device can be installed (901). The controller 600 identifies a plane from the image of the space captured by the camera 100 based on guide lines 12a, 12b and 12c (902). The controller 600 controls the display 310 to display a grid 20 on a virtual plane VP corresponding to the identified plane (903).

[0098] The controller 600 controls the display 310 to display a virtual image 30 of the image device on a grid 20 displayed on a virtual plane VP based on user input (904). For example, based on user input for selecting a specific area of ​​the grid 20, the virtual image 30 of the image device can be displayed on the selected specific area of ​​the grid 20. The controller 600 can control the display 310 to display a virtual image 30 of the image device with a predetermined size on the selected specific area of ​​the grid 20.

[0099] Furthermore, based on user input for selecting at least one unit 21, the controller 600 can control the display 310 to display a virtual image 30 of the image device on the selected at least one unit 21. That is, a virtual image 30 of the image device with a size corresponding to the number of selected units 21 can be displayed.

[0100] Subsequently, the controller 600 can control the display 310 to display content (905) on the area where the virtual image 30 of the image device is displayed, based on user input. For example, the controller 600 can generate at least one virtual content image VC around the virtual image 30 of the image device based on user input for selecting the virtual image 30 of the image device via the user interface 210, and control the display 310 to display one or more virtual content images VC. The controller 600 can also control the display 310 to display content corresponding to the selected virtual content image on the area where the virtual image 30 of the image device is displayed, based on user input for selecting one of the one or more virtual content images VC.

[0101] As described above, a realistic simulation of installing an image device can be performed by displaying a virtual image corresponding to the actual size of the image device and an image of the real space in which the image device can be installed, and playing the content in the area of ​​the virtual image corresponding to the actual size of the image device.

[0102] Simultaneously, the controller 600 can obtain purchase information of the real image device corresponding to the virtual image 30 of the image device from an external server (not shown), and control the display 310 to display the purchase information of the image device and the virtual image 30 of the image device together. Since the user can purchase and install the same image device as the image device that the user has already simulated, the user can minimize errors that may occur when actually installing the image device.

[0103] In the following text, reference will be made to Figures 6 to 29 Describe the additional features in more detail.

[0104] Figure 6 A method for initiating a simulation related to the installation of an imaging device is shown.

[0105] Reference Figure 6 The simulation related to the installation of the imaging device can be performed by running or driving an application installed on the mobile device 1. When the application is run or driven, the controller 600 can control the display 310 to display icon 5 and pop-up message 7, which are configured to guide the movement of the mobile device 1 to initiate recognition of the plane in the space in which the imaging device can be installed.

[0106] When movement of the mobile device 1 is detected, the controller 600 may stop displaying the icon 5 and pop-up message 7 configured to guide the movement of the mobile device 1. The mobile device 1 may include sensors for detecting movement.

[0107] Subsequently, the controller 600 can control the display 310 to display guide lines 12a, 12b, and 12c, for example, based on the environment or physical configuration of the space in which the imaging device will be installed. However, the pop-up message 7 and icon 5 configured to guide the movement of the mobile device 1 can be omitted. In other words, when a simulation relating to the installation of the imaging device begins, the camera 100 can capture images of the space in which the imaging device can be installed, and the process of identifying the plane from the captured images of the space can begin immediately.

[0108] Figures 7 to 9 A method is shown for identifying planes and displaying grids from an image of a space in which an imaging device may be mounted.

[0109] Reference Figure 7 The controller 600 can control the display 310 to display the basic guide line 12a used for plane identification. The controller 600 can display the image of the space captured by the camera 100 and the basic guide line 12a together on the display 310, and can identify the plane based on the guide line 12a.

[0110] For example, when the base guide line 12a displayed on the display 310 is adapted to the boundary between the wall surface W and the floor surface F, the controller 600 can identify the upper surface relative to the base guide line 12a as the wall surface W and the lower surface relative to the base guide line 12a as the floor surface F. The controller 600 can control the display 310 to display the grid 20 on the identified wall surface W.

[0111] The controller 600 can display a pop-up message 7 on the display 310, which includes guidance statements instructing the user to adapt the basic guide line 12a to the boundary between the wall surface W and the floor surface F. Additionally, the controller 600 can control the display 310 to display a confirmation button 14 below the basic guide line 12a. Based on user input for pressing the confirmation button 14, the controller 600 can determine that the basic guide line 12a is adapted to the boundary between the wall surface W and the floor surface F and can identify the wall surface W. Furthermore, even when there is no input for the confirmation button 14, the controller 600 can automatically identify the wall surface W when the basic guide line 12a is adapted to the boundary between the wall surface W and the floor surface F.

[0112] The controller 600 can generate a virtual plane VP corresponding to the wall surface W that is not a recognized plane. Furthermore, the controller 600 can control the display 310 to display a grid 20 on the virtual plane VP. The grid 20 can include multiple cells 21, each cell having a predetermined size.

[0113] Simultaneously, the controller 600 can control the display 310 to display a return icon 40, a pattern icon 50, a restart icon 60, a recording icon 70, and a screenshot icon 80. The command to return to the previous screen can be entered into the return icon 40; the command to change the pattern of the grid 20 and / or to turn the display of the grid 20 on or off can be entered into the pattern icon 50; the command to re-perform planar recognition and / or to delete the virtual image 30 of the image device can be entered into the restart icon 60; the command to store the simulation process and / or simulation results as video can be entered into the recording icon 70; and the command to store the simulation process and / or simulation results as a screenshot image can be entered into the screenshot icon 80.

[0114] Furthermore, the controller 600 can control the display 310 to show a menu bar 90 that allows selection of the category, model, and size of the image device. Image devices can have various categories. For example, image devices can be categorized as digital signage, stand-alone displays, video walls, outdoor equipment, interactive devices, professional equipment, etc. Such categories of image devices can be determined based on product characteristics, market demands, etc. Furthermore, each category can include multiple product models. Additionally, each product model can have different sizes.

[0115] Reference Figure 8 and Figure 9 The controller 600 can control the display 310 to display corner guide lines 12b and 12c for identifying planes at the corners of the space. The controller 600 can display an image of the space captured by the camera 100 along with the corner guide lines 12b and 12c on the display 310, and can identify planes relative to the corner guide lines 12b and 12c. The controller 600 can control the display 310 to display either the base guide line 12a or the corner guide lines 12b and 12c based on user input for changing the guide lines.

[0116] Reference Figure 8 When the first corner guide line 12b displayed on the display 310 is adapted to the boundary between the wall surface W and the floor surface F, and to the position where the wall surface W curves inward, the controller 600 can identify the two upper surfaces relative to the first corner guide line 12b as the left wall surface W and the right wall surface W, and the lower surface relative to the first corner guide line 12b as the floor surface F. The controller 600 can control the display 310 to display the grid 20 on the identified left wall surface W and right wall surface W.

[0117] Reference Figure 9When the second corner guide line 12c displayed on the display 310 is adapted to the boundary between the wall surface W and the floor surface F, and to the outward curvature of the wall surface W, the controller 600 can identify the two upper surfaces relative to the second corner guide line 12c as the left wall surface W and the right wall surface W, and the lower surface relative to the second corner guide line 12c as the floor surface F. The controller 600 can control the display 310 to display the grid 20 on the identified left and right wall surfaces W.

[0118] The first corner guide line 12b and the second corner guide line 12c can be displayed as icons such as three-dimensional coordinate axes (x-axis, y-axis, and z-axis), and can have different shapes. For example, the first corner guide line 12b can have the shape of three line segments connecting the vertex of the triangle and the center of the triangle. The second corner guide line 12c can have the shape of a downward arrow.

[0119] Simultaneously, the grid 20 can be continuously displayed as the viewpoint of the camera 100 moves. For example, when a plane is identified and the grid 20 is displayed on a virtual plane VP corresponding to that plane, the controller 600 can control the display 310 to continuously display the grid 20 on the plane as the viewpoint of the camera 100 moves.

[0120] Since the wall surface W can be wider than the surface displayed on the display 310 by the camera 100, the controller 600 can generate a virtual plane VP with a wider area than the plane existing in the field of view of the camera 100. The grid 20 can be understood as being displayed on the entire virtual plane VP. Therefore, the grid 20 can be displayed continuously even when the viewpoint of the camera 100 moves. In this case, the three-dimensional coordinate axes on which the grid 20 is displayed do not change.

[0121] As described above, because the space or plane in which the imaging device can be installed can be automatically identified, the installation of the imaging device can be simulated more easily. Furthermore, by displaying a grid on the plane where the imaging device can be installed, the dimensions of the imaging device to be installed can be easily determined. A more realistic simulation can be achieved by displaying a virtual image of the imaging device with the same dimensions as the real imaging device.

[0122] Figures 10 to 12 The method for adjusting the grid is shown.

[0123] Reference Figures 10 to 12 Grid 20 may be displayed as deviating from the wall surface in space. This phenomenon may be due to an error in the process of identifying the plane in space. Therefore, when grid 20 is displayed as deviating from the identified plane, it is necessary to adjust the functionality of grid 20.

[0124] Figure 10 From Figure 7 Continue. (Refer to...) Figure 10 The controller 600 can move the grid 20 based on user input. User input includes touch input and / or gesture input. For example, the controller 600 can change at least one of the position and orientation of the grid 20 based on user input for adjusting the grid 20. The controller 600 can match the starting line of the grid 20 to the boundary of an identified plane based on user input for moving the grid 20. User input for moving the grid 20 can be prolonged touching and dragging of the grid 20 on the display 310. When the grid 20 is touched for a prolonged period (e.g., a period of time greater than or equal to a predetermined time period), the grid 20 can be displayed in a movable state. For example, the grid 20 can be changed to a dashed line. The color of the grid 20 can be changed.

[0125] Furthermore, the controller 600 can rotate the orientation of the grid 20 based on user input for rotating the grid 20. User input for rotating the grid 20 can be simultaneously touching two points on the display 310 and dragging one of the two points. For example, the orientation of the grid 20 can be rotated to the left within a range of 0° to 180°. Additionally, the orientation of the grid 20 can be rotated to the right within a range of 0° to 180°.

[0126] Figure 11 From Figure 8 Continue, and Figure 12 From Figure 9 continue.

[0127] Reference Figure 11 and Figure 12 The grid 20 can be displayed offset from the left wall surface W and the right wall surface W. In this case, the controller 600 can move or rotate the entire grid 20 based on user input. Furthermore, the controller 600 can independently control the grid 20 corresponding to the left wall surface and the grid 20 corresponding to the right wall surface. That is, the controller 600 can independently control the grid 20 corresponding to each plane based on user input.

[0128] like Figure 10 As shown, the controller 600 can match the starting line of the grid 20 with the boundary of the identified plane based on user input for moving the grid 20. User input for moving the grid 20 can be a prolonged touch and dragging of the grid 20 on the display 310. When the grid 20 is touched for a prolonged period, a virtual plane VP including the corresponding grid 20 can be displayed in a movable state. For example, the grid 20 can be changed to a dashed line. The color of the grid 20 can be changed.

[0129] User input for rotating the grid 20 can be achieved by simultaneously touching two points on the grid 20 on the display 310 and dragging one of the points. For example, the grid 20 can be rotated to the left within a range of 0° to 180°. Furthermore, the grid 20 can be rotated to the right within a range of 0° to 180°.

[0130] Simultaneously, the menu bar 90 can be hidden. In other words, the controller 600 can control the display 310 to hide the menu bar 90 and display the menu indicator 95 based on user input for turning off the menu bar 90. Furthermore, the controller 600 can switch the menu bar 90 to the menu indicator 95 after a predetermined time has elapsed. Additionally, the controller 600 can switch the menu bar 90 to the menu indicator 95 in response to a specific function being performed (e.g., the function of rotating a virtual image on an imaging device). Conversely, the controller 600 can control the display 310 to show the menu bar 90 based on user input for turning on the menu bar 90 while simultaneously displaying the menu indicator 95.

[0131] Figure 13 The pattern of changing the grid is shown.

[0132] Reference Figure 13 The controller 600 can change the pattern of the displayed grid 20. For example, the grid 20 can be displayed as a square pattern, a rectangular pattern, or a diagonal pattern. The pattern of the grid 20 can be set differently, and is not limited to this. A diagonal pattern can be selected to more easily distinguish the virtual image 30 of the imaging device from the recognized plane (e.g., wall surface W).

[0133] The controller 600 can change the pattern of the grid 20 based on user input. Furthermore, the controller 600 can turn the display of the grid 20 on or off based on user input. When the user touches the pattern icon 50, the square pattern of the grid 20 can be changed to a diagonal pattern. Additionally, when the user touches the pattern icon 50 again, the display of the grid 20 can be turned off. In other words, using the pattern icon 50, commands to change the pattern of the grid 20 and / or to turn the display of the grid 20 on or off can be entered.

[0134] Figures 14 to 16 An implementation of displaying a virtual image of an image device on a grid is shown. Figure 17 A virtual image of a rotating and displaying image device is shown.

[0135] Reference Figure 14As described above, the controller 600 can control the display 310 to show a menu bar 90 that allows selection of the category, model, and size of the image device. Furthermore, the controller 600 can control the display 310 to show a virtual image 30 of the image device based on user input for selecting the image device from the menu bar 90.

[0136] The menu bar 90 may include multiple category icons 91. Furthermore, the multiple category icons 91 can be scrolled (e.g., horizontal and / or vertical scrolling). When the number of category icons 91 exceeds the number of category icons 91 that can be displayed on the menu bar 90 area, the category icons 91 not displayed on the menu bar 90 can be presented or viewed through user-input scrolling (e.g., horizontal and / or vertical scrolling).

[0137] Furthermore, each category may include multiple product models. Based on user input for selecting one of the multiple category icons 91, the controller 600 can control the display 310 to show the multiple model icons 92 included in the selected category. For example, when the user selects the first category, model icons 92 representing the multiple models included in the first category can be displayed on the menu bar 90. In this case, category information can be displayed at the top of the menu bar 90. The multiple model icons 92 can also be scrolled (e.g., horizontally and / or vertically).

[0138] Reference Figure 15 The controller 600 can control the display 310 to further display a size icon 93 and a rotation icon 94 on the menu bar 90. The size icon 93 enables the selection of the size of the image device, and the rotation icon 94 enables the rotation of the virtual image 30 of the image device. Based on user input for selecting one of a plurality of model icons 92 (e.g., model 1), the controller 600 can control the display 310 to display the size icon 93, which enables the selection of the size of the selected model (e.g., model 1).

[0139] Reference Figure 16 The controller 600 can control the display 310 to show multiple size icons 93a, 93b, and 93c. When size icon 93 is displayed on the menu bar 90, category information and model information can be displayed at the top of the menu bar 90. When the user selects a specific size, a virtual image 30 of the image device with the selected size can be displayed on the grid 20.

[0140] Furthermore, based on user input for selecting a specific area of ​​grid 20, controller 600 can control display 310 to display a virtual image 30 of the imaging device on that specific area of ​​grid 20. For example, when a user touches a 46-inch icon 93b and then touches a specific area of ​​grid 20, a virtual image 30 corresponding to the imaging device of model 1 with a 46-inch size can be displayed on that specific area of ​​grid 20. The specific area may refer to at least one cell 21 included in grid 20. Additionally, controller 600 can move the displayed virtual image 30 of the imaging device based on user input.

[0141] Furthermore, the controller 600 can control the display 310 to further display text, which includes at least one of the following: the model name, size, resolution, and distance from the floor of the image device corresponding to the virtual image 30 displayed by the image device. Figure 16 In the image, at the top of the virtual image 30 of the image device, text indicating that the model name is Model 1 and the size is 46 inches is displayed, and text indicating that the distance from the floor is 1m is also displayed.

[0142] Reference Figure 17 Based on user input for rotating the virtual image 30 of the image device, the controller 600 can control the display 310 to display the rotated virtual image 30 of the image device on the virtual plane VP. For example, when there is user input for touching the rotation icon 94 while the virtual image 30 of the image device is being displayed, the virtual image 30 of the image device can be rotated 90° in the left or right direction.

[0143] Furthermore, the controller 600 can rotate the virtual image 30 of the display device based on user gestures input onto an area of ​​the virtual image 30. For example, when there is user input for simultaneously touching two points on the virtual image 30 and dragging one of those points, the virtual image 30 can be rotated. For example, the virtual image 30 can rotate left or right within a range of 0° to 90°. The controller 600 can switch the menu bar 90 to the menu indicator 95 in response to the rotation of the virtual image.

[0144] At the same time, even when the user moves the mobile device 1 to change the direction the camera 100 is facing after displaying the virtual image 30 on the imaging device, the virtual image 30 on the imaging device can be viewed as if it were fixed to the wall surface in real space.

[0145] Figure 18 Multiple virtual images are shown on the display image device.

[0146] Reference Figure 18The controller 600 can control the display 310 to display multiple virtual images 30 of the image device based on user input for selecting another image device. For example, when a user input is present to select a model 4 of the same category (e.g., category 1) and to select the size of model 4 (e.g., 55 inches) while the virtual image 30a of the first image device is being displayed, the controller 600 can display a virtual image 30b of the second image device in an area that does not overlap with the virtual image 30a of the first image device via the display 310. Furthermore, as described above, the virtual image 30b of the second image device can be displayed on a specific area of ​​the grid 20 based on user input for selecting a specific area of ​​the grid 20.

[0147] Although the case of displaying a virtual image of an image device associated with another model in the same category has been described, when an image device included in another category is selected, virtual images of multiple image devices included in different categories can be displayed.

[0148] Furthermore, the controller 600 can control the display 310 to display another grid 20 according to the category or model of the image device. For example, when model 1 is selected, a grid 20 with cells 21 of a first size can be displayed, and when model 2 is selected, a grid 20 with cells 21 of a second size can be displayed.

[0149] This allows for the simulation of installations of imaging devices of various categories, models, and sizes, making it easier to simulate such installations. Furthermore, it increases the level of utilization as a simulation tool.

[0150] Figures 19 to 23 An implementation of a virtual image display device is shown.

[0151] When the category of the image device is one that connects multiple display modules (e.g., digital signage), the method for displaying a virtual image of the image device described below may be more useful. One cell 21 of grid 20 may correspond to one display module.

[0152] Reference Figure 19 Based on user input for selecting at least one cell 21 included in the grid 20, the controller 600 can control the display 310 to display a virtual image 30 of the imaging device on the selected at least one cell 21. For example, a user can select multiple cells 21 by touching and dragging the cells 21 at any location on the grid 20. The controller 600 can generate a virtual image 30 of the imaging device having dimensions corresponding to the selected multiple cells 21, and display the virtual image 30 on the display 310.

[0153] Simultaneously, the display module can include multiple LED pixels. The spacing refers to the distance between the LED pixels. Furthermore, the smaller the spacing, the higher the resolution. The size icon 93 displays the spacing. Users can select the category, model, and spacing of the image device to be simulated from the menu bar 90. Figure 19 In the middle, select model 2 that belongs to category 2 (e.g., digital signage) and has a spacing of 0.8. Afterwards, the controller 600 can switch the menu bar 90 to the menu indicator 95.

[0154] The virtual image 30 of the image device can be displayed after multiple units 21 are selected, or it can be displayed sequentially according to the order of the selected units 21 while multiple units 21 are selected. Furthermore, the controller 600 can control the display 310 so that the selected units 21 can be distinguished and displayed while multiple units 21 are selected. For example, the color of the selected unit 21 can be changed and displayed.

[0155] Figure 19 A virtual image 30 of an imaging device with a size (43 inches) corresponding to 14 units (7×2) is shown. The size of the units 21 can be preset to correspond to the size of the real imaging device. The size of the units 21 can vary according to the spacing of the imaging device. Furthermore, the size of the units 21 can be changed based on user input. The grid 20 includes multiple units 21, each with a predetermined size, and the user can adjust the number of units 21 to be selected, thereby adjusting the size of the virtual image of the imaging device differently.

[0156] As described above, the controller 600 can control the display 310 to further display text, which includes at least one of the following: the model name, size, resolution, and distance from the floor of the image device corresponding to the virtual image 30 displayed by the image device. Figure 19 In the image, the information indicating that the model name is Model 2, the size is 7×2 (43 inches), and the resolution is 3840×1200 is displayed at the top of the virtual image 30 of the image device, and the information indicating that the distance from the floor is 1m is displayed at the bottom of the virtual image 30.

[0157] Furthermore, when the camera 100 is facing another area of ​​the wall surface W, the user can connect to and select the unit 21 displayed on that other area of ​​the wall surface W. In other words, while maintaining user input for selecting unit 21, the controller 600 can connect units 21 that have been selected but are not displayed on the display 310 and units 21 that have been selected and are displayed on the display 310. That is, when the user selects a unit 21 of the grid 20, the facing direction of the camera 100 does not need to be fixed.

[0158] Reference Figure 20The controller 600 can change the position of the virtual image 30 displayed on the image device based on user input for changing the position of the virtual image 30. That is, the position of the virtual image 30 on the image device can be changed and displayed on the grid 20 according to user input for dragging and dropping.

[0159] Reference Figure 21 The controller 600 can adjust the size of the virtual image 30 of the display image device based on user gestures input to an area of ​​the virtual image 30. For example, when there is user input for simultaneously touching two points of the virtual image 30 of the image device and dragging the two points in opposite directions, the virtual image 30 of the image device can be enlarged.

[0160] Reference Figure 22 Based on user input for selecting additional unit 21 on grid 20, controller 600 can control display 310 to add and display virtual image 30b of second image device.

[0161] For example, when the user selects a number of units 21 that is different from the number of units 21 corresponding to the virtual image 30a of the first image device that already exists on the grid 20, the controller 600 can control the display 310 so that the added virtual image 30b of the second image device is displayed while being distinguished from the virtual image 30a of the first image device.

[0162] exist Figure 22 In the first image device, the displayed virtual image 30a comprises 14 units (7×2). When the number of additionally selected units 21 is 8 (2×4), the controller 600 can control the display 310 such that the virtual image 30b of the second image device, corresponding to the additionally selected units 21, is displayed while being distinguished from the virtual image 30a of the first image device. In other words, when the horizontal or vertical length of the additionally selected units 21 differs from the horizontal or vertical length of the virtual image 30a of the first image device, the virtual image 30b of the second image device can be displayed while being distinguished from the virtual image 30a of the first image device.

[0163] Furthermore, the controller 600 can control the display 310 to further display text, which includes at least one of the following: the model name, size, resolution, and distance from the floor of the image device corresponding to the virtual image 30b of the second image device. Figure 22 In the image, information indicating the model name is Model 2, the size is 7×2 (43 inches), and the resolution is 3840×1200 is displayed at the top of the virtual image 30a on the first image device, and information indicating the distance from the floor is 1m is displayed at the bottom of the virtual image 30a. Figure 22In the image, information indicating that the model name is Model 3, the size is 2×4 (26 inches), and the resolution is 3840×1200 is displayed at the bottom of the virtual image 30b of the second image device, and information indicating that the distance from the floor is 60cm is also displayed at the bottom of the virtual image 30b. However, this disclosure is not limited to this, and the information can be displayed in a manner similar to the virtual image 30a of the first image device, or the information can be displayed differently (e.g., some or all of the information can be displayed on the horizontal side of the virtual image).

[0164] Furthermore, when the user selects another area of ​​unit 21, including even the area of ​​the virtual image 30a of the first image device, the controller 600 can control the display 310 to display the virtual image 30b of the second image device in an area other than the area where the virtual image 30a of the first image device is displayed.

[0165] Reference Figure 23 The controller 600 can further select a unit 21 corresponding to the size of the virtual image 30 of the image device in the area following the virtual image 30 of the image device. In this case, the controller 600 can display a magnified virtual image 30a of the image device on the display 310 based on the additionally selected unit 21.

[0166] Figure 24 Indicates whether to show or hide the grid.

[0167] Reference Figure 24 When a user touches the pattern icon 50, the grid 20 displayed on the display 310 can be removed. That is, the controller 600 can control the display 310 to show the image of the space captured by the camera 100 and the virtual image 30 of the imaging device, but excluding the grid 20. In this way, even when the user moves the mobile device 1 and the camera 100 changes its orientation, the virtual image 30 of the imaging device can be viewed as if it were fixed to the wall surface of the real space. Therefore, the user can more easily confirm the form in which the imaging device can be mounted on the wall surface W of the real space.

[0168] Figure 25 and Figure 26 A virtual image of the image device is shown.

[0169] Reference Figure 25 The controller 600 can delete virtual images of the image device based on user input. When multiple virtual images of the image device are displayed, the controller 600 can delete all virtual images of the multiple image devices based on user input for deleting virtual images 30a and 30b of the image device.

[0170] For example, the restart icon 60 can be used to input a command to delete the virtual image 30 of the imaging device. When the user touches the restart icon 60 while the virtual image 30 of the imaging device is displayed on the grid 20, the entire virtual image 30 of the imaging device can be deleted.

[0171] Furthermore, the restart icon 60 can also be used to input a command to re-perform plane recognition. The controller 600 can re-recognize the plane based on user input and generate a new grid 20 on the virtual plane corresponding to the re-recognized plane. For example, when a user touches the restart icon 60 while the virtual image 30 of the imaging device is not displayed on the grid 20, the controller 600 can re-recognize the plane and generate a new grid 20 on the virtual plane corresponding to the re-recognized plane. Re-recognizing the plane and generating a new grid 20 can be performed via user input for touching the return icon 40.

[0172] Reference Figure 26 When displaying multiple virtual images of an image device, the controller 600 can selectively delete multiple virtual images of the image device based on user input for deleting virtual images 30a and 30b of the image device. For example, when displaying virtual image 30a of the first image device and virtual image 30b of the second image device, one of the virtual images 30a and 30b of the first image device can be deleted. For example, the controller 600 can change the virtual image 30b of the second image device into a movable state based on user input for long-term pressing of the virtual image 30b of the second image device. Figure 26 In this process, when an "×" is displayed in the central area of ​​the virtual image 30b of the second image device, the virtual image 30b of the second image device is changed to a movable state. Thereafter, the controller 600 can delete the virtual image 30b of the second image device based on user input for touching the "×" displayed in the central area of ​​the virtual image 30b. That is, the controller 600 can delete some of the multiple virtual images of the image device based on user input.

[0173] Additionally, as described above, the return icon 40 can be used to input a command to return the screen to the previous screen. The controller 600 can also sequentially delete multiple virtual images of the imaging device based on user input for touching the return icon 40. For example, whenever a return command is entered, multiple sequentially displayed virtual images of the imaging device can be deleted in reverse order.

[0174] As mentioned above, since users can easily adjust the size and number of virtual images on the imaging device, the installation of the imaging device can be simulated in various ways.

[0175] Figures 27 to 29This illustrates playing content on an area of ​​a virtual image displayed on an image display device.

[0176] Reference Figure 27 The controller 600 can control the display 310 to display content on an area of ​​the virtual image 30 of the display image device based on user input for selecting content. For example, the controller 600 can control the display 310 to display one or more virtual content images VC around the virtual image 30 of the image device in response to user input for selecting the virtual image 30 of the image device. The virtual content image VC is an image that includes content information (e.g., thumbnail images) that can be played in the virtual image 30 of the image device.

[0177] One or more virtual content images (VCs) can be arranged in various patterns. For example, one or more virtual content images (VCs) can be arranged in a ring around the virtual image 30 of the image device, or they can be arranged in the longitudinal or diagonal direction with respect to the virtual image 30 of the image device.

[0178] The controller 600 can control the display 310 to display content corresponding to the selected virtual content image VC on the area of ​​the virtual image 30 of the display image device based on user input for selecting one of one or more virtual content images VC. Figure 27 In this context, when a user selects a virtual content image VC that includes content information of movie 4, movie 4 can be played on the virtual image 30 area of ​​the display device. One or more virtual content images VC may include different content information segments, and movie 4 may be a moving image.

[0179] Furthermore, one or more virtual content images (VCs) can be displayed for each category. For example, content categories can be categorized as music video categories, movie categories, image categories, etc. The content can be obtained from an external server, etc., via communicator 400, or it can be content stored in storage 500 and / or memory 602. Based on user input for selecting the "My Works" icon MG, controller 600 can control display 310 to display a list of content fragments stored in storage 500 and / or memory 602.

[0180] Reference Figure 28When a user's swipe gesture is input to any area of ​​the display 310 while one or more virtual content images (VCs) are displayed around the virtual image 30 of the image device, the controller 600 can display another virtual content image (VC) around the virtual image 30 of the image device on the display 310. For example, when the user's swipe gesture is input while virtual content images (VCs) representing movies 1, 2, 3, 4, and 5 are displayed, virtual content images (VCs) representing movies 6, 7, 8, 9, and 10 can be displayed around the virtual image 30 of the image device.

[0181] Reference Figure 29 When content (e.g., movie 4) is displayed on the area of ​​the virtual image 30 of the display image device, a command to rotate the virtual image 30 of the image device can be entered. In this case, the controller 600 can control the display 310 to display another content segment (e.g., movie 6) at a screen ratio suitable for the rotation of the virtual image 30 of the image device.

[0182] As mentioned above, since various content clips can be played on the area of ​​the virtual image displayed on the image device, users can perform more realistic simulations.

[0183] Furthermore, as described above, based on the mobile device and the method for controlling the mobile device described herein, a user can realistically simulate the installation of an image device by displaying a virtual image corresponding to the actual size of the image device together with an image of the real space in which the image device will be installed.

[0184] Furthermore, according to various examples of this disclosure, mobile devices and methods for controlling mobile devices can perform more realistic simulations by playing content on an area displaying a virtual image corresponding to the actual size of the image device.

[0185] Furthermore, the disclosed embodiments can be implemented in the form of a recording medium storing computer-executable instructions. The instructions can be stored as program code, and when run by a processor, a program module can be generated to perform the operations of the disclosed embodiments. The recording medium can be implemented as a computer-readable recording medium, for example, as a non-transitory computer-readable recording medium.

[0186] Computer-readable recording media include all types of recording media that store instructions that can be decrypted by a computer. For example, computer-readable recording media can include ROM, RAM, magnetic tape, magnetic disk, flash memory, optical data storage devices, etc.

[0187] Exemplary embodiments have now been described with reference to the accompanying drawings. Those skilled in the art will understand that other forms, different from the exemplary embodiments, may be implemented without departing from the technical spirit and characteristics of the exemplary embodiments. Therefore, the exemplary embodiments are merely examples and should not be construed as limiting. That is, various modifications can be made without departing from the scope of this disclosure.

Claims

1. Mobile devices, including: monitor; A camera, configured to capture images of the space, and an image device capable of displaying the images will be installed in the space; User interface; as well as The controller is configured as follows: The image of the space captured by the camera is displayed on the monitor. In the image of the space being displayed on the monitor, identify the plane on which the imaging device is to be mounted. The display is controlled to display a grid on a virtual plane corresponding to the plane, the grid comprising cells having a preset size corresponding to the size of the image device. The grid is adjusted based on user input received via the user interface, such that the starting line of the grid displayed on the display matches the boundary of the identified plane. Control the display to show an augmented reality image, including an image of the space and a virtual image of the imaging device, on an adjusted grid displayed on the virtual plane. The display is controlled to show the virtual image of the image device in a designated area of ​​the grid selected by the user.

2. The mobile device according to claim 1, wherein, The controller is configured to adjust the grid by moving the grid based on the user input.

3. The mobile device according to claim 1, wherein, The controller is configured as follows: The display is controlled to show the guide lines and the image of the space captured by the camera, and The plane is identified based on the guide line.

4. The mobile device according to claim 3, wherein, The controller is configured to control the display to display the guide lines as either base guide lines or corner guide lines.

5. The mobile device according to claim 1, wherein, The controller is configured to control the display to rotate the virtual image of the image device on the virtual plane based on user input received via the user interface.

6. The mobile device according to claim 1, wherein, The controller is configured to control the display to further display text, the text including at least one of the model name, size, resolution, and distance from the floor of the image device corresponding to the virtual image of the image device.

7. The mobile device according to claim 1, wherein, The controller is configured as follows: A menu bar is displayed that allows selection of at least one of the image device's category, model, and size. The display is controlled to show the virtual image of the image device based on user input received via the user interface from selecting the image device from the menu bar.

8. The mobile device according to claim 1, wherein, When multiple virtual images from multiple image devices are displayed, the controller is configured to completely or partially delete the multiple virtual images from the multiple image devices based on user input received via the user interface to delete one or more of the multiple virtual images from the multiple image devices.

9. The mobile device according to claim 1, wherein, The controller is configured to control the display to show the content on the area of ​​the virtual image displayed by the image device based on user input of selected content received via the user interface.

10. The mobile device according to claim 1, wherein, The controller is configured to store simulation results related to the installation of the imaging device as at least one of video and screenshot images.

11. The mobile device according to claim 1, wherein, The controller is configured to adjust the grid by rotating the grid direction based on the user input.

12. A method for controlling a mobile device, the method comprising: An imaging device capable of displaying images of a space, which captures images of the space using a camera, will be installed in the space. The image of the space captured by the camera is displayed on the monitor; In the image of the space being displayed on the monitor, identify the plane on which the image device is to be installed; A grid is displayed on a virtual plane corresponding to the plane via the display, the grid comprising cells having a preset size corresponding to the size of the image device; The grid is adjusted based on user input received via the user interface, such that the starting line of the grid displayed on the display matches the boundary of the identified plane; Augmented reality images that display images of the space and virtual images of the imaging device on an adjusted grid displayed on the virtual plane; as well as The virtual image of the image device is displayed in a designated area of ​​the grid selected by the user.

13. The method according to claim 12, wherein, Adjusting the grid includes moving the grid based on the user input.

14. The method of claim 12, further comprising displaying, together with the guide lines and an image of the space captured by the camera, on the display. in, Identifying the plane includes identifying the plane based on the guide line.

15. The method according to claim 14, wherein, Displaying the guide lines together with the image of the space captured by the camera on the display includes displaying the guide lines as base guide lines or corner guide lines.

16. The method of claim 12, further comprising: The user input for rotating the virtual image of the imaging device is received via the user interface. Based on user input received via the user interface to rotate the virtual image of the image device, the virtual image of the image device is rotated on the virtual plane; as well as A rotated virtual image of the image device is displayed on the grid.

17. The method of claim 12, further comprising displaying text, the text including at least one of a model name, size, resolution, and distance from the floor of the image device corresponding to the virtual image of the image device.

18. The method of claim 12, further comprising displaying a menu bar, the menu bar enabling selection of at least one of the category, model, and size of the imaging device. in, Displaying the virtual image of the imaging device includes displaying the virtual image of the imaging device based on user input received via the user interface from selecting the imaging device from the menu bar.

19. The method of claim 12, further comprising: When multiple virtual images from multiple image devices are displayed, the multiple virtual images from the multiple image devices are completely or partially deleted based on user input received via the user interface to delete one or more of the multiple virtual images from the multiple image devices.

20. The method of claim 12, further comprising displaying the content on an area of ​​the virtual image displayed by the image device based on user input of selected content received via the user interface.

21. The method of claim 12, further comprising storing simulation results relating to the installation of the imaging device as at least one of a video and a screenshot image.

22. The method according to claim 12, wherein, Adjusting the grid includes: adjusting the grid by rotating the grid's orientation based on the user input.

Images