Server and method for linking interactive images between portable terminal and n screen-based heterogeneous displays

Abstract

The present disclosure can comprise: receiving, from a first external device, behavior information of an experience visitor in an experience space; determining whether the behavior information is object flick information obtained when an object image is flicked on a portable terminal of the experience visitor; if the behavior information is object flick information, extracting object movement coordinates on the basis of the object flick information; determining a first movement direction and a first movement position of the object image on the basis of the object movement coordinates; when the determination of the first movement direction and the first movement position is completed, extracting a preset first interactive image linked to the first movement direction and the first movement position; and controlling a second external device such that the first interactive image is displayed on a wall surface of the experience space corresponding to the first movement direction and the first movement position.

Description

Server and method for interlinking interactive images between portable terminals and N-screen-based heterogeneous displays

[0001] The present disclosure relates to a server and a method for interlinking interactive images between a portable terminal and a heterogeneous display based on an N screen.

[0002] In general, conventional N-screen refers to a service that provides interactive images through multiple screens.

[0003] However, the existing N-screen-based interactive video provision system had limitations in implementing the naturalness of the motion sensing effect without interruption when moving between walls.

[0004] In addition, the existing N-screen-based interactive video provision system had limitations in allowing non-experts and those who are digitally disadvantaged to easily and quickly produce interactive videos without difficult or complex coding.

[0005] In addition, the existing N-screen-based interactive video provision system had limitations in displaying interactive videos on the walls and floors of the experience space by changing them for each experience viewer, and thus had limitations in providing enjoyable and useful interactive videos for each experience viewer.

[0006] In addition, the conventional N-screen-based interactive video provision system had limitations in efficiently storing interactive videos on portable terminals, and thus had limitations in preserving the interactive videos one wanted.

[0007] The purpose of the embodiments disclosed in the present disclosure is to provide a natural and seamless motion sensing effect when moving between walls.

[0008] In addition, the embodiment disclosed in the present disclosure aims to provide a method for even non-experts and those who are digitally disadvantaged to be able to create interactive images very easily and very quickly without difficult or complex coding.

[0009] In addition, the embodiment disclosed in the present disclosure is capable of displaying interactive images that are changed for each viewer on the walls and floor of an experience space, thereby providing enjoyable and useful interactive images for each viewer.

[0010] In addition, the embodiment disclosed in the present disclosure aims to provide a portable terminal capable of storing interactive images, thereby allowing a user to keep desired interactive images.

[0011] The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0012] A server for mutually linking interactive images between a portable terminal and an N-screen-based heterogeneous display according to one aspect of the present disclosure for achieving the above-described technical task comprises: a first external device for obtaining object images and behavior information in a portable terminal of an experience viewer in an experience space; a communication unit for performing communication with a plurality of second external devices for displaying N-screen-based interactive images corresponding to the object images in the experience space;And a processor for controlling an operation related to the mutual linkage of interactive images between the portable terminal and the N-screen-based heterogeneous display, wherein the processor, when the action information received from the first external device through the communication unit is object flick information for flicking the object image, extracts object movement coordinates based on the object flick information, and when the determination of the first movement direction and the first movement position of the object image is completed based on the object movement coordinates, controls the second external device to display the first interactive image extracted in connection with the first movement direction and the first movement position on the wall surface of the experiential space corresponding to the first movement direction and the first movement position, and when the edited object image produced by the experiential viewer is opposed to the wall surface for a preset period of time, extracts the position coordinates of the opposed wall surface based on the image of the portable terminal and the image of the wall surface acquired through the first external device, and when the determination of the second movement direction and the second movement position of the edited object image is completed based on the position coordinates, controls the second external device to display the first interactive image, which is extracted in connection with the first movement direction and the first movement position, on the wall surface of the experiential space corresponding to the first movement direction and the first movement position. The second external device may be controlled to display a second interactive image extracted in connection with a movement position on the wall corresponding to the second movement direction and the second movement position, and the portable terminal and the second external device may be controlled to automatically store the second interactive image in the portable terminal when the image of the portable terminal is in a state facing the wall for a preset period of time and the display time of the second interactive image exceeds a preset target time.

[0013] In addition, the processor, when a first edited object image among the plurality of edited object images produced by the experience viewer is opposed to a first position on the wall of the experience space for a first preset time, and a second edited object image among the plurality of edited object images is opposed to a second position different from the first position for a second time later than the first preset time, extracts a first coordinate of the opposed first position and a second coordinate of the opposed second position based on an image of the portable terminal acquired through the first external device and an image of the wall, and when a determination of a third movement direction and a third movement position of the first edited object image is completed based on the first coordinate, and a determination of a fourth movement direction and a fourth movement position of the second edited object image is completed based on the second coordinate, displays a third interactive image extracted in connection with the third movement direction and the third movement position at the first position corresponding to the third movement direction and the third movement position, and displays a fourth interactive image extracted in connection with the fourth movement direction and the fourth movement position. It may be characterized by controlling the second external device to display the interactive image at the second position corresponding to the fourth movement direction and the fourth movement position.

[0014] In addition, the processor may be characterized in that it controls the portable terminal and the second external device so that when the image of the portable terminal acquired through the first external device is in a state facing the first position for a preset time and the display time of the third interactive image exceeds a preset target time, the third interactive image is automatically stored in the portable terminal, and when the image of the portable terminal acquired through the first external device is in a state facing the second position for a preset time and the display time of the fourth interactive image exceeds a preset target time, the processor controls the portable terminal and the second external device so that the fourth interactive image is automatically stored in the portable terminal.

[0015] In addition, the processor may be characterized in that, when a third edited object among the plurality of edited object images is opposed to a third position on the floor of the experiential space during a third time later than a preset second time, the processor extracts a third coordinate of the opposed third position based on an image of the portable terminal and an image of the floor obtained through the first external device, and when the determination of the fifth movement direction and the fifth movement position of the third image is completed based on the third coordinate, the processor controls the second external device to display a fifth interactive image extracted in connection with the fifth movement direction and the fifth movement position at the third position corresponding to the fifth movement direction and the fifth movement position.

[0016] In addition, the processor may be characterized in that it controls the portable terminal and the second external device so that, when the image of the portable terminal acquired through the first external device is in a state of facing the third location for a preset period of time and the display time of the fifth interactive image exceeds a preset target time, the fifth interactive image is automatically stored in the portable terminal.

[0017] In addition, the processor may be characterized in that it distinguishes a background layer, a fixed object layer, a moving object layer, and a responsive layer in at least one of the first interactive image and the second interactive image, and adds a preset moving object change layer and a responsive change layer linked to a motion image, a gesture image, and a touch image in the action information so as to change the moving object layer and the responsive layer according to the action of the experience viewer, and when the display time of at least one of the first interactive image and the second interactive image exceeds a preset target time, it controls the second external device so as to change and display an interactive image for each experience viewer, including the moving object change layer and the responsive change layer, on at least two or more walls of the experience space, an area between the walls, and a floor.

[0018] In addition, the processor may be characterized in that, when the motion image is a fourth image of the experience viewer moving, the gesture image is a fifth image of the experience viewer's gesture, and the touch image is a sixth image of the experience viewer's touch, the processor controls the second external device to add a responsive change layer to the surroundings of the experience viewer preset in connection with the fourth image, the fifth image, and the sixth image, and to display a changeable interactive image for each experience viewer including the responsive change layer on the wall, the space between the wall surfaces, and the floor.

[0019] In addition, a method for mutually linking interactive images between a portable terminal and an N-screen-based heterogeneous display, performed by a server according to another aspect of the present disclosure, comprises the steps of: receiving, through a communication unit of the server, action information of an experience viewer in an experience space from a first external device; determining, by a processor of the server, whether the action information is object flick information for flicking an object image in a portable terminal of the experience viewer; extracting, by the processor, object movement coordinates based on the object flick information when the action information is the object flick information; determining, by the processor, a first movement direction and a first movement position of the object image based on the object movement coordinates; extracting, by the processor, a first interactive image preset in connection with the first movement direction and the first movement position when the determination of the first movement direction and the first movement position is completed; and controlling, by the processor, a second external device to display the first interactive image on a wall surface of the experience space corresponding to the first movement direction and the first movement position.Including, by the processor, when the edited object image produced by the experiential viewer is opposed to the wall for a preset period of time, a step of extracting the position coordinates of the opposed wall based on the image of the portable terminal and the image of the wall obtained through the first external device, and the control step may further include, by the processor, when the determination of the second movement direction and the second movement position of the edited object image is completed based on the position coordinates, controlling the second external device to display the second interactive image extracted in connection with the second movement direction and the second movement position on the wall corresponding to the second movement direction and the second movement position, and by the processor, when the image of the portable terminal is opposed to the wall for a preset period of time and the display time of the second interactive image exceeds a preset target time, controlling the portable terminal and the second external device to automatically store the second interactive image in the portable terminal.;

[0020] In addition, a computer program stored in a computer-readable recording medium for executing the present disclosure may be further provided.

[0021] In addition, a computer-readable recording medium recording a computer program for executing a method for implementing the present disclosure may be further provided.

[0022] According to the aforementioned problem solving means of the present disclosure, it is possible to implement a natural, uninterrupted motion sensing effect when moving between walls.

[0023] In addition, according to the aforementioned problem solving means of the present disclosure, even non-experts and those who are digitally disadvantaged can create interactive images very easily and very quickly without difficult or complicated coding.

[0024] In addition, according to the aforementioned problem solving means of the present disclosure, interactive images can be displayed on the walls and floor of the experience space by changing them for each experience viewer, thereby providing enjoyable and useful interactive images for each experience viewer.

[0025] In addition, according to the aforementioned problem solving means of the present disclosure, an interactive image can be stored in a portable terminal, so that one can keep the interactive image one wants.

[0026] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0027] FIG. 1 is a diagram illustrating a system for interlinking interactive images between heterogeneous displays of the present disclosure.

[0028] Figure 2 is a drawing showing the configuration of the server of Figure 1.

[0029] FIGS. 3 to 10 are drawings illustrating a method for interlinking interactive images between heterogeneous displays of the present disclosure.

[0030] Figures 11a and 11b are drawings showing an example of a process for producing an interactive image between heterogeneous displays.

[0031] FIGS. 12 and 13f are drawings showing examples of a process for displaying and storing interactive images between heterogeneous displays.

[0032] Figure 14 is a diagram showing an example of a background layer, a fixed object layer, a moving object layer, and a responsive layer within an interactive video.

[0033] Figure 15 is a diagram showing an example of a process for outputting recommended moving object change layer data and responsive change layer data based on the input of experiential viewer data based on the artificial intelligence model of the server of Figure 1.

[0034] FIG. 16 is a drawing showing an example of a process of changing and displaying an interactive video for each viewer, including a responsive change layer, on a wall of an experience space through the first external device of FIG. 1.

[0035] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and any content that is common in the technical field to which this disclosure pertains or that overlaps between embodiments is omitted. The terms "part, module, element, block" used in the specification may be implemented in software or hardware, and depending on the embodiments, multiple "parts, modules, elements, blocks" may be implemented as a single component, or a single "part, module, element, block" may include multiple components.

[0036] Throughout the specification, when a part is said to be "connected" to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection via a wireless communication network.

[0037] Additionally, when a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0038] Throughout the specification, when we say that an element is "on" another element, this includes not only cases where the element is in contact with the other element, but also cases where another element exists between the two elements.

[0039] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0040] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0041] The identification codes for each step are used for convenience of explanation and do not describe the order of each step. Each step may be performed in a different order than specified unless the context clearly indicates a specific order.

[0042] The operating principle and embodiments of the present disclosure are described below with reference to the attached drawings.

[0043] The present disclosure may be characterized in that it receives behavior information of an experience viewer in an experience space from a first external device, determines whether the behavior information is object flick information that flicks an object image in a portable terminal of the experience viewer, if the behavior information is object flick information, extracts object movement coordinates based on the object flick information, determines a first movement direction and a first movement position of the object image based on the object movement coordinates, and if the determination of the first movement direction and the first movement position is completed, extracts a first interactive image preset in connection with the first movement direction and the first movement position, and controls a second external device to display the first interactive image on a wall surface of the experience space corresponding to the first movement direction and the first movement position.

[0044] This disclosure can realize the naturalness of seamless motion sensing effects when moving between walls. Furthermore, this disclosure allows even non-experts and those unfamiliar with digital technology to easily and quickly create interactive videos without difficult or complex coding. Furthermore, this disclosure can display interactive videos on the walls and floor of an experience space, changing them for each viewer, thereby providing enjoyable and informative interactive videos to each viewer.

[0045] Below, a system for interlinking interactive images between heterogeneous displays of the present disclosure will be examined in detail.

[0046] FIG. 1 is a diagram illustrating a system for interlinking interactive images between heterogeneous displays according to the present disclosure. FIG. 2 is a diagram illustrating the configuration of the server of FIG. 1.

[0047] FIGS. 3 to 10 are diagrams illustrating a method for interlinking interactive images between heterogeneous displays according to the present disclosure. FIGS. 11a and 11b are diagrams illustrating an example of a process for producing interactive images between heterogeneous displays.

[0048] Figures 12 and 13f are diagrams illustrating an example of a process for displaying and storing an interactive image between different displays. Figure 14 is a diagram illustrating an example of a background layer, a fixed object layer, a moving object layer, and a responsive layer within an interactive image.

[0049] Figure 15 is a diagram showing an example of a process for outputting recommended moving object change layer data and responsive change layer data based on the input of experiential viewer data based on the artificial intelligence model of the server of Figure 1.

[0050] FIG. 16 is a drawing showing an example of a process of changing and displaying an interactive video for each viewer, including a responsive change layer, on a wall of an experience space through the first external device of FIG. 1.

[0051] Referring to FIGS. 1 to 16, a system for interlinking interactive images between heterogeneous displays of the present disclosure may include a first external device (110), a second external device (120), and a server (130).

[0052] First, referring to FIG. 3, the method for interlinking interactive images between heterogeneous displays of the present disclosure may include a first receiving step (S310), a first judgment step (S320), a first extraction step (S330), a second judgment step (S340), a second extraction step (S350), and a first control step (S360).

[0053] As illustrated in Fig. 12, the first receiving step (S310) can receive an object image (11a) and behavior information in a portable terminal (10) of a viewer in an experience space from a first external device (110) via a communication unit (131). At this time, the viewer can retrieve the object image (11a) in the portable terminal (10) while running an interactive image production application of the portable terminal (10).

[0054] Here, the action information may further include object flick information (F1, F2, F3) that flicks the object image (11a). At this time, as illustrated in FIGS. 11a and 11b, the first external device (110) may include cameras (111, 112, 113) for obtaining motion coordinates, gesture coordinates, and touch coordinates in the action information. In addition, the first external device (110) may include cameras (111, 112, 113) for obtaining object movement coordinates (x1, y1)(x2, y2)(x3, y3) based on the object flick information (F1, F2, F3). Meanwhile, the first external device (110) may include a sensor for obtaining motion coordinates, gesture coordinates, and touch coordinates in the action information. At this time, the sensor may be a motion sensor, a gesture sensor, a touch sensor, a lidar sensor, etc.

[0055] The server (130) may include a communication unit (131), memory (132), and processor (133).

[0056] The communication unit (131) can communicate with the first external device (110) and the second external device (120) and receive action information from the first external device (110).

[0057] At this time, the communication unit (131) may include a wireless communication module that supports various wireless communication methods such as GSM (global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), UMTS (universal mobile telecommunications system), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, and 6G, in addition to a WiFi module and a WiBro (Wireless broadband) module.

[0058] In addition, the memory (132) can store data supporting various functions of the device, programs for the operation of the control unit, input / output data, and a plurality of application programs (or applications) run on the device, data for the operation of the device, and commands. At least some of these application programs can be downloaded from an external server via wireless communication.

[0059] The memory (132) may include at least one type of storage medium among a flash memory type, a hard disk type, an SSD (Solid State Disk type), an SDD (Silicon Disk Drive type), a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory (132) may be a database that is separate from the device but is connected by wire or wirelessly.

[0060] The processor (133) can control operations related to the interoperability of interactive images between heterogeneous displays. Here, the memory (132) and the processor (133) may each be implemented as separate chips. Alternatively, the memory (132) and the processor (133) may be implemented as a single chip.

[0061] The first judgment step (S320) can determine, through the processor (133), whether the action information is object flick information (F1, F2, F3) that flicks an object image (11a) in the portable terminal (10) of the experience viewer, as shown in FIG. 12.

[0062] The first extraction step (S330) can extract object movement coordinates (x1, y1)(x2, y2)(x3, y3) based on object flick information (F1, F2, F3) through the processor (133) when the action information is object flick information (F1, F2, F3).

[0063] The second judgment step (S340) can determine the first movement direction and first movement position of the object image (11a) based on the object movement coordinates (x1, y1)(x2, y2)(x3, y3) through the processor (133).

[0064] In the second extraction step (S350), when the first movement direction and first movement position of the object image (11a) have been determined through the processor (133), the first interactive images (11b1, 11b2, 11b3) set in connection with the first movement direction and the first movement position can be extracted.

[0065] The first control step (S360) can control the second external device (120) to display the first interactive images (11b1, 11b2, 11b3) on the walls (P4 to P6) and the floor (P7) of the experience space corresponding to the first movement direction and the first movement position through the processor (133).

[0066] At this time, the second external device (120) can display the first interactive images (11b1, 11b2, 11b3) based on N screens corresponding to the object image (11a) on the wall surfaces (P4 to P6) and the floor surface (P7). At this time, the experience space can be a space of various forms, such as a digital exhibition hall, an experience hall, or a sports facility.

[0067] Here, the second external device (120) may be a video content providing device based on a two-dimensional or three-dimensional stereoscopic projector. At this time, the second external device (120) may display at least one interactive image on at least two wall surfaces (P4 to P6) and a floor surface (P7). At this time, as illustrated in FIG. 11b, the second external device (120) may be four video content providing devices (121, 122, 123, 124) for displaying first interactive images (11b1, 11b2, 11b3) on three wall surfaces (P4, P5, P6) and a floor surface (P7), respectively.

[0068] Referring to FIGS. 4 and 5, the method for interlinking interactive images between heterogeneous displays of the present disclosure may include a second receiving step (S410), a third judgment step (S420), a third extraction step (S430), a fourth judgment step (S440), a fifth extraction step (S450), a second control step (S460), a fifth judgment step (S470), a sixth judgment step (S480), and a third control step (S490).

[0069] The second receiving step (S410) can receive an edited object image (12a) produced by an experience viewer from a first external device (110) through a communication unit (131), as shown in (a) of FIG. 13a.

[0070] The third judgment step (S420) can determine, through the processor (133), as shown in (a) of FIG. 13a, whether the edited object image (12a) produced by the experience viewer is facing one of the wall surfaces (P4, P5, P6) and the floor surface (P7) for a preset period of time.

[0071] The third extraction step (S430) can extract the position coordinates (x4, y4) of either the wall surface (P4, P5, P6) or the floor surface (P7) based on the image of the portable terminal (10) acquired through the first external device (110) and the image of either the wall surface (P4, P5, P6) or the floor surface (P7) when the edited object image (12a) is opposed to either the wall surface (P4, P5, P6) or the floor surface (P7) for a preset period of time, through the processor (133), as shown in (a) of FIG. 13a.

[0072] The fourth judgment step (S440) can determine the second movement direction and second movement position of the edited object image (12a) based on the position coordinates (x4, y4) through the processor (133), as shown in (a) of FIG. 13a.

[0073] In the fifth extraction step (S450), as illustrated in (a) of FIG. 13a, when the second movement direction and second movement position of the edited object image (12a) have been determined through the processor (133), a second interactive image set in advance in connection with the second movement direction and second movement position can be extracted. At this time, the second interactive image can include the edited object image (12a) and surrounding images of the edited object image (12a).

[0074] The second control step (S460) can control the second external device (120) to display the second interactive image (12a1) on one of the wall surfaces (P4, P5, P6) corresponding to the second movement direction and the second movement position, through the processor (133), as shown in (b) of FIG. 13a.

[0075] Here, the second external device (120) can display a second interactive image (12a1) based on N screens corresponding to the edited object image (12a) on one of the walls (P4, P5, P6). At this time, the second external device (120) can display a second interactive image (12a1) including the edited object image and the surrounding image of the edited object image on one of the walls (P4, P5, P6).

[0076] The fifth judgment step (S470) can determine, through the processor (133), whether the image of the portable terminal (10) received from the first external device (110) is facing one of the walls (P4, P5, P6) for a preset period of time, as illustrated in (c) of FIG. 13a.

[0077] The sixth judgment step (S480) can determine, through the processor (133), whether the display time of the second interactive image (12b1) received from the first external device (110) exceeds a preset target time when the image of the portable terminal (10) is facing one of the walls (P4, P5, P6), as shown in (c) of FIG. 13a.

[0078] The third control step (S490) can control the portable terminal (11) and the second external device (120) via the processor (133) so that, if the display time of the second interactive image (12b1) exceeds the target time, the second interactive image (12b1) is automatically stored in the portable terminal (11). At this time, the communication unit (131) transmits the second interactive image (12b1) received from the second external device (120) to the portable terminal (11), and the experience viewer can store the second interactive image (12b1) using the portable terminal (11).

[0079] Referring to FIGS. 6 to 9, the method for interlinking interactive images between heterogeneous displays of the present disclosure comprises a third receiving step (S610), a seventh judgment step (S621), an eighth judgment step (S622), a sixth extraction step (S630), a ninth judgment step (S640), a seventh extraction step (S650), a fourth control step (S660), a tenth judgment step (S671), an eleventh judgment step (S672), a fifth control step (S673), a twelfth judgment step (S681), a thirteenth judgment step (S682), a sixth control step (S683), a fourteenth judgment step (S810), an eighth extraction step (S820), a fifteenth judgment step (S830), a ninth extraction step (S840), a seventh control step (S850), a sixteenth judgment step (S860), a seventeenth judgment step (S870), an eighth control step (S850), a sixteenth judgment step (S860), a seventeenth judgment step (S870), and an eighth control step (S850). It may include step (S880).

[0080] The third receiving step (S610) can receive multiple edited object images produced by the experience viewer from the first external device (110) through the communication unit (131).

[0081] The seventh judgment step (S621) can determine, through the processor (133), as illustrated in (a) of FIG. 13b, whether a first edited object image (12b) among multiple edited object images produced by an experience viewer is facing a first position among the walls (P4, P5, P6) of the experience space during a preset first time period.

[0082] The eighth judgment step (S622) can determine, through the processor (133), whether, when the first edited object image (12b) is opposed to the first position, the second edited object image (12c) among the images including a plurality of edited objects is opposed to the second position, which is different from the first position, during a second time later than the first time set in advance, as shown in (b) of FIG. 13b.

[0083] In the sixth extraction step (S630), as shown in (a) and (b) of FIG. 13b, when the second edited object image (12c) is opposed to the second position, the processor (133) can extract the first coordinate (x5, y5) of the opposed first position and the second coordinate (x6, y6) of the opposed second position based on the image of the portable terminal (11) obtained through the first external device (110) and the image of one of the wall surfaces (P4, P5, P6).

[0084] The ninth judgment step (S640) can determine, as shown in (a) and (b) of FIG. 13b, whether the judgment of the third movement direction and the third movement position of the first edited object image (12b) based on the first coordinate (x5, y5) is completed through the processor (133), and whether the judgment of the fourth movement direction and the fourth movement position of the second edited object image (12c) based on the second coordinate (x6, y6) is completed.

[0085] In the seventh extraction step (S650), as illustrated in (a) and (b) of FIG. 13b, when the third movement direction and the third movement position of the first edited object image (12b) are determined based on the first coordinates (x5, y5) and the fourth movement direction and the fourth movement position of the second edited object image (12c) are determined based on the second coordinates (x6, y6), a third interactive image set in connection with the third movement direction and the third movement position can be extracted, and a fourth interactive image set in connection with the fourth movement direction and the fourth movement position can be extracted. At this time, the third interactive image can include the first edited object image (12b) and a surrounding image of the first edited object image (12b). In addition, the fourth interactive image can include the second edited object image (12c) and a surrounding image of the second edited object image (12c).

[0086] The fourth control step (S660) can control the second external device (120) to display the third interactive image (12b1) at a first position corresponding to the third movement direction and the third movement position, and to display the fourth interactive image (12c1) at a second position corresponding to the fourth movement direction and the fourth movement position, through the processor (133), as shown in (c) of FIG. 13c.

[0087] Here, the second external device (120) can display a third interactive image (12b1) and a fourth interactive image (12c1) based on N screens corresponding to the first edited object image (12b) and the second edited object image (12c) at the first location and the second location. At this time, the second external device (120) can display the third interactive image (12b1) including the first edited object image (12b) and the surrounding image of the first edited object image (12b) at the first location of any one of the walls (P4, P5, P6).

[0088] The 10th judgment step (S671) can determine, through the processor (133), whether the image of the portable terminal (10) received from the first external device (110) is facing the first position for a preset period of time, as illustrated in (d) of FIG. 13d.

[0089] The 11th judgment step (S672) can determine, through the processor (133), whether the display time of the third interactive image (12b1) exceeds the preset target time when the image of the portable terminal (10) is facing the first position, as shown in (d) of FIG. 13d.

[0090] The fifth control step (S673) can control the portable terminal (11) and the second external device (120) so that the third interactive image (12b1) is automatically stored in the portable terminal (11) when the display time of the third interactive image (12b1) exceeds the target time, as shown in (d) of FIG. 13d, through the processor (133).

[0091] At this time, the communication unit (131) transmits the third interactive image (12b1) received from the second external device (120) to the portable terminal (11), and the experience viewer can keep the third interactive image (12b1) using the portable terminal (11).

[0092] The 12th judgment step (S681) can determine, through the processor (133), whether the image of the portable terminal (10) received from the first external device (110) is facing the second position for a preset period of time, as shown in (e) of FIG. 13d.

[0093] The 13th judgment step (S682) can determine, through the processor (133), whether the display time of the fourth interactive image (12c1) exceeds the preset target time when the image of the portable terminal (10) is facing the second position, as shown in (e) of FIG. 13d.

[0094] The sixth control step (S683) can control the portable terminal (11) and the second external device (120) so that the fourth interactive image (12c1) is automatically stored in the portable terminal (11) when the display time of the fourth interactive image (12c1) exceeds the target time, as shown in (e) of FIG. 13d.

[0095] At this time, the communication unit (131) transmits the fourth interactive image (12c1) received from the second external device (120) to the portable terminal (11), and the experience viewer can keep the fourth interactive image (12c1) using the portable terminal (11).

[0096] The 14th judgment step (S810) can determine, through the processor (133), as shown in (f) of FIG. 13e, whether a third edited object image (12d) among the images including multiple edited objects produced by the experience viewer is facing a third position on the floor surface (P7) of the experience space during a third time later than a preset second time.

[0097] In the 8th extraction step (S820), as shown in (f) of FIG. 13e, when the third edited object image (12d) is opposed to the third position, the third coordinates (x7, y7) of the opposed third position can be extracted based on the image of the portable terminal (11) obtained through the first external device (110) and the image of the floor surface (P7) through the processor (133).

[0098] The 15th judgment step (S830) can determine the fifth movement direction and the fifth movement position of the third edited object image (12d) based on the third coordinate (x7, y7) through the processor (133), as shown in (f) of FIG. 13e.

[0099] In the ninth extraction step (S840), as illustrated in (f) of FIG. 13e, when the fifth movement direction and fifth movement position of the third edited object image (12d) have been determined through the processor (133), a preset fifth interactive image can be extracted in connection with the fifth movement direction and fifth movement position. At this time, the fifth interactive image can include the third edited object image (12d) and surrounding images of the third edited object image (12d).

[0100] The seventh control step (S850) can control the second external device (120) to display the fifth interactive image (12d1) at a third position corresponding to the fifth movement direction and the fifth movement position through the processor (133), as shown in (g) of FIG. 13e.

[0101] Here, the second external device (120) can display a fifth interactive image (12d1) based on an N screen corresponding to the third edited object image (12d) at a third location. At this time, the second external device (120) can display a fifth interactive image (12d1) including the third edited object image (12d) and a surrounding image of the third edited object image (12d) at a first location on the floor surface (P7).

[0102] The 16th judgment step (S860) can determine, through the processor (133), whether the image of the portable terminal (11) received from the first external device (110) is facing the third position for a preset period of time, as shown in (h) of FIG. 13f.

[0103] In the 17th judgment step (S870), as shown in (h) of FIG. 13f, the processor (133) can determine whether the display time of the fifth interactive image (12d1) exceeds the preset target time when the image of the portable terminal (11) is facing the third position.

[0104] The eighth control step (S880) can control the portable terminal (11) and the second external device (120) so that the fifth interactive image (12d1) is automatically stored in the portable terminal (11) when the display time of the fifth interactive image (12d1) exceeds the target time, as illustrated in (h) of FIG. 13f, through the processor (133).

[0105] At this time, the communication unit (131) transmits the fifth interactive image (12d1) received from the second external device (120) to the portable terminal (11), and the experience viewer can keep the fifth interactive image (12d1) using the portable terminal (11).

[0106] Referring to FIG. 10, the method for interlinking interactive images between heterogeneous displays of the present disclosure may include a distinction step (S1010), an additional step (S1020), and a ninth control step (S1030).

[0107] The distinction step (S1010) can distinguish, through the processor (133), a background layer (L1 of FIG. 14), a fixed object layer (L2 of FIG. 14), a moving object layer (L3 of FIG. 14), and a responsive layer (L4 of FIG. 14) within at least one of the first interactive image (11b) to the fifth interactive image (12d1).

[0108] An additional step (S1020) may add a preset moving object change layer and a responsive change layer linked to motion images, gesture images, and touch images in the action information to change the moving object layer (L3 of FIG. 14) and the responsive layer (L4 of FIG. 14) according to the actions of the experience viewer through the processor (133).

[0109] In the ninth control step (S1030), if the display time of at least one of the first interactive image (11b) to the fifth interactive image (12d1) exceeds a preset target time, the processor (133) can control the second external device (120) to display a changed interactive image for each viewer, including a moving object change layer and a responsive change layer, on at least two or more walls (any one of P4 to P6), a space between walls (any one of P4 to P6), and a floor surface (P7) of the experience space.

[0110] For example, the processor (133) may change the moving object layer (L3) and the responsive layers (L4, L41, L42) for each child, woman, and man based on the motion video, gesture video, touch video, and face video of the experience viewer received from the first external device (110), and may add a preset moving object change layer and a responsive change layer linked to the motion video, gesture video, touch video, and face video. At this time, the processor (133) may control the second external device (120) to change and display the interactive video for each experience viewer, including the moving object change layer and the responsive change layer, on at least two or more walls (any one of P4 to P6), the space between the walls (any one of P4 to P6), and the floor (P7) of the experience space.

[0111] For another example, the processor (133) may change the moving object layer (L3) and the responsive layers (L4, L41, L42) according to the age group of children, women, and men based on the motion video, gesture video, touch video, and face video of the experience viewer received from the first external device (110), and may add a preset moving object change layer and a responsive change layer linked to the motion video, gesture video, touch video, and face video. At this time, the processor (133) may control the second external device (120) to change and display the interactive video for each experience viewer, including the moving object change layer and the responsive change layer, on at least two or more walls (any one of P4 to P6), the space between the walls (any one of P4 to P6), and the floor (P7) of the experience space.

[0112] As another example, the processor (133) may determine that a woman and a man are in a preset area of ​​interest based on the motion video, gesture video, touch video, and face video of the experience viewer received from the first external device (110), and may add a preset moving object change layer and a responsive change layer linked to the motion video, gesture video, touch video, and face video to change the moving object layer (L3) and the responsive layers (L4, L41, L42) for each couple. At this time, the processor (133) may further add a preset moving object change layer and a responsive change layer linked to the motion video, gesture video, touch video, and face video to change the moving object layer (L3) and the responsive layers (L4, L41, L42) for each age group of the couple. At this time, the processor (133) can control the second external device (120) to display a changed interactive image for each viewer, including a moving object change layer and a responsive change layer, on at least two or more walls (any one of P4 to P6) of the experience space, a space between walls (any one of P4 to P6), and a floor surface (P7).

[0113] As another example, the processor (133) may determine that a child, a woman, and a man are in a preset area of ​​interest based on the motion video, gesture video, touch video, and face video of the experience viewer received from the first external device (110), and may add a preset moving object change layer and a responsive change layer linked to the motion video, gesture video, touch video, and face video to change the moving object layer (L3) and the responsive layers (L4, L41, L42) for each family member. At this time, the processor (133) may further add a preset moving object change layer and a responsive change layer linked to the motion video, gesture video, touch video, and face video to change the moving object layer (L3) and the responsive layers (L4, L41, L42) for each age group of the children in the family. At this time, the processor (133) can control the second external device (120) to display a changed interactive image for each viewer, including a moving object change layer and a responsive change layer, on at least two or more walls (any one of P4 to P6) of the experience space, a space between walls (any one of P4 to P6), and a floor surface (P7).

[0114] For another example, if the motion image is a fourth image of a moving experience viewer, the processor (133) may add a first responsive change layer to the surroundings of the preset experience viewer linked to the fourth image. At this time, the processor (133) may control the second external device (120) to change and display at least two or more walls (any one of P4 to P6), the space between the walls (any one of P4 to P6), and the floor (P7) of the experience space with an interactive image for each experience viewer that includes the first responsive change layer.

[0115] For another example, if the motion image is a fourth image of a moving viewer and the gesture image is a fifth image with a gesture of the viewer, the processor (133) may add a second responsive change layer to the surroundings of the viewer preset in connection with the fourth and fifth images. At this time, the processor (133) may control the second external device (120) to change and display at least two or more walls (any one of P4 to P6), the space between the walls (any one of P4 to P6), and the floor (P7) of the experience space into an interactive image for each viewer that includes the second responsive change layer.

[0116] For another example, if the motion image is a fourth image of a moving viewer, the gesture image is a fifth image with a gesture of the viewer, and the touch image is a sixth image with a touch of the viewer, the processor (133) may add a third responsive change layer to the surroundings of the viewer in advance linked to the fourth image, the fifth image, and the sixth image. At this time, the processor (133) may control the second external device (120) to change and display an interactive image for each viewer including the third responsive change layer on at least two or more walls (any one of P4 to P6), a space between walls (any one of P4 to P6), and a floor surface (P7) of the experience space.

[0117] Additionally, the processor (133) can extract interactive images for each viewer related to motion coordinates, gesture coordinates, and touch coordinates within the behavior information of the viewer.

[0118] For example, the processor (133) can extract interactive images for the first wall (P4), the second wall (P5), the third wall (P6), and the floor (P7) related to motion coordinates, gesture coordinates, and touch coordinates for each of a child, a woman, and a man based on the action information and face of the experience viewer acquired through a motion sensor, a gesture sensor, a touch sensor, and a camera.

[0119] For another example, the processor (133) can extract interactive images for the first wall (P4), the second wall (P5), the third wall (P6), and the floor (P7) related to motion coordinates, gesture coordinates, and touch coordinates by age group of children, women, and men, based on the action information and faces of the experience viewers acquired through the motion sensor, gesture sensor, touch sensor, and camera.

[0120] As another example, the processor (133) may determine that a woman and a man are a couple in a preset area of ​​interest based on the behavioral information and faces of the experience viewers acquired through a motion sensor, a gesture sensor, a touch sensor, and a camera, and may extract interactive images of the first wall (P4), the second wall (P5), the third wall (P6), and the floor (P7) related to the couple's motion coordinates, gesture coordinates, and touch coordinates.

[0121] As another example, the processor (133) may determine that a child, a woman, and a man are in a preset area of ​​interest based on the action information and faces of the experience viewers acquired through a motion sensor, a gesture sensor, a touch sensor, and a camera, and may extract interactive images of the first wall (P4), the second wall (P5), the third wall (P6), and the floor (P7) related to the motion coordinates, gesture coordinates, and touch coordinates of the family, and may determine that a child, a woman, and a man are in a preset area of ​​interest, and may extract interactive images of the first wall (P4), the second wall (P5), the third wall (P6), and the floor (P7), respectively, based on the motion coordinates, gesture coordinates, and touch coordinates of the family.

[0122] In addition, the processor (133) can control the second external device (120) to display interactive images that change for each viewer, including a moving object change layer and a responsive change layer, on the first wall (P4), the second wall (P5), the third wall (P6), and the floor (P7) of the experience space. At this time, the processor (133) can set a background layer produced in an unfolded form of the first wall (P4), the second wall (P5), the third wall (P6), and the floor (P7). In addition, the processor (133) can further set a fixed object layer (L2), a moving object layer (L3), responsive layers (L4, L41, L42), a moving object change layer, and a responsive change layer produced in an unfolded form of the first wall (P4), the second wall (P5), the third wall (P6), and the floor (P7).

[0123] For example, the processor (133) can control the second external device (120) to display a child, woman, and man-specific change interactive image including a moving object change layer and a responsive change layer for each child, woman, and man on the first wall (P4), the second wall (P5), the third wall (P6), and the floor (P7) of the experience space.

[0124] For another example, the processor (133) may control the second external device (120) to display interactive images that change according to the age groups of a child, a woman, and a man, including a moving object change layer and a responsive change layer according to the age groups of a child, a woman, and a man, on the first wall (P4), the second wall (P5), the third wall (P6), and the floor (P7) of the experience space.

[0125] As another example, the processor (133) may control the second external device (120) to display, by changing, a couple-specific interactive image including a moving object change layer and a responsive change layer for each couple on the first wall (P4), the second wall (P5), the third wall (P6), and the floor (P7) of the experience space. At this time, the processor (133) may further control the second external device (120) to display, by changing, a couple-specific interactive image including a moving object change layer and a responsive change layer for each age group of the couple on the first wall (P4), the second wall (P5), the third wall (P6), and the floor (P7) of the experience space.

[0126] As another example, the processor (133) may control the second external device (120) to display, by changing, a family-specific interactive image including a moving object change layer and a responsive change layer for each family member, on the first wall (P4), the second wall (P5), the third wall (P6), and the floor (P7) of the experience space. At this time, the processor (133) may further control the second external device (120) to display, by changing, a family-specific interactive image including a moving object change layer and a responsive change layer for each age group of the children in the family, on the first wall (P4), the second wall (P5), the third wall (P6), and the floor (P7) of the experience space.

[0127] Referring to FIG. 15, the server (130) can learn input values ​​of content data by age group of a child (ID1), content data by age group of a woman (ID2), content data by age group of a man (ID3), content data by age group of a couple (ID4), and content data by age group of a child in a family (ID5) among metadata based on an artificial intelligence model (AIM) and output result values ​​of recommended moving object change layer data (OD1) and responsive change layer data (OD2).

[0128] At this time, the artificial intelligence model (AIM) can be constructed and reinforced learning as a learning data set using a CNN algorithm or an RNN algorithm using content data by age group of various children (ID1), content data by age group of various women (ID2), content data by age group of various men (ID3), content data by age group of various couples (ID4), and content data by age group of various children in various families (ID5). At this time, the memory (132) can store the recommended moving object change layer data (OD1) and responsive change layer data (OD2) learned and analyzed based on the artificial intelligence model (AIM).

[0129] For example, content data (ID1) by child's age group may be content for finding animals, content for finding plants, etc. for toddlers, content for finding numbers, content for finding Korean letters, content for finding alphabets, etc. for children, and content for finding English words, etc. for elementary school students.

[0130] At this time, the server (130) can control the second external device (120) to change and display the moving object change layer data of the animal moving in connection with the content of finding the animal and the responsive change layer data of the animal appearing on the walls and floor of the experience space.

[0131] Additionally, the server (130) can control the second external device (120) to change and display the moving object change layer data of the plant moving in connection with the content of finding the plant and the responsive change layer data of the plant appearing on the walls and floor of the experience space.

[0132] Additionally, the server (130) can control the second external device (120) to change and display the moving object change layer data in which the number moves and the responsive change layer data in which the number appears, which are linked to the content for finding the number, on the walls and floor of the experience space.

[0133] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data in which the Korean text moves and the responsive change layer data in which the Korean text appears, linked to the content searching for the Korean text, on the walls and floor of the experience space.

[0134] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data in which the alphabet moves and the responsive change layer data in which the alphabet appears, linked to the content for finding the alphabet, on the walls and floor of the experience space.

[0135] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data in which the English word is linked to the content for finding the English word and the responsive change layer data in which the English word appears on the walls and floor of the experience space.

[0136] Here, if the number of touches on content for finding animals, content for finding plants, content for finding numbers, content for finding Korean characters, content for finding alphabets, and content for finding English words exceeds a preset standard number of times, the server (130) can output the face of a child whose number of touches exceeds the standard number of times based on the facial information acquired through the camera, so as to provide a product to the child whose number of touches exceeds the standard number of times. At this time, the administrator can check the output face of the child and provide the product to the child.

[0137] For another example, content data (ID2) by age group for women in their 20s and 30s could be content looking for male celebrities, cosmetics, clothes, flowers, etc., and for women in their 40s and 60s, it could be content looking for cosmetics, clothes, flowers, health food, skin care products, etc.

[0138] At this time, the server (130) can control the second external device (120) to change and display the moving object change layer data of the male celebrity moving in connection with the content searching for the male celebrity and the responsive change layer data in which the male celebrity appears on the walls and floor of the experience space.

[0139] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data in which the cosmetics are moved and the responsive change layer data in which the cosmetics appear, linked to the content for finding cosmetics, on the walls and floor of the experience space.

[0140] Additionally, the server (130) can control the second external device (120) to change and display moving object change layer data in which clothes move and responsive change layer data in which clothes appear, linked to the content for finding clothes, on the walls and floor of the experience space.

[0141] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data of the flower moving linked to the flower finding content and the responsive change layer data of the flower appearing on the walls and floor of the experience space.

[0142] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data in which the health food linked to the content for finding health food moves and the responsive change layer data in which the health food appears on the walls and floor of the experience space.

[0143] In addition, the server (130) can control the second external device (120) to change and display moving object change layer data in which skin care products move and responsive change layer data in which skin care products appear, linked to content searching for skin care products, on the walls and floor of the experience space.

[0144] Here, if the number of touches on content searching for male celebrities, cosmetics, clothes, flowers, health food, and skin care products exceeds a preset standard number of touches, the server (130) can output the face of a woman whose touches exceed the standard number based on the facial information acquired through the camera, thereby providing a product to the woman whose touches exceed the standard number. At this time, the administrator can confirm the outputted face of the woman and provide the product to the woman.

[0145] For another example, content data by age group (ID3) for men in their 20s and 30s could be content looking for female celebrities, cosmetics, clothes, cars, etc., and for men in their 40s and 60s, it could be content looking for cosmetics, clothes, cars, health food, real estate, etc.

[0146] At this time, the server (130) can control the second external device (120) to change and display the moving object change layer data of the female celebrity moving in connection with the content searching for the female celebrity and the responsive change layer data in which the female celebrity appears on the walls and floor of the experience space.

[0147] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data in which the cosmetics are moved and the responsive change layer data in which the cosmetics appear, linked to the content for finding cosmetics, on the walls and floor of the experience space.

[0148] Additionally, the server (130) can control the second external device (120) to change and display moving object change layer data in which clothes move and responsive change layer data in which clothes appear, linked to the content for finding clothes, on the walls and floor of the experience space.

[0149] Additionally, the server (130) can control the second external device (120) to change and display the moving object change layer data of the car moving in connection with the content of finding the car and the responsive change layer data of the car appearing on the walls and floor of the experience space.

[0150] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data in which the health food linked to the content for finding health food moves and the responsive change layer data in which the health food appears on the walls and floor of the experience space.

[0151] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data of the real estate listing linked to the content for finding real estate listings and the responsive change layer data in which the real estate listings appear on the walls and floor of the experience space.

[0152] Here, if the number of touches on content searching for female celebrities, cosmetics, clothes, cars, health foods, and real estate listings exceeds a preset standard number of touches, the server (130) can output the face of a man whose touches exceed the standard number based on the facial information acquired through the camera, thereby providing a product to the man whose touches exceed the standard number. At this time, the administrator can confirm the outputted face of the man and provide the product to the man.

[0153] The content data (ID4) by age group of couples may include content for finding musical venues, movies, cafes, travel destinations, and restaurants for those in their 20s to 40s, and content for finding massage venues, sports facilities, and hot springs for those in their 50s to 60s.

[0154] At this time, the server (130) can control the second external device (120) to change and display the moving object change layer data of the musical place linked to the content for finding the musical place and the responsive change layer data of the musical place appearing on the walls and floor of the experience space.

[0155] Additionally, the server (130) can control the second external device (120) to change and display the moving object change layer data in which the movie moves and the responsive change layer data in which the movie appears, linked to the content for finding the movie, on the walls and floor of the experience space.

[0156] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data of the cafe moving in connection with the content of finding the cafe and the responsive change layer data of the cafe appearing on the walls and floor of the experience space.

[0157] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data of the travel destination linked to the content for finding the travel destination and the responsive change layer data showing the travel destination on the walls and floor of the experience space.

[0158] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data of the restaurant linked to the content of finding a restaurant and the responsive change layer data of the restaurant appearing on the walls and floor of the experience space.

[0159] In addition, the server (130) can control the second external device (120) to change and display moving object change layer data of a massage place linked to content for finding a massage place and responsive change layer data of a massage place appearing on the walls and floor of the experience space.

[0160] In addition, the server (130) can control the second external device (120) to change and display moving object change layer data of a moving sports facility linked to content for finding a sports facility and responsive change layer data of a sports facility appearing on the walls and floor of the experience space.

[0161] In addition, the server (130) can control the first external device (110) to change and display the moving object change layer data of the hot spring location linked to the content for finding a hot spring location and the responsive change layer data in which the hot spring location appears on the walls and floor of the experience space.

[0162] Here, if the number of touches on content for finding a musical venue, content for finding a movie, content for finding a cafe, content for finding a travel destination, content for finding a restaurant, content for finding a massage venue, content for finding a sports facility, and content for finding a hot spring is greater than a preset standard number of times, the server (130) can output the faces of couples whose touches are greater than the standard number of times based on the facial information acquired through the camera, so as to provide a product to the couple whose touches are greater than the standard number of times. At this time, the administrator can check the faces of the output couples and provide a product to the couple.

[0163] Content data by age group of the child in the family (ID5) can be content for finding kids' cafes, travel destinations with toddlers, and restaurants with toddlers if the child is a toddler, content for finding children's musicals, and children's movies if the child is a young child, and content for finding history tours, and history museums if the child is a teenager.

[0164] At this time, the server (130) can control the second external device (120) to change and display the moving object change layer data of the moving kids cafe linked to the content for finding the kids cafe and the responsive change layer data of the kids cafe appearing on the walls and floor of the experience space.

[0165] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data of the travel destination with infants linked to the content for finding a travel destination with infants and the responsive change layer data showing the travel destination with infants on the walls and floor of the experience space.

[0166] In addition, the server (130) can control the second external device (120) to change and display, on the walls and floor of the experience space, moving object change layer data of a restaurant with toddlers linked to content for finding a restaurant with toddlers and responsive change layer data showing a travel destination with toddlers.

[0167] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data of the moving children's musical linked to the content searching for the children's musical and the responsive change layer data of the children's musical appearing on the walls and floor of the experience space.

[0168] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data of the children's movie linked to the content searching for the children's movie and the responsive change layer data in which the children's movie appears on the walls and floor of the experience space.

[0169] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data of the historical tour linked to the content seeking the historical tour and the responsive change layer data of the historical tour appearing on the walls and floor of the experience space.

[0170] In addition, the server (130) can control the second external device (120) to change and display the moving object change layer data of the history museum linked to the content of finding the history museum and the responsive change layer data of the history museum appearing on the walls and floor of the experience space.

[0171] Here, if the number of touches on content for finding a kids' cafe, a travel destination for children, a restaurant for children, a musical for children, a movie for children, a historical tour, or a history museum is greater than a preset standard number of times, the server (130) can output the faces of families whose touches are greater than the standard number of times based on the facial information acquired through the camera, so as to provide a product to the family whose touches are greater than the standard number of times. At this time, the manager can check the faces of the families whose touches are output and provide the product to the corresponding families.

[0172] Meanwhile, the processor (133) can also control the second external device (120) to display interactive images that change for each viewer on the wall and the floor, respectively.

[0173] Additionally, the processor (133) can control the second external device (120) to display an interactive image including a fixed object layer (L2), a moving object layer (L3), and responsive layers (L4, L41, L42) between the first wall surface (P4) and the second wall surface (P5), between the first wall surface (P4) and the floor surface (P7), between the second wall surface (P5) and the floor surface (P7), and between the first wall surface (P4), the second wall surface (P5), and the floor surface (P7).

[0174] In addition, the processor (133) can control the second external device (120) to display an interactive image for each viewer, including a moving object change layer and a responsive change layer, between the first wall (P4) and the second wall (P5), between the first wall (P4) and the floor (P7), between the second wall (P5) and the floor (P7), and between the first wall (P4), the second wall (P5), and the floor (P7).

[0175] Additionally, the processor (133) can control the second external device (120) to display a changed interactive image for each viewer on the first wall (P4), the second wall (P5), the third wall (P6), and the floor (P7).

[0176] In addition, the processor (133) generates an interactive image including a fixed object layer (L2), a moving object layer (L3), and responsive layers (L4, L41, L42) between a first wall surface (P4) and a second wall surface (P5), between a first wall surface (P4) and a third wall surface (P6), between a first wall surface (P4) and a floor surface (P7), between a second wall surface (P5) and a third wall surface (P6), between a second wall surface (P5) and a floor surface (P7), between a third wall surface (P6) and a floor surface (P7), between a first wall surface (P4), a second wall surface (P5), and a third wall surface (P6), between a first wall surface (P4), a second wall surface (P5), and a floor surface (P7), between a first wall surface (P4), a second wall surface (P5), and a floor surface (P7), between a first wall surface (P4), a third wall surface (P6), and a floor surface (P7), and a second The second external device (120) can be controlled to display between the wall surface (P5), the third wall surface (P6), and the floor surface (P7), and between the first wall surface (P4), the second wall surface (P5), the third wall surface (P6), and the floor surface (P7).

[0177] In addition, the processor (133) displays an interactive image for each viewer, including a moving object change layer and a responsive change layer, between the first wall (P4) and the second wall (P5), between the first wall (P4) and the third wall (P6), between the first wall (P4) and the floor (P7), between the second wall (P5) and the third wall (P6), between the second wall (P5) and the floor (P7), between the third wall (P6) and the floor (P7), between the first wall (P4), the second wall (P5), and the third wall (P6), between the first wall (P4), the second wall (P5), and the floor (P7), between the first wall (P4), the third wall (P6), and the floor (P7). The second external device (120) can be controlled to display between the second wall surface (P5), the third wall surface (P6), and the floor surface (P7), and to display between the first wall surface (P4), the second wall surface (P5), the third wall surface (P6), and the floor surface (P7).

[0178] As illustrated in FIG. 16, if the motion image is a seventh image of an experience viewer moving along a flower, the processor (133) can extract a first responsive change layer in which flowers bloom in front, beside, and behind the experience viewer linked to the seventh image. At this time, the processor (133) can control the second external device (120) to display an interactive image (P41) for each experience viewer, including the first responsive change layer (P41-1), by changing it on the wall (P4) and the floor of the experience space.

[0179] In addition, if the motion image is a seventh image of an experience viewer moving along a flower, and the gesture image is an eighth image of a viewer holding a flower with a hand, kicking a flower with a foot, or shaking a flower with a hand, the processor (133) can extract a second responsive change layer in which flowers fall in front, to the side, and behind the experience viewer linked to the seventh and eighth images. At this time, the processor (133) can control the second external device (120) to change and display an interactive image for each experience viewer, including the second responsive change layer, on the walls and floor of the experience space.

[0180] In addition, if the motion image is a seventh image of an experience viewer moving along a flower, the gesture image is a ninth image of an experience viewer embracing a flower, and the touch image is a tenth image of an experience viewer touching a flower embracing the flower, the processor (133) may extract a third responsive change layer in which flowers grow to a preset size and scatter from the experience viewer's arms linked to the seventh, ninth, and tenth images. At this time, the processor (133) may control the second external device (120) to change and display an interactive image for each experience viewer, including the third responsive change layer, on the walls and floor of the experience space.

[0181] The present disclosure can produce an N-screen-based interactive video for application in a digital immersive experience space. In other words, the present disclosure can produce an innovative N-screen-based interactive video that can maximize the immersion of an experiential exhibition hall with floor, ceiling, and wall surfaces of varying sizes and shapes.

[0182] The present disclosure can render and project interactive images on a surface in one computer / program.

[0183] The present disclosure can realize the naturalness of a motion sensing effect without interruption when moving between walls.

[0184] This disclosure allows even non-experts and those with limited digital skills to create interactive videos very easily and quickly without difficult or complex coding.

[0185] The present disclosure can display interactive images that change for each viewer on the walls and floor of an experience space, thereby providing enjoyable and informative interactive images for each viewer.

[0186] The present disclosure enables the storage of interactive images in a portable terminal, allowing a user to keep the interactive images he or she desires.

[0187] At least one component may be added or deleted in accordance with the performance of the components illustrated in FIGS. 1, 2, and 11a through 16. Furthermore, it will be readily apparent to those skilled in the art that the relative positions of the components may be altered in accordance with the performance or structure of the system.

[0188] Although FIGS. 3 to 10 describe the sequential execution of multiple steps, this is merely an example of the technical idea of ​​the present embodiment, and a person having ordinary skill in the art to which the present embodiment pertains can modify and apply various modifications and variations by changing the order described in FIGS. 3 to 10 without departing from the essential characteristics of the present embodiment, or by executing one or more of the multiple steps in parallel. Therefore, FIGS. 3 to 10 are not limited to a chronological order.

[0189] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing computer-executable instructions. The instructions may be stored in the form of program code, and when executed by a processor, may generate program modules to perform the operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0190] Computer-readable storage media include all types of storage media that store instructions that can be deciphered by a computer. Examples include read-only memory (ROM), random access memory (RAM), magnetic tape, magnetic disks, flash memory, and optical data storage devices.

[0191] The disclosed embodiments have been described with reference to the attached drawings as described above. Those skilled in the art will understand that the present disclosure can be implemented in forms other than the disclosed embodiments without altering the technical spirit or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be construed as limiting.

Claims

1. A first external device that obtains object images and behavior information from a portable terminal of an experience viewer in an experience space, a communication unit that performs communication with a plurality of second external devices that display N-screen-based interactive images corresponding to the object images in the experience space; and A processor for controlling operations related to the mutual linkage of interactive images between the portable terminal and the N-screen-based heterogeneous display, The above processor, If the action information received from the first external device through the communication unit is object flick information that flicks the object image, the object movement coordinates are extracted based on the object flick information, When the determination of the first movement direction and the first movement position of the object image is completed based on the object movement coordinates, the second external device is controlled to display the first interactive image extracted in connection with the first movement direction and the first movement position on the wall surface of the experiential space corresponding to the first movement direction and the first movement position. When the edited object image produced by the above-mentioned experience viewer is opposed to the wall for a preset period of time, the position coordinates of the opposed wall are extracted based on the image of the portable terminal obtained through the first external device and the image of the wall, When the second movement direction and the second movement position of the edited object image are determined based on the above location coordinates, the second external device is controlled to display the second interactive image extracted in connection with the second movement direction and the second movement position on the wall corresponding to the second movement direction and the second movement position. A server characterized in that it controls the portable terminal and the second external device so that the second interactive image is automatically stored in the portable terminal when the image of the portable terminal is in a state facing the wall for a preset period of time and the display time of the second interactive image exceeds a preset target time.

2. In paragraph 1, The above processor, When a first edited object image among the plurality of edited object images produced by the above-mentioned experience viewer is opposed to a first position on the wall of the above-mentioned experience space for a first preset time, and a second edited object image among the plurality of edited object images is opposed to a second position different from the first position for a second time later than the first preset time, a first coordinate of the opposed first position and a second coordinate of the opposed second position are extracted based on the image of the portable terminal acquired through the first external device and the image of the wall, A server characterized in that, when the determination of the third movement direction and the third movement position of the first edited object image is completed based on the first coordinates, and the determination of the fourth movement direction and the fourth movement position of the second edited object image is completed based on the second coordinates, the server controls the second external device to display a third interactive image extracted in connection with the third movement direction and the third movement position at the first position corresponding to the third movement direction and the third movement position, and to display a fourth interactive image extracted in connection with the fourth movement direction and the fourth movement position at the second position corresponding to the fourth movement direction and the fourth movement position.

3. In paragraph 2, The above processor, When the image of the portable terminal obtained through the first external device is in a state of facing the first position for a preset time and the display time of the third interactive image exceeds a preset target time, the portable terminal and the second external device are controlled so that the third interactive image is automatically stored in the portable terminal, A server characterized in that it controls the portable terminal and the second external device so that, when the image of the portable terminal obtained through the first external device is in a state of facing the second location for a preset period of time and the display time of the fourth interactive image exceeds a preset target time, the fourth interactive image is automatically stored in the portable terminal.

4. In paragraph 2, The above processor, When a third edited object among the plurality of edited object images is opposed to a third position on the floor of the experiential space during a third time later than a preset second time, a third coordinate of the opposed third position is extracted based on the image of the portable terminal acquired through the first external device and the image of the floor, A server characterized in that, when the determination of the fifth movement direction and the fifth movement position of the third image is completed based on the third coordinates, the second external device is controlled to display the fifth interactive image extracted in connection with the fifth movement direction and the fifth movement position at the third position corresponding to the fifth movement direction and the fifth movement position.

5. In paragraph 4, The above processor, If the image of the portable terminal obtained through the first external device is in a state facing the third location for a preset period of time, and the display time of the fifth interactive image exceeds the preset target time, A server characterized in that it controls the portable terminal and the second external device so that the fifth interactive image is automatically stored in the portable terminal.

6. In paragraph 1, The above processor, Distinguish a background layer, a fixed object layer, a moving object layer, and a responsive layer within at least one of the first interactive image and the second interactive image, respectively; The moving object layer and the responsive layer according to the actions of the above-mentioned experience viewers To change the motion image, gesture image, and touch image in the above action information, add a preset moving object change layer and a responsive change layer, A server characterized in that, when the display time of at least one of the first interactive image and the second interactive image exceeds a preset target time, the server controls the second external device to display a changed interactive image for each viewer, including the moving object change layer and the responsive change layer, on at least two walls of the experience space, the space between the walls, and the floor.

7. In paragraph 6, The above processor, The above motion video is the fourth video of the above experience viewer moving, The above gesture video is the fifth video with the gestures of the above experience viewer, If the above touch image is the 6th image with the touch of the experience viewer, Pre-set in connection with the above 4th image, the above 5th image, and the above 6th image Add a responsive change layer around the experience viewer, A server characterized in that it controls the second external device to display a changeable interactive image for each viewer, including the responsive change layer, on the wall, the space between the wall surfaces, and the floor.

8. A method for interconnecting interactive images between a portable terminal and an N-screen-based heterogeneous display performed by a server, A step of receiving behavior information of an experience viewer in an experience space from a first external device through a communication unit of the above server; A step of determining, by the processor of the server, whether the action information is object flick information that flicks an object image in the portable terminal of the experience viewer; A step of extracting object movement coordinates based on the object flick information, when the action information is the object flick information, by the processor; A step of determining a first movement direction and a first movement position of the object image based on the object movement coordinates by the processor; A step of extracting a preset first interactive image linked to the first movement direction and the first movement position when the determination of the first movement direction and the first movement position is completed by the processor; and A step of controlling a second external device to display the first interactive image on a wall surface of the experiential space corresponding to the first movement direction and the first movement position by the processor; including; The processor further includes a step of extracting the position coordinates of the opposed wall based on the image of the portable terminal obtained through the first external device and the image of the wall when the edited object image produced by the experiential viewer is opposed to the wall for a preset period of time. The above control step is, When the second movement direction and the second movement position of the edited object image are determined based on the position coordinates by the processor, the second external device is controlled to display the second interactive image extracted in connection with the second movement direction and the second movement position on the wall surface corresponding to the second movement direction and the second movement position. A method characterized in that, when the image of the portable terminal is in a state of facing the wall for a preset time and the display time of the second interactive image exceeds a preset target time, the portable terminal and the second external device are controlled by the processor so that the second interactive image is automatically stored in the portable terminal.

Images