Method for controlling virtual reality apparatus and virtual reality apparatus using same

Abstract

Disclosed are a method for controlling a virtual reality apparatus and a virtual reality apparatus, the method and apparatus: setting a first region and a second region different from the first region in a space in which content is viewed, based on scanning space information on the space in which the content is viewed and content information on the content; using at least one image sensor to sense the first region and the second region to track an object located in the first region or the second region based on a sensing result; and controlling display of a screen of the virtual reality device based on the tracking result.

Description

Technical Field

[0001] Provided are a method for controlling a virtual reality device and a virtual reality device using the method. Background Art

[0002] Virtual reality refers to the display of virtual images in a virtual space that can be experienced as real life. A virtual reality device indicates a device that can express "virtual reality" and may include a head-mounted display device that completely covers the user's field of view, a virtual reality helmet, or a goggle-type display device. Summary of the invention

[0003] According to an embodiment of the present disclosure, a method for controlling a virtual reality device is provided. The method for controlling the virtual reality device may include: setting a first area and a second area different from the first area in the space where the content is viewed based on scanning space information about the space where the content is viewed and content information about the content. In addition, the method for controlling the virtual reality device may include: sensing the first area and the second area by using at least one image sensor. In addition, the method for controlling the virtual reality device may include: tracking an object located in the first area or the second area based on a result of the sensing. In addition, the method for controlling the virtual reality device may include: controlling a screen display of the virtual reality device based on a result of the tracking.

[0004] According to an embodiment of the present disclosure, there may be provided a computer-readable recording medium having recorded thereon a program for executing the method.

[0005] According to an embodiment of the present disclosure, a virtual reality device is provided. The virtual reality device may include a memory storing one or more instructions, a processor, at least one image sensor, a display engine, and a display. The processor is configured to run one or more instructions to set a first area and a second area different from the first area in the space where the content is viewed based on scanned space information about the space where the content is viewed and content information about the content. In addition, the processor is configured to run one or more instructions to sense the first area and the second area by using at least one image sensor. In addition, the processor is configured to run one or more instructions to track an object located in the first area or the second area based on the result of sensing. In addition, the processor is configured to run one or more instructions to control the screen display of the virtual reality device by using the display engine and the display based on the result of tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 is a diagram illustrating that the virtual reality device controls a screen display of the virtual reality device when an object approaches while a user wearing the virtual reality device views content via the virtual reality device according to an embodiment of the present disclosure.

[0007] Figure 2 is a flowchart for describing a method of controlling a virtual reality device according to an embodiment of the present disclosure.

[0008] Figure 3 is a detailed flowchart of an operation of setting a first area and a second area in a space for viewing content according to an embodiment of the present disclosure.

[0009] Figure 4 is a diagram illustrating an example of a process for setting a first area and a second area in a space for viewing content on a screen of a virtual reality device according to an embodiment of the present disclosure.

[0010] Figure 5 2 is a diagram for describing an area range of a first area according to a type of content in a virtual reality device according to an embodiment of the present disclosure.

[0011] Figure 6 is a detailed flowchart of an operation of setting a first area and a second area in a space for viewing content according to an embodiment of the present disclosure.

[0012] Figure 7 is a diagram illustrating an example of a process for setting a first area and a second area in a space for viewing content on a screen of a virtual reality device according to an embodiment of the present disclosure.

[0013] Figure 8 is a detailed flowchart of an operation of setting a first area and a second area in a space for viewing content according to an embodiment of the present disclosure.

[0014] Fig. 9 is a diagram illustrating an example of a process for setting a first area and a second area in a space for viewing content on a screen of a virtual reality device according to an embodiment of the present disclosure.

[0015] Fig.10 is a detailed flowchart of an operation of tracking an object located in a first area or a second area according to an embodiment of the present disclosure.

[0016] Fig.11 is a detailed flowchart of an operation of controlling a screen display of a virtual reality device according to an embodiment of the present disclosure.

[0017] Fig.12 2 is a diagram illustrating that a virtual reality device controls a screen display according to whether an object is located in a first area and a second area according to an embodiment of the present disclosure.

[0018] Fig.13 is a diagram illustrating an example in which a virtual reality device tracks an object by communicating with a device of the object located in a second area and displays an indicator corresponding to the object on a content reproduction screen according to an embodiment of the present disclosure.

[0019] Fig.14 is a block diagram for describing a configuration of a virtual reality device according to an embodiment of the present disclosure.

[0020] Fig.15 is a block diagram for describing a configuration of a server for communicating with a virtual reality device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] The terms used in this specification will be briefly defined, and the present disclosure will be described in detail. Throughout the disclosure, the expression "at least one of a, b, or c" indicates only a, only b, only c, a and b, a and c, b and c, and all of a, b, and c or variations thereof.

[0022] Although the terms used in the present disclosure are selected from common terms currently widely used in consideration of their functions in the present disclosure, these terms may change according to the intention of a person of ordinary skill in the art, precedents, or the emergence of new technologies. In addition, in specific cases, the terms are arbitrarily selected by the applicant of the present disclosure, and the meanings of these terms will be described in detail in the corresponding parts of the detailed description. Therefore, the terms used in the present disclosure are not just the names of the terms, but are defined based on the meanings of the terms and the entire disclosure.

[0023] As used herein, the singular form is intended to also include the plural form, unless the context clearly indicates otherwise. Unless otherwise defined, all terms including technical terms or scientific terms used herein may have the same meaning as those generally understood by those of ordinary skill in the art of this specification. It should be understood that although the terms "first", "second", etc. may be used in this specification to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element.

[0024] Throughout the specification, when a component "includes" or "contains" an element, unless there is a specific description to the contrary, the component may also include other elements, and other elements are not excluded. In addition, terms such as "unit", "module", etc. used in this specification indicate a unit that processes at least one function or operation, and the unit or module may be implemented by hardware or software, or a combination of hardware and software.

[0025] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to allow those skilled in the art to easily implement the embodiments. However, the present disclosure may be implemented in many different forms and should not be construed as being limited to the embodiments set forth herein.

[0026] Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

[0027] Figure 1 is a diagram illustrating that the virtual reality device 100 controls a screen display of the virtual reality device 100 when an object approaches while a user wearing the virtual reality device 100 views content via the virtual reality device 100 according to an embodiment of the present disclosure.

[0028] In the present disclosure, the virtual reality device 100 is a device capable of expressing virtual reality. "Virtual reality" indicates that a virtual image is also displayed in the digital environment space of the virtual world, or an image corresponding to a real-world object is displayed together with the virtual image. The virtual reality device 100 may be in the form of a head mounted display (HMD) device or a virtual reality helmet.

[0029] The server 200 provides virtual reality services or content based on virtual reality services to the virtual reality devices 100. The server 200 may receive information related to the virtual reality services from the virtual reality devices 100, or may send information related to the virtual reality services to the virtual reality devices 100. The server 200 may provide a platform for the virtual reality services, and may provide the virtual reality services or content based on the virtual reality services to the virtual reality devices 100 of users who register in or subscribe to the virtual reality services. The virtual reality devices 100 may transmit and receive information to each other via the server 200, and the virtual reality devices 100 within a preset range may be connected to each other via mutual directional communication.

[0030] Reference Figure 1 , the virtual reality device 100 according to an embodiment of the present disclosure may set different areas in the space where the content is viewed, and may track objects in the set areas. The virtual reality device 100 may control the screen display of the virtual reality device 100 according to whether the object exists in the first area or the second area set in the space where the content is viewed. The first area is a space having a size required for the user wearing the virtual reality device 100 to immersively view or experience the content, and corresponds to an immersive area. The second area is a preset space where the user wearing the virtual reality device 100 recognizes the approach of an external object, and corresponds to a safe area.

[0031] Reference Figure 1 As shown, when a user wears the virtual reality device 100 and plays a game, the virtual reality device 100 can sense the first area and the second area set in the space where the content is viewed. Since the user wearing the virtual reality device 100 is in a light-shielded state where the external environment is not visible, the virtual reality device 100 can sense the surrounding environment by using a camera, a microphone, etc.

[0032] When an object outside the second area (such as another user) enters the second area, the virtual reality device 100 may detect and track the object in the second area. When the virtual reality device 100 recognizes that the object enters the second area, the virtual reality device 100 may recognize the object and may display information about the recognized object on a portion of the game screen, thereby notifying the user. When the user wears the virtual reality device 100, the user may recognize the object located in the second area.

[0033] When the object located in the second area enters the first area, the virtual reality device 100 may recognize that the object enters the first area, and may switch the game screen to a perspective full screen and display the switched screen. The virtual reality device 100 may end the game screen, and alternatively, may shoot the real world by using a camera of the virtual reality device 100, and display the real world in real time on the display of the virtual reality device 100. The user may recognize the external environment while the user wears the virtual reality device 100.

[0034] Hereinafter, a method of controlling the virtual reality device 100 so as to control the screen display of the virtual reality device 100 according to whether an external object enters a preset area set in a space where content is viewed and to identify the object entering the preset area while the user wears the virtual reality device 100, and the virtual reality device 100 that executes the method will now be described.

[0035] Figure 2 Detailed description of the method for controlling the virtual reality device 100 according to an embodiment of the present disclosure.

[0036] The virtual reality device 100 may set a first area and a second area different from the first area in the space where the content is viewed based on scanned space information about the space where the content is viewed and content information about the content (S210). The virtual reality device 100 may provide content in various fields including movies, education, sports, games, etc. The scanned space information is information obtained by scanning a space in which a user wearing the virtual reality device 100 views content for surrounding terrain or surrounding objects using at least one sensor. The area range of the first area may vary according to the type of content, and the second area may have a preset area range outside the first area. For ease of description, in an embodiment of the present disclosure, an example of a total of two areas including a first area and a second area will now be described, but the present disclosure is not limited thereto, and a total of N areas from the first area to the Nth area may be set. In the case where the virtual reality device 100 sets a total of N areas in the space where the content is viewed, the screen display of the virtual reality device 100 may be controlled differently whenever an object enters each area. In the following, reference is made to Figures 3 to 9, a process for setting the first area and the second area in a space for viewing content will now be described in detail.

[0037] Figure 3 is a detailed flowchart of an operation of setting a first area and a second area in a space for viewing content according to an embodiment of the present disclosure.

[0038] Reference Figure 3 , an example in which the virtual reality device 100 directly senses a space in which content is viewed by using at least one image sensor will now be described.

[0039] The virtual reality device 100 may sense a space in which the content is viewed based on a reproduction input for the content being received (S310). The virtual reality device 100 may sense the surrounding environment by using an image sensor (such as a camera or a laser radar) included in the virtual reality device 100.

[0040] For example, the virtual reality device 100 may scan the space where the content is viewed by using at least one image sensor. The virtual reality device 100 may obtain depth information from the scanned image. The virtual reality device 100 may obtain a three-dimensional scanning result about the space where the content is viewed by using a depth map including the depth information and a plurality of captured images. As another example, the virtual reality device 100 may obtain a three-dimensional scanning result by using a plurality of captured images of the space where the content is viewed.

[0041] The virtual reality device 100 may determine the first area and the second area based on the scanned space information sensed and the content information about the content for which the reproduction input is received (S320). The virtual reality device 100 may determine the area range of the first area and the area range of the second area based on the scanned space information obtained by scanning the space for viewing the content and the content information about the content selected by the user. The content information about the content may include the title of the content, the type or field of the content, the method of viewing or experiencing the content, the distance required to immerse in the content, the safety distance required to safely execute the content, etc. The content information about the content may be included in the running file of the application corresponding to each content in the form of metadata, or may be obtained from the content itself or an external device. For example, the virtual reality device 100 may analyze the content selected by the user, and may set the first area by determining whether the user wearing the virtual reality device 100 moves a lot or needs a large area when reproducing the content. The virtual reality device 100 may set the first area, and then may set the second area to a larger preset area.

[0042] In addition, the virtual reality device 100 may obtain information about permitted objects that may be located in the second area (S330). The user of the virtual reality device 100 may have registered object information about objects that are expected or permitted to enter the second area. The object information may be object identification information, image information such as an image or avatar of the object, affiliation information of the object, etc.

[0043] The virtual reality device 100 may transmit the information about the first area, the information about the second area, and the obtained information about the permitted object to the server 200 providing the content (S340). By storing the information in the server 200, the virtual reality device 100 may simply and quickly process the process of setting the first area and the second area when reproducing the same or similar content.

[0044] Figure 4 is a diagram illustrating an example of a process for setting a first area and a second area in a space for viewing content on a screen of a virtual reality device 100 according to an embodiment of the present disclosure.

[0045] Reference Figure 4 , a user wearing the virtual reality device 100 may run an application corresponding to content to be reproduced among a plurality of pieces of content in various fields and types in a main screen displayed on a display of the virtual reality device 100 (S400).

[0046] Before the virtual reality device 100 reproduces content selected by a user, the virtual reality device 100 may obtain scanning space information by sensing a space in which the content is viewed ( S410 ).

[0047] The virtual reality device 100 may set a first area and a second area in a space where content is viewed based on the obtained scanning space information and content information about content selected according to a user input (S420).

[0048] The user wearing the virtual reality device 100 may register information about the allowed object that is expected or allowed to enter the second area (S430). The user wearing the virtual reality device 100 may add or remove the allowed object, and may input information about the allowed object.

[0049] The virtual reality device 100 may transmit information about the setting area or information about the allowed object to the server 200 that provides the content selected according to the user input (S440). Thereafter, when the user of the virtual reality device 100 requests the server 200 to run the content again, the virtual reality device 100 may receive the information about the setting area or information about the allowed object from the server 200, and thus, the operations of S410 to S440 may be replaced.

[0050] The virtual reality device 100 may set a first area and a second area in a space for viewing content, and then may display a content reproduction screen according to a content reproduction start request from a user (S450). While reproducing the content, the virtual reality device 100 may sense the first area and the second area by detecting the surrounding environment using at least one sensor.

[0051] Figure 5 2 is a diagram for describing an area range of a first area according to a type of content in the virtual reality device 100 according to an embodiment of the present disclosure.

[0052] Reference Figure 5 , it is obvious that the area range of the first area may vary according to the type of content. For example, in the case of the first type of content including movies, racing games, etc., the virtual reality device 100 may consider a real-world object such as a chair on which a user may sit, and may set the first area by considering an area corresponding to the chair. As another example, in the case of the second type of content including a dance or rhythm game, the virtual reality device 100 may set the first area around the position of the user wearing the virtual reality device 100. In this regard, when there is an obstacle in the first area to be set, the virtual reality device 100 may change and set the first area, and may induce the user wearing the virtual reality device 100 to enter the set first area. As another example, in the case of the third type of content including a boxing or sandbox-type game, the user needs to move a lot, and therefore, in order to ensure the first area with a large range, the virtual reality device 100 may induce the user wearing the virtual reality device 100 to remove the obstacle.

[0053] Figure 6 is a detailed flowchart of an operation of setting a first area and a second area in a space for viewing content according to an embodiment of the present disclosure.

[0054] Figure 6 An example is shown in which the virtual reality device 100 receives information requesting setting of the first area and the second area from the external server 200 and sets the first area and the second area.

[0055] In response to receiving a reproduction input for the content with the execution history, the virtual reality device 100 may receive information about the first area and information about the second area from the server 200 providing the content with the execution history (S510). For example, when there is a user selection for content that is the same as or similar to the previously executed content, the virtual reality device 100 may request the server 200 providing the content for information about the first area and information about the second area. In response to the request from the virtual reality device 100, the server 200 obtains the information about the first area and the information about the second area stored in the server 200, and may transmit them to the virtual reality device 100.

[0056] The virtual reality device 100 may determine the first area and the second area based on the information about the first area and the information about the second area received from the server 200 (S520). In this case, the area range of the first area and the area range of the second area may be determined without scanning the space where the content is viewed or analyzing the content selected by the user.

[0057] The virtual reality device 100 may also receive information about a pre-registered allowed object from the server 200. The virtual reality device 100 may check whether an object located in the second area is an object pre-registered as a allowed object by using the information of the allowed object received from the server 200.

[0058] Figure 7 is a diagram illustrating an example of a process for setting a first area and a second area in a space for viewing content on a screen of a virtual reality device 100 according to an embodiment of the present disclosure.

[0059] Reference Figure 7 , a user wearing the virtual reality device 100 may run an application corresponding to content to be reproduced among a plurality of pieces of content in various fields and types in a main screen displayed on a display of the virtual reality device 100 (S600).

[0060] In response to receiving a reproduction input for the content with the execution history, the virtual reality device 100 may communicate with the server 200 providing the content with the execution history. The virtual reality device 100 may receive information about the first area and information about the second area from the server 200 providing the content with the execution history (S610).

[0061] The virtual reality device 100 may set the first area and the second area based on the information about the first area and the information about the second area received from the server 200 (S620).

[0062] The virtual reality device 100 may set a first area and a second area in a space for viewing content, and then may display a content reproduction screen according to a content reproduction start request from a user (S630). While reproducing the content, the virtual reality device 100 may sense the first area and the second area by detecting the surrounding environment using at least one sensor.

[0063] Figure 8 is a detailed flowchart of an operation of setting a first area and a second area in a space for viewing content according to an embodiment of the present disclosure.

[0064] In response to receiving a reproduction input for content, the virtual reality device 100 may determine a first area and a second area based on scan space information obtained by sensing a space in which content is viewed using at least one image sensor and content information about the content in which the reproduction input is received (S710).

[0065] The virtual reality device 100 may adjust the first area and the second area according to an input of a user of the virtual reality device 100 (S720). According to the user input, the virtual reality device 100 may manually adjust the automatically determined first area and the second area.

[0066] Fig. 9 is a diagram illustrating an example of a process for setting a first area and a second area in a space for viewing content on a screen of a virtual reality device 100 according to an embodiment of the present disclosure.

[0067] Reference Fig. 9 , a user wearing the virtual reality device 100 may run an application corresponding to content to be reproduced among a plurality of pieces of content in various fields and types in a main screen displayed on a display of the virtual reality device 100 (S800).

[0068] The virtual reality device 100 may determine a first region and a second region based on scan space information obtained by sensing a space in which content is viewed and content information about content to which a reproduction input is received ( S810 ).

[0069] The virtual reality device 100 may designate and display the automatically determined first and second regions, may provide a tool for editing the first and second regions, and may receive an input for adjusting the first and second regions from a user ( S820 ).

[0070] The virtual reality device 100 may set a first area and a second area in a space for viewing content, and then may display a content reproduction screen according to a content reproduction start request from a user (S830). While reproducing the content, the virtual reality device 100 may sense the first area and the second area by detecting the surrounding environment using at least one sensor.

[0071] Return to reference Figure 2 , the virtual reality device 100 may sense the first area and the second area by using at least one image sensor (S220). The virtual reality device 100 may sense the first area and the second area by using an image sensor disposed at an upper portion, a front surface, a rear surface, or a side portion of the virtual reality device 100. In addition, the virtual reality device 100 may obtain a sound or voice occurring in the surrounding environment by using a microphone.

[0072] The virtual reality device 100 may track an object located in the first area or the second area based on the sensing result (S230). Fig.10 This is described in detail.

[0073] Fig.10 is a detailed flowchart of an operation of tracking an object located in a first area or a second area according to an embodiment of the present disclosure.

[0074] The virtual reality device 100 may detect an object based on a sensing result using at least one image sensor (S910). In addition, the virtual reality device 100 may additionally detect an object based on a sound or voice obtained by using a microphone. For example, the virtual reality device 100 may recognize a direction in which the sound or voice has occurred, and may recognize the intensity of the sound or voice. The virtual reality device 100 may determine the position of the object and the distance from the virtual reality device 100 based on the direction and intensity of the sound or voice.

[0075] When the detected object is located in the second area, the virtual reality device 100 may determine whether communication with the device of the object located in the second area is available (S920).

[0076] As a result of the determination, when communication with the device of the object located in the second area is available, the virtual reality device 100 may receive the object position information and the user information from the detected device of the object (S930). However, since the detected object does not have the device, or communication may not be available even if it has the device, S920 and S930 may be skippable processes.

[0077] When communication with the device of the object located in the second area is unavailable, the virtual reality device 100 may track the detected object based on the result of sensing, and when communication is available, the virtual reality device 100 may track the detected object based on the result of sensing and object position information and user information received from the device of the detected object (S940).

[0078] Return to reference Figure 2, the virtual reality device 100 may control the screen display of the virtual reality device 100 based on the tracking result (S240). Figures 11 to 13 This is described in detail.

[0079] Fig.11 is a detailed flowchart of an operation of controlling a screen display of a virtual reality device 100 according to an embodiment of the present disclosure.

[0080] The virtual reality device 100 may determine whether the tracked object is located in the first area or the second area, and may display the screen of the virtual reality device 100 according to the screen display mode based on the determination result. Fig.11 , it is assumed that the user wears the virtual reality device 100 and watches the content after the user checks that an external object that may interfere with immersion in the content does not exist in the space where the user watches the content.

[0081] The virtual reality device 100 may determine whether the object is located in the second area (S1010). The virtual reality device 100 may determine whether the tracked object enters the second area. When there is no object that has entered the second area, the virtual reality device 100 may continuously detect and track the object from the results of sensing the first area and the second area.

[0082] When the object is located in the second area, the virtual reality device 100 may recognize the object (S1020). For example, the virtual reality device 100 may obtain an image via at least one image sensor, and may recognize the object based on the classification information and appearance information of the object based on the feature information extracted from the obtained image. For example, the virtual reality device 100 may input the feature vector to an image classification model including a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a bidirectional recurrent deep neural network (BRDNN), etc., an object positioning model, an object detection model, etc., thereby performing target classification, appearance information measurement, object detection, etc. The virtual reality device 100 may classify the object according to the image category, and may measure the appearance information by positioning the object with a bounding box, and thus, may recognize the object. In addition, the virtual reality device 100 may recognize the object based on the speech content of the object obtained via a microphone. For example, the virtual reality device 100 may obtain recognition information (such as a relationship with the user of the virtual reality device 100, the name of the object, etc.) from the speech content of the object.

[0083] The virtual reality device 100 may determine whether the recognized object is an object pre-registered as an allowed object (S1030). The user of the virtual reality device 100 may have registered object information about objects that are expected or allowed to enter the second area. The virtual reality device 100 may compare the information of the recognized object with the information of the allowed object, and thus, may determine whether the recognized object is a pre-registered object.

[0084] When the recognized object is a pre-registered object, the virtual reality device 100 may display an indicator corresponding to the object on the content reproduction screen (S1040). Since the recognized object is an allowed object, the virtual reality device 100 may generate an indicator corresponding to the recognized object based on recognition information of the object pre-registered as the allowed object, image information such as a photo or avatar of the object, affiliation information of the object, etc., and may display the indicator on a portion of the content reproduction screen. The indicator may be in the form of an icon or a thumbnail.

[0085] When the recognized object is not a pre-registered object, the virtual reality device 100 may display a see-through partial screen on the content reproduction screen (S1050). Since the virtual reality device 100 cannot recognize information about the recognized object, the virtual reality device 100 may display a see-through partial screen on a portion of the content reproduction screen for the user to directly recognize the object located in the second area.

[0086] The virtual reality device 100 may determine whether the object is located in the first area (S1060). The virtual reality device 100 may determine whether the tracked object enters the first area from the second area. When there is no object that has entered the first area, the virtual reality device 100 may continuously track the object located in the second area, and may continuously display information about the recognized object on a portion of the content reproduction screen.

[0087] When the tracked object has entered the first area from the second area and is thus located in the first area, the virtual reality device 100 may switch the content reproduction screen to a see-through full screen and display the object (S1070). When a situation occurs in which the user cannot be immersed in the content, the virtual reality device 100 may end the content reproduction screen and may display a see-through full screen for the user to check the external environment or the object when the user wears the virtual reality device 100.

[0088] Fig.12 is a diagram illustrating that the virtual reality device 100 controls screen display according to whether an object is located in a first area and a second area according to an embodiment of the present disclosure.

[0089] The virtual reality device 100 can provide the content selected by the user on the display of the virtual reality device 100 according to a content reproduction start request from the user wearing the virtual reality device 100. Therefore, the user wearing the virtual reality device 100 can immerse in the content and watch the content in a light shielding mode in which the user cannot see the external environment.

[0090] Reference Fig.12 , when there is no object located in the first area or the second area in the space where the content is viewed, the virtual reality device 100 may display only the content reproduction screen so that the user can be continuously immersed in the content.

[0091] When the object is recognized in the second area in the space where the content is viewed, the virtual reality device 100 may display the screen of the virtual reality device 100 in two types of screen display modes according to whether the object is a pre-registered object.

[0092] When the recognized object is a registered object, the virtual reality device 100 may display an indicator corresponding to the object on the content reproduction screen. The indicator may be generated based on recognition information of the object, image information such as a photo or avatar of the object, affiliation information of the object, and the like. The indicator may be displayed on a portion of the content reproduction screen, and the position of the indicator on the content reproduction screen may be determined based on the position, moving direction, and moving speed of the recognized object. For example, when the recognized object is located in the north direction in the second area and then moves in the west direction, the indicator is displayed as being located in the north direction and then moving in the west direction in the content reproduction screen. Fig.12 As shown, when two objects are located in the second area, indicators respectively corresponding to the objects may be displayed at corresponding positions of the content reproduction screen in consideration of the positions of the objects.

[0093] When the recognized object is not a pre-registered object, the virtual reality device 100 may display a partial screen through perspective on the content reproduction screen. The partial screen through perspective may be displayed on a portion of the content reproduction screen, and the position of the partial screen through perspective on the content reproduction screen may be determined based on the position, moving direction, and moving speed of the recognized object. For example, when the recognized object is located in the south direction in the second area and then moves in the east direction, the partial screen through perspective is displayed as being located in the south direction in the content reproduction screen and then moving in the east direction.

[0094] When an object is recognized in the first area in the space where the content is viewed, the virtual reality device 100 may switch and display the content reproduction screen as a see-through full screen. The virtual reality device 100 may end the content reproduction screen, and may display the see-through full screen for the user to check the external environment or the object when the user wears the virtual reality device 100. However, when the object located in the first area corresponds to an object registered as an exception object, for example, when the object is a pet such as a dog or a cat, the virtual reality device 100 does not switch to the see-through full screen by performing exception processing, and may maintain the content reproduction screen or may display an indicator corresponding to the exception object on the content reproduction screen.

[0095] Fig.13 is a diagram illustrating an example in which the virtual reality device 100 tracks an object by communicating with a device of the object located in a second area and displays an indicator corresponding to the object on a content reproduction screen according to an embodiment of the present disclosure.

[0096] When the object is located in the second area, the virtual reality device 100 may determine whether communication with the device of the object is available, and as a result of the determination, when communication with the device of the object located in the second area is available, the virtual reality device 100 may receive the position information and user information of the object from the device of the object. Based on the sensing result and the position information and user information of the object received from the detected device of the object, the virtual reality device 100 may display an indicator corresponding to the object on a portion of the content reproduction screen.

[0097] Reference Fig.13 , the first object and the second object have entered the second area, and the virtual reality device 100 may send a message about whether communication can be performed to the device 310 (e.g., a smart phone) of the first object and the device 320 (e.g., a virtual reality device) of the second object, and may receive a response message corresponding thereto. Fig.13 As shown, object device information may be additionally displayed at the lower right of the indicators corresponding to the first object and the second object, respectively, but the present disclosure is not limited thereto. The position of the indicator on the content reproduction screen may be determined based on the position, moving direction, and moving speed of the recognized object.

[0098] Fig.14 1 is a block diagram for describing the configuration of the virtual reality device 100 according to an embodiment of the present disclosure. The contents described above regarding the method of controlling the virtual reality device 100 may be applied to the virtual reality device 100 unchanged even if they are not described below.

[0099] Reference Fig.14, the virtual reality device 100 may include a memory 110, a processor 120, an image sensor 130, a display engine 140, a display 150, a gaze tracking sensor 160, a communication interface module 170, and an input unit 180. In addition, a position sensor for sensing the position of the virtual reality device 100 or a power supply for supplying power to the virtual reality device 100 may be included, and it is obvious to a person skilled in the art related to the embodiments of the present disclosure that other than the virtual reality device 100, the virtual reality device 100 may also be included. Fig.14 Other common elements besides those shown in FIG.

[0100] The memory 110 may store instructions that can be executed by the processor 120. The memory 110 may store a program composed of instructions. The memory 110 may include at least one type of hardware device including a random access memory (RAM), a static RAM (SRAM), a read-only memory (ROM), a flash memory, an electrically erasable programmable ROM (EEPROM), a programmable ROM (PROM), a magnetic memory, a magnetic disk, and an optical disk.

[0101] The memory 110 may store at least one software module including instructions. Each software module may be executed by the processor 120 to enable the virtual reality device 100 to perform a preset operation or function. For example, a space sensing module, an object tracking module, a screen display control module, etc. may be executed by the processor 120, but the present disclosure is not limited thereto and may further include other software modules.

[0102] The processor 120 may execute instructions stored in the memory 110 or a programming software module to control operations or functions performed by the virtual reality device 100. The processor 120 may be configured as a hardware element for performing arithmetic, logic, and input and output calculations, signal processing.

[0103] The processor 120 may be configured as at least one of a central processing unit (CPU), a microprocessor, a graphics processor unit, an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), an application processor, a neural processing unit, or an artificial intelligence dedicated processor designed to have a hardware structure dedicated to processing an artificial intelligence model, but the present disclosure is not limited thereto. Each processor constituting the processor 120 may be a dedicated processor for performing a preset function.

[0104] The image sensor 130 may be a device for capturing a real-world scene, and may be a stereo camera or a single camera. In addition, the image sensor 130 may be a depth camera for sensing depth information or a sensor in the form of a laser radar. The virtual reality device 100 may include a plurality of image sensors 130 corresponding to functions, respectively. For example, a first camera for three-dimensional (3D) scanning of a real-world object, a second camera for performing capture in a direction having a specific angle, a third camera for obtaining depth information, and the like may be included.

[0105] The virtual reality device 100 may control screen display via the display engine 140 and the display 150. A virtual image may be generated via the optical engine, and the virtual image may include a static image and a dynamic image. The virtual image may be an image shown to a user via the virtual reality device 100 in a virtual space, and may be an image of content, an image of a real-world object captured by capturing a real-world scene, or an image indicating information about the operation of the virtual reality device 100 or a control menu.

[0106] The display engine 140 may include an optical engine for generating and projecting a virtual image, and a guide unit for guiding light of the virtual image projected from the optical engine to the display 150 .

[0107] The display 150 may display an image of content, a real-world object image, or a virtual image such as an image indicating information about the operation of the virtual reality device 100 or a control menu. When the virtual image is displayed via the display 150, the user wearing the virtual reality device 100 may manipulate the virtual image via the image sensor 130, the gaze tracking sensor 160, or the input unit 180 in order to manipulate the virtual image.

[0108] The gaze tracking sensor 160 may detect gaze information including the gaze direction of the user's eyes, the position of the pupils of the user's eyes, the coordinates of the center point of each pupil, and the like. For example, the gaze tracking sensor 160 may detect the pupils from an image captured by receiving the received light by irradiating infrared light to the user's eyes, and thus may track their movement. The processor 120 may determine the eye movement form based on the user's gaze information detected by the gaze tracking sensor 160. For example, the processor 120 may determine various forms of eye movement, including fixation of gaze at one point, tracking of a moving object, saccade of rapidly moving the gaze from one gaze point to another gaze point, and the like. The processor 120 of the virtual reality device 100 may determine the user's gaze point or the user's gaze movement by using the gaze tracking sensor 160, and thus may use them to control the virtual reality device 100.

[0109] The communication interface module 170 may be connected to a device located outside the virtual reality device 100, and thus, information obtained or generated by the virtual reality device 100 may be sent to the device located outside, or various types of information may be received from the device located outside. For example, the device located outside may be the server 200 or a user terminal (not shown). The communication interface module 170 may communicate with other devices or networks. To this end, the communication interface module 170 may include a communication module configured to support at least one of various communication methods. For example, a communication module configured to perform short-range communication (such as wireless fidelity (Wi-Fi), Bluetooth, etc.), various types of mobile communication, or ultra-wideband communication.

[0110] The input unit 180 may receive an input of a user of the virtual reality device 100. For example, the input unit 180 may be a touch screen, a microphone for receiving a user's voice, or a motion sensor for sensing a user's motion, but the present disclosure is not limited thereto. According to an embodiment, the input unit 180 may be replaced in a manner of receiving a user's input by using the image sensor 130 and the display engine 140. For example, the processor 120 may obtain a line of sight coordinate based on line of sight information obtained using the line of sight tracking sensor 160. The processor 120 may recognize a gesture for selecting a virtual image corresponding to the line of sight coordinate based on an image captured via the image sensor 130.

[0111] According to the configuration, the processor 120 according to an embodiment of the present disclosure may execute one or more instructions stored in the memory 110 to control all operations for controlling the virtual reality device 100 .

[0112] The processor 120 according to an embodiment of the present disclosure or the first processor configuring the processor 120 may run a space sensing module to set a first area and a second area different from the first area in the space where the content is viewed based on scan space information about the space where the content is viewed and content information about the content. The processor 120 or the first processor may sense the first area and the second area by using at least one image sensor 130. The first area may be determined based on the scan space information and the content information, and the second area may be determined based on the scan space information.

[0113] According to an embodiment, the processor 120 or the first processor may sense a space where the content is viewed by using at least one image sensor 130 based on a reproduction input for the content received via the input unit 180. The processor 120 or the first processor may determine the first area and the second area based on the sensed scanning space information and the content information about the content for which the reproduction input is received. In addition, the processor 120 or the first processor may obtain information about an allowed object that may be located in the second area via the input unit 180. The processor 120 or the first processor may transmit the information about the first area, the information about the second area, and the obtained information about the allowed object to the server 200 providing the content via the communication interface module 170.

[0114] According to an embodiment, in response to receiving a reproduction input for the content with the operation history via the input unit 180, the processor 120 or the first processor may receive information about the first area and information about the second area from the server 200 providing the content with the operation history via the communication interface module 170. The processor 120 or the first processor may determine the first area and the second area based on the received information about the first area and the received information about the second area. In the case where the server 200 stores the content viewing history of the virtual reality device 100, when there is a request for the content with the viewing history from the virtual reality device 100, the server 200 may look up information about the first area and the second area that were previously set, and may send the information to the virtual reality device 100. In this case, the virtual reality device 100 may skip some processes of setting the first area and the second area.

[0115] According to an embodiment, the processor 120 or the first processor may determine the first area and the second area based on the scanning space information and the content information about the content, and may adjust the first area and the second area according to the input of the user of the virtual reality device 100 via the input unit 180. The processor 120 or the first processor may differently set the area range of the first area according to the type of content included in the content information.

[0116] According to an embodiment of the present disclosure, the processor 120 or a second processor configuring the processor 120 may run an object tracking module, and may track an object located in the first area or the second area based on the result of sensing the first area and the second area. The processor 120 or the second processor may sequentially detect objects in the first area or the second area, and thus, may recognize the current position, moving direction, and moving speed of the object.

[0117] According to an embodiment, the processor 120 or the second processor may detect an object, and may determine whether communication with the device of the object located in the second area is available. As a result of the determination, when communication with the device of the object located in the second area is not available, the processor 120 or the second processor may track the detected object based on the sensing result, and when communication is available, the processor 120 or the second processor may track the detected object based on the sensing result and the location information and user information of the object received from the device of the object via the communication interface module 170.

[0118] According to an embodiment, the processor 120 or a third processor configuring the processor 120 may execute a screen display control module, and may control a screen display of the virtual reality device 100 by using the display engine 140 and the display 150 based on a result of tracking an object located in the first area or the second area.

[0119] According to an embodiment, the processor 120 or the third processor may determine whether the object is located in the first area or the second area. The processor 120 or the third processor may display the screen of the virtual reality device 100 by controlling the display engine 140 and the display 150 according to the screen display mode based on the determination result.

[0120] According to an embodiment, when the object is located in the second area, the processor 120 or the third processor may identify the object, and therefore, may determine whether the identified object is an object pre-registered as an allowed object. When the identified object is a pre-registered object, the processor 120 or the third processor may display an indicator corresponding to the object on the content reproduction screen, and when the identified object is not a pre-registered object, the processor 120 or the third processor may control the display engine 140 and the display 150 to display a perspective partial screen on the content reproduction screen. The processor 120 or the third processor may determine the position of the indicator or the perspective partial screen on the content reproduction screen based on the position, moving direction, and moving speed of the identified object.

[0121] According to an embodiment, when the object is located in the first area, the processor 120 or the third processor may control the display engine 140 and the display 150 to switch the content reproduction screen to the see-through full screen. When the content reproduction screen is switched to the see-through full screen, the content being reproduced may be ended, and an image obtained by capturing the real world may be displayed in real time on the display 150 of the virtual reality device 100, so that the user wearing the virtual reality device 100 has a feeling of directly seeing the real world scene.

[0122] When the user wearing the virtual reality device 100 checks the object that has entered the first area via the see-through full screen and then performs an input or gesture for reproducing the content again, the see-through full screen may be switched to the content reproduction screen. Here, an indicator or a see-through partial screen may be displayed on the content reproduction screen according to the user's setting, or may not be displayed on the content reproduction screen so that the user does not care about the object that has been checked.

[0123] Fig.15 1 is a block diagram for describing a configuration of a server 200 for communicating with a virtual reality device 100 according to an embodiment of the present disclosure.

[0124] Reference Fig.15 The server 200 includes a memory 210, a processor 220, a communication interface module 230, and a memory 240. It is obvious to a person skilled in the art in the field related to the embodiment that the server 200 may include other than Fig.15 Other common elements besides those shown in FIG.

[0125] Depending on the implementation of the server 200, the Fig.15 For example, according to the embodiment, one element may be divided into two or more elements, two or more elements may be combined into one element, and some elements may be added or removed.

[0126] The server 200 is a device for providing virtual reality services, and can provide a virtual reality service platform. Alternatively, the server 200 is a device for providing content, and can provide content based on the virtual reality service.

[0127] The memory 210 may store instructions executable by the processor 220. The memory 210 may store software or programs.

[0128] The processor 220 may execute instructions stored in the memory 210. The processor 220 may perform overall control of the server 200. The processor 220 may obtain information and requests received via the communication interface module 230, and may store the received information in the memory 240. In addition, the processor 220 may process the received information. For example, the processor 220 may perform an operation of obtaining information to be used in the second external device according to the information received from the first external device or processing the received information. In addition, the processor 220 may send the information received from the first external device to the second external device.

[0129] The communication interface module 230 may communicate with an external device such as the virtual reality device 100. The server 200 may receive information related to the virtual reality service from the virtual reality device 100, and may transmit the information related to the virtual reality service in response to a request from the virtual reality device 100. For example, the server 200 may receive information about a first area, information about a second area, and information about permitted objects from the virtual reality device 100, and may store the received information for each virtual reality device 100.

[0130] The memory 240 may store various software and information items. For example, the memory 240 may store various data or information items for providing programs, applications, and virtual reality services running in the server 200.

[0131] The server 200 may be used as a mediating device through which the virtual reality devices 100 within a preset range exchange information. To this end, the server 200 may register information about the virtual reality device 100 for each user in the storage 240 in advance, and may update information related to the virtual reality service. The server 200 may transmit the virtual reality service reflecting the information received from the virtual reality device 100 to other virtual reality devices 100.

[0132] Embodiments of the present disclosure may be embodied as computer-readable recording media, for example, program modules to be run in a computer including computer-readable instructions. Computer-readable recording media may include any available media, volatile media and non-volatile media, and removable and non-removable media that can be accessed by a computer. In addition, computer-readable recording media may include computer storage media and communication media. Computer storage media include all volatile and non-volatile media and removable and non-removable media that are technically implemented to store information including computer-readable instructions, data structures, program modules, or other data. Communication media may include computer-readable instructions, data structures, program modules, or other data such as modulation type data signals.

[0133] In addition, the computer-readable recording medium may be provided in the form of a non-transitory recording medium. Here, the term "non-transitory storage medium" may indicate that the storage medium is a tangible device and does not include a signal (e.g., an electromagnetic wave), and may indicate that data may be permanently or temporarily stored in the storage medium. For example, the "non-transitory storage medium" may include a buffer that temporarily stores data.

[0134] According to an embodiment, the method according to various embodiments of the present disclosure may be included and provided in a computer program product. The computer program product can be traded between a seller and a buyer as a product. The computer program product can be distributed in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or can be distributed online (e.g., downloaded or uploaded) through an application store or distributed online directly (e.g., downloaded or uploaded) between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable application) may be at least temporarily stored or temporarily generated in a machine-readable storage medium, such as a memory of a manufacturer's server, an application store's server, or a relay server.

[0135] According to an embodiment of the present disclosure, a method of controlling a virtual reality device 100 is provided. The method of controlling the virtual reality device 100 may include: setting a first area and a second area different from the first area in the space where the content is viewed based on scan space information about the space where the content is viewed and content information about the content (S210). In addition, the method of controlling the virtual reality device 100 may include: sensing the first area and the second area by using at least one image sensor 130 (S220). In addition, the method of controlling the virtual reality device 100 may include: tracking an object located in the first area or the second area based on a result of the sensing (S230). In addition, the method of controlling the virtual reality device 100 may include: controlling a screen display of the virtual reality device 100 based on a result of the tracking (S240).

[0136] According to an embodiment of the present disclosure, controlling the screen display of the virtual reality device 100 (S240) may include: determining whether the object is located in the first area or the second area (S1010, S1060). In addition, controlling the screen display of the virtual reality device 100 (S240) may include: displaying the screen of the virtual reality device 100 according to a screen display mode based on a result of the determination (S1040, S1050, S1070).

[0137] In addition, the display (S1040, S1050, S1070) of the screen of the virtual reality device 100 may include: when the object is located in the second area (S1010), identifying the object, and determining whether the identified object is an object pre-registered as a permitted object (S1020, S1030). In addition, the display (S1040, S1050, S1070) of the screen of the virtual reality device 100 may include: based on the result of the determination, when the identified object is a pre-registered object, displaying an indicator corresponding to the object on the content reproduction screen, and when the identified object is not a pre-registered object, displaying a perspective partial screen on the content reproduction screen (S1040, S1050). In addition, the display (S1040, S1050, S1070) of the screen of the virtual reality device 100 may include: when the object is located in the first area (S1060), switching the content reproduction screen to a perspective full screen (S1070).

[0138] Furthermore, the position of the indicator or the perspective partial screen on the content reproduction screen may be determined based on the position, moving direction, and moving speed of the object.

[0139] According to an embodiment of the present disclosure, tracking of an object (S230) may include: detecting an object (S910). In addition, tracking of an object (S230) may include: determining whether communication with a device of an object located in a second area is available (S920). In addition, tracking of an object (S230) may include: based on a result of the determination, when communication with a device of an object located in the second area is not available, tracking the detected object according to a result of sensing, and when communication with a device of an object located in the second area is available, tracking the detected object according to a result of sensing and location information and user information of the object received from the device of the detected object (S930, S940).

[0140] According to an embodiment of the present disclosure, setting a first area and a second area different from the first area in a space for viewing content (S210) may include: sensing the space for viewing content based on a reproduction input for content being received (S310, S410). In addition, setting of the first area and the second area (S210) may include: determining the first area and the second area based on the sensed scanning space information and content information about the content for which the reproduction input is received (S320, S420).

[0141] In addition, the setting of the first area and the second area (S210) may include: obtaining information about the allowed objects that may be located in the second area (S330, S430). In addition, the setting of the first area and the second area (S210) may include: sending the information about the first area, the information about the second area, and the obtained information about the allowed objects to the server 200 providing content (S340, S440).

[0142] According to an embodiment of the present disclosure, setting a first area and a second area different from the first area in a space for viewing content (S210) may include: in response to receiving a reproduction input for content with an operation history, receiving information about the first area and information about the second area from a server 200 providing content with an operation history (S510, S610). In addition, setting the first area and the second area (S210) may include: determining the first area and the second area based on the received information about the first area and the received information about the second area (S520, S620).

[0143] According to an embodiment of the present disclosure, setting a first area and a second area different from the first area in a space for viewing content (S210) may include: determining the first area and the second area based on scanning space information and content information about the content (S710, S810). In addition, setting the first area and the second area (S210) may include: adjusting the first area and the second area according to an input of a user of the virtual reality device 100 (S720, S820).

[0144] According to an embodiment of the present disclosure, there may be provided a computer-readable recording medium having recorded thereon a program for executing the method.

[0145] According to an embodiment of the present disclosure, a virtual reality device 100 is provided. The virtual reality device 100 may include a memory 110 storing one or more instructions, a processor 120, at least one image sensor 130, a display engine 140, and a display 150. The processor 120 is configured to execute one or more instructions to set a first area and a second area different from the first area in the space where the content is viewed based on scan space information about the space where the content is viewed and content information about the content. In addition, the processor 120 is configured to execute one or more instructions to sense the first area and the second area by using at least one image sensor 130. In addition, the processor 120 is configured to execute one or more instructions to track an object located in the first area or the second area based on the result of sensing. In addition, the processor 120 is configured to execute one or more instructions to control the screen display of the virtual reality device 100 by using the display engine 140 and the display 150 based on the result of tracking.

[0146] According to an embodiment of the present disclosure, the processor 120 may be configured to execute one or more instructions to determine whether the object is located in the first area or the second area. In addition, the processor 120 may be configured to display the screen of the virtual reality device 100 by controlling the display engine 140 and the display 150 according to the screen display mode based on the determination result.

[0147] In addition, the processor 120 may be configured to run one or more instructions to identify the object when the object is located in the second area, and determine whether the identified object is an object pre-registered as a permitted object. The processor 120 may be configured to control the display engine 140 and the display 150 to display an indicator corresponding to the object on the content reproduction screen when the identified object is a pre-registered object based on the result of the determination, and to display a perspective partial screen on the content reproduction screen when the identified object is not a pre-registered object. In addition, the processor 120 may be configured to run one or more instructions to control the display engine 140 and the display 150 to switch the content reproduction screen to a perspective full screen when the object is located in the first area.

[0148] Furthermore, the processor 120 may be configured to execute one or more instructions to determine a position of an indicator or a see-through partial screen on the content reproduction screen based on the position, moving direction, and moving speed of the recognized object.

[0149] According to an embodiment of the present disclosure, the virtual reality device 100 may further include a communication interface module 170. In addition, the processor 120 may be configured to execute one or more instructions to detect an object and determine whether communication with the device of the object located in the second area is available. The processor 120 may be configured to track the detected object according to the result of sensing when communication with the device of the object located in the second area is not available, and to track the detected object according to the result of sensing and the position information and user information of the object received from the device of the detected object via the communication interface module 170 when communication is available, based on the result of the determination.

[0150] According to an embodiment of the present disclosure, the virtual reality device 100 may further include an input unit 180. In addition, the processor 120 may be configured to execute one or more instructions to sense a space where content is viewed by using at least one image sensor 130 based on a reproduction input for content received via the input unit 180. The processor 120 may be configured to determine the first area and the second area based on the sensed scanning space information and content information about the content for which the reproduction input is received.

[0151] In addition, the virtual reality device 100 may further include a communication interface module 170. In addition, the processor 120 may be configured to execute one or more instructions to obtain information about permitted objects that may be located in the second area via the input unit 180. The processor 120 may be configured to send the information about the first area, the information about the second area, and the obtained information about the permitted objects to the server 200 providing content via the communication interface module 170.

[0152] According to an embodiment of the present disclosure, the virtual reality device 100 may further include a communication interface module 170 and an input unit 180. In response to receiving a reproduction input for the content with the operation history via the input unit 180, the processor 120 may be configured to execute one or more instructions to receive information about the first area and information about the second area from the server 200 providing the content with the operation history via the communication interface module 170. The processor 120 may be configured to determine the first area and the second area based on the received information about the first area and the received information about the second area.

[0153] According to an embodiment of the present disclosure, the virtual reality device 100 may further include an input unit 180. The processor 120 may be configured to execute one or more instructions to determine the first area and the second area based on the scan space information and the content information about the content, and may adjust the first area and the second area according to the input of the user of the virtual reality device 100 via the input unit 180.

[0154] According to an embodiment of the present disclosure, the processor 120 may be configured to execute one or more instructions to differently set the area range of the first area according to the type of content included in the content information.

[0155] It is obvious to those skilled in the art that the description of the present disclosure is exemplary and that the present disclosure can be easily embodied in many different forms without changing the technical concept or essential features of the present disclosure. Therefore, it should be understood that the embodiments described herein should be considered only in a descriptive sense and not for limiting purposes. For example, singular configuration elements can be executed in a distributed manner, and distributed configuration elements can also be combined and executed.

[0156] The scope of the present disclosure is defined not by the detailed description but by the appended claims, and all differences and modifications that can be derived from the meaning and scope of the claims and other equivalent embodiments thereof are to be construed as being included in the present disclosure.

Claims

1. A method for controlling a virtual reality device, the method comprising: Based on scan space information about a space in which content is viewed and content information about the content, setting a first area and a second area different from the first area in the space (S210); sensing the first area and the second area by using at least one image sensor (S220); tracking an object located in the first area or the second area based on the sensing result (S230); as well as The screen display of the virtual reality device is controlled based on the tracking result (S240).

2. The method of claim 1, wherein: The control of the screen display (S240) includes: determining whether the object is located in the first area or the second area (S1010, S1060); and The screen of the virtual reality device is displayed according to a screen display mode based on a result of the determination (S1040, S1050, S1070).

3. The method according to claim 1 or claim 2, wherein: The display (S1040, S1050, S1070) of the screen of the virtual reality device includes: When the object is located in the second area (S1010), identifying the object, and determining whether the identified object is an object pre-registered as an allowed object (S1020, S1030); and Based on the result of the determination, when the identified object is the pre-registered object, an indicator corresponding to the object is displayed on the content reproduction screen, and when the identified object is not the pre-registered object, a perspective partial screen is displayed on the content reproduction screen (S1040, S1050), and When the object is located in the first area (S1060), The content reproduction screen is switched to a see-through full screen (S1070).

4. The method according to any one of claims 1 to 3, wherein: The position of the indicator or the see-through partial screen on the content reproduction screen can be determined based on the recognized position, moving direction, and moving speed of the object.

5. The method according to any one of claims 1 to 4, wherein: The tracking of the object (S230) includes: Detecting the object (S910); determining whether communication with the device of the object located in the second area is available (S920); and Based on the determined result, when the communication is not available, the detected object is tracked according to the sensing result, and when the communication is available, the detected object is tracked according to the sensing result and the location information and user information of the object received from the device of the detected object (S930, S940).

6. The method according to any one of claims 1 to 5, wherein: The setting (S210) includes: sensing a space in which the content is viewed based on a reproduction input for the content being received (S310, S410); and The first region and the second region are determined based on the scan space information according to the sensing and content information about the content to which the reproduction input is received (S320, S420).

7. The method according to any one of claims 1 to 6, wherein: The setting (S210) further includes: obtaining information about permitted objects that can be located in the second area (S330, S430); and The information about the first area, the information about the second area, and the obtained information about the permitted object are transmitted to a server that provides the content (S340, S440).

8. The method according to any one of claims 1 to 7, wherein: The setting (S210) further includes: receiving, in response to receiving a reproduction input for the content having the execution history, information about the first area and information about the second area from a server providing the content having the execution history (S510, S610); and The first area and the second area are determined based on the received information about the first area and the received information about the second area (S520, S620).

9. The method according to any one of claims 1 to 8, wherein: The setting (S210) further includes: determining the first area and the second area based on the scanning space information and content information about the content (S710, S810); and The first area and the second area are adjusted according to an input of a user of the virtual reality device (S720, S820). 10 . A computer-readable recording medium having recorded thereon a program for executing the method according to claim 1 .

11. A virtual reality device (100), comprising: A memory (110) storing one or more instructions; Processor (120); at least one image sensor (130); Display engine (140); and Display (150), The processor (120) is configured to execute the one or more instructions to perform the following operations: Based on scanned space information about a space in which content is viewed and content information about the content, a first area and a second area different from the first area are set in the space, the first area and the second area are sensed by using the at least one image sensor (130), an object located in the first area or the second area is tracked based on a result of the sensing, and a screen display of the virtual reality device (100) is controlled by using the display engine (140) and the display (150) based on a result of the tracking.

12. The virtual reality device (100) according to claim 11, wherein: The processor (120) is configured to execute the one or more instructions to perform the following operations: It is determined whether the object is located in the first area or the second area, and the screen of the virtual reality device (100) is displayed by controlling the display engine (140) and the display (150) according to a screen display mode based on a result of the determination.

13. The virtual reality device (100) according to claim 11 or claim 12, wherein: The processor (120) is configured to execute the one or more instructions to perform the following operations: When the object is located in the second area, identifying the object, determining whether the identified object is an object pre-registered as an allowed object, and based on the result of the determination, when the identified object is the pre-registered object, controlling the display engine (140) and the display (150) to display an indicator corresponding to the object on the content reproduction screen, and when the identified object is not the pre-registered object, displaying a partial screen through the content reproduction screen, and When the object is located in the first area, the display engine (140) and the display (150) are controlled to switch the content reproduction screen to a see-through full screen.

14. The virtual reality device (100) according to any one of claims 11 to 13, further comprising: a communication interface module (170), The processor (120) is configured to execute the one or more instructions to perform the following operations: Detect the object, determine whether communication with the device of the object located in the second area is available, and based on the result of the determination, when the communication is not available, track the detected object according to the result of the sensing, and when the communication is available, track the detected object according to the result of the sensing and the location information and user information of the object received from the device of the detected object via the communication interface module (170).

15. The virtual reality device (100) according to any one of claims 11 to 14, further comprising: Input unit (180), The processor (120) is configured to execute the one or more instructions to perform the following operations: Based on a reproduction input for the content received via the input unit (180), a space in which the content is viewed is sensed by using the at least one image sensor (130), and the first area and the second area are determined based on scanned space information according to the sensed space and content information about the content to which the reproduction input is received.

Images