Information processing device, information processing method, and program

Abstract

[Problem] The present disclosure provides a device, etc. for automatically recognizing an allowable region to remove the burden of specifying the allowable region each time. [Solution] An information processing device according to an aspect of the present disclosure comprises a classification unit, a region specification unit, and an allowable region determination unit. The classification unit classifies a plurality of planes included in space information, which represents an object existing in a three-dimensional space by using the plurality of planes, into at least a plane corresponding to a bottom surface and a plane corresponding to an obstacle. The region specification unit calculates a bottom surface region associated with the plane corresponding to the bottom surface and an obstacle region associated with the plane corresponding to the obstacle. The allowable region determination unit calculates an allowable region, in which a subject object existing in the three-dimensional space is allowed to be positioned, on the basis of the bottom surface region and the obstacle region.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In recent years, many devices have appeared that perform processing in response to user movements. For example, there are games in which characters displayed on the screen are moved by synchronizing them with the user's movements. In games where the user is constantly operating the device, as in these games, the user can become so engrossed in the operation that they lose awareness of their surroundings, which can lead to problems such as bumping into nearby objects. In particular, when enjoying virtual reality (VR) content played while wearing a head-mounted display (HMD), the user may not be able to see their surroundings at all, increasing the risk of bumping into real-world objects. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-190432 [Patent Document 2] Japanese Patent Application Laid-Open No. 2017-119032 [Patent Document 3] Japanese Patent Application Publication No. 2020-115353 [Non-patent literature]

[0004] Denis Tome, et al. “xR-EgoPose: Egocentric 3D Human Pose from an HMD Camera”, International Conference on Computer Vision (ICCV), 2019 Summary of the Invention [Problem to be solved by the invention]

[0005] To protect the user's physical safety, it is necessary to identify a safe area (region) that avoids contact with real-world objects and keep the user within that area. However, currently, identifying a safe area requires the user's effort. For example, the user places a controller on the floor and then uses the controller to draw the boundary of a seemingly safe area on the floor. This method requires the user to specify the boundary each time they attempt to play the game, and to avoid this effort, they may choose to specify the same boundary as the previous time. However, if furniture or other obstacles have moved or been added, this can lead to problems such as the user bumping into the obstacles because they specify the same boundary as the previous time.

[0006] The present disclosure provides a device or the like for automatically recognizing an area, eliminating the need for a user to specify an area. [Means for solving the problem]

[0007] An information processing device according to one aspect of the present disclosure includes a classification unit, a region identification unit, and an allowable region determination unit. The classification unit classifies a plurality of planes included in space information representing an object existing in a three-dimensional space using a plurality of planes into at least planes corresponding to bottom surfaces and planes corresponding to obstacles. The region identification unit calculates a bottom surface region corresponding to the planes corresponding to the bottom surfaces and an obstacle region corresponding to the planes corresponding to the obstacles. The allowable region determination unit calculates an allowable region in which a target object existing in the three-dimensional space is allowed to be located, based on the bottom surface region and the obstacle region.

[0008] This automatically determines the allowable area, eliminating the need for the user to specify the allowable area.

[0009] The classification unit may also be configured to calculate the angle that each normal to the plurality of planes makes with the direction of gravity, select a plane from the plurality of planes to be regarded as a horizontal plane based on the angle, and select the bottom plane from the plane regarded as a horizontal plane.

[0010] The classification unit may also be configured to calculate the height in the three-dimensional space of the plane deemed to be the horizontal plane, classify the plane deemed to be the horizontal plane into a plurality of groups based on the calculated height, and select the plane belonging to the group with the largest number of planes among the plurality of groups as the bottom plane.

[0011] The classification unit may also be configured to calculate the heights of the multiple planes in the three-dimensional space, and based on the calculated heights, classify the multiple planes other than the bottom surface into planes that correspond to the obstacles and planes that do not correspond to the obstacles.

[0012] The information processing device may further include an output unit that outputs an image showing the allowable region.

[0013] The information processing device may further include an output unit that outputs an image or sound indicating a warning when the distance between the target object and the boundary of the allowable area is equal to or less than a predetermined value.

[0014] The information processing device may also be configured to further include an output unit that outputs instructions regarding the movement of the target object and adjusts the instructions so that the movement of the target object remains within the allowable area.

[0015] The information processing device may further include a spatial information generating unit that generates the spatial information.

[0016] The spatial information generating unit may also be configured to generate the spatial information from distance measurement information indicating the distance to a peripheral object, the distance being obtained by a distance measuring device installed on the target object.

[0017] The spatial information generation unit may also be configured to generate the spatial information from ranging information measured by a ranging device, and remove information corresponding to the target object from the ranging information or the spatial information based on the position of the target object.

[0018] The information processing device may also be configured to further include a ranging unit that measures the distance to the subject and generates ranging information, a spatial information generation unit that generates the spatial information from the ranging information, a gravity direction acquisition unit that acquires the gravity direction, and an output unit that outputs information regarding the allowable area.

[0019] The information processing device may also be configured to further include an unknown area determination unit that determines an unknown area based on an occupation grid map, the occupation grid map representing the three-dimensional space using a plurality of three-dimensional unit grids, at least some of the plurality of unit grids having unknown information indicating whether or not they are occupied by an object, the unknown area determination unit determining the unknown area based on the position of the three-dimensional unit grid having the unknown information, and the allowed area determination unit preventing the unknown area from being included in the allowed area.

[0020] The unknown region determining unit may also be configured to include, in the unknown region, a location where a predetermined number or more of three-dimensional unit cells having the unknown information are stacked in the vertical direction.

[0021] The unknown area determination unit may also be configured to exclude from the unknown area a location where a predetermined number or more of three-dimensional unit cells having the unknown information are stacked in the vertical direction, but also a predetermined number or more of three-dimensional unit cells having unoccupied information indicating that the unit is not occupied by an object are stacked.

[0022] The information processing device may further include a hollow area identification unit that identifies hollow areas based on an occupancy grid map, the occupancy grid map representing the three-dimensional space using a plurality of three-dimensional unit cells, at least some of which have non-occupancy information indicating that they are not occupied by an object, the hollow area identification unit determining hollow areas based on the positions of the three-dimensional unit cells having the non-occupancy information, and the allowable area determination unit preventing the hollow areas from being included in the allowable area.

[0023] The hollow region specifying unit may also be configured to include, in the hollow region, the location of a three-dimensional unit cell that has the unoccupied information and is located lower than the plane corresponding to the bottom surface.

[0024] The information processing device may also be configured to further include a peripheral area determination unit that calculates, based on at least the position of the target object, an area that is part of the bottom area and includes the position of the target object as a peripheral area, and the allowable area determination unit calculates the allowable area further based on the peripheral area.

[0025] The surrounding area determination unit may also be configured to adjust the shape or size of the surrounding area based on the shooting direction of the camera and the direction of gravity.

[0026] The surrounding area determination unit may also be configured to recognize the general size of the target object based on information about the attributes of the target object, and adjust the shape or size of the surrounding area according to the general size of the target object.

[0027] The surrounding area determination unit may also be configured to adjust the shape or size of the surrounding area depending on the orientation of the target object.

[0028] In another aspect of the present disclosure, there is provided an information processing method comprising the steps of: classifying a plurality of planes included in spatial information representing an object existing in a three-dimensional space using a plurality of planes into at least planes corresponding to bottom surfaces and planes corresponding to obstacles; calculating a bottom surface area corresponding to the planes corresponding to the bottom surfaces and an obstacle area corresponding to the planes corresponding to the obstacles; and calculating an allowable area in which a target object existing in the three-dimensional space is allowed to be located based on the bottom surface area and the obstacle area.

[0029] In another aspect of the present disclosure, there is provided a program executed by a computer, comprising the steps of: classifying a plurality of planes included in spatial information representing an object existing in a three-dimensional space using a plurality of planes into at least planes corresponding to bottom surfaces and planes corresponding to obstacles; calculating a bottom surface area corresponding to the planes corresponding to the bottom surfaces and an obstacle area corresponding to the planes corresponding to the obstacles; and calculating an allowable area in which a target object existing in the three-dimensional space is allowed to be located based on the bottom surface area and the obstacle area. [Brief explanation of the drawings]

[0030] [Figure 1] FIG. 1 is a diagram showing an example of the arrangement of an area determination device according to a first embodiment. [Figure 2] FIG. 10 is a diagram showing an example of an allowable region. [Figure 3] FIG. 10 is a diagram showing an example of spatial information. [Figure 4] FIG. 10 is a diagram showing an example of a bottom surface region. [Figure 5] FIG. 10 is a diagram showing an example of an obstacle region. [Figure 6] FIG. 10 is a diagram showing an example of an allowable region. [Figure 7] 3 is a schematic flowchart of the overall processing of the area determination device according to the first embodiment. [Figure 8] 10 is a schematic flowchart of an area calculation process according to the first embodiment. [Figure 9] FIG. 10 is a diagram showing an example of the arrangement of a region determining device according to a second embodiment. [Figure 10] FIG. 10 is a diagram showing an example of an occupancy grid map. [Figure 11] FIG. 10 is a diagram illustrating calculation of an unknown area. [Figure 12] FIG. 10 is a diagram illustrating calculation of a hollow region. [Figure 13] 10 is a schematic flowchart of an area calculation process according to the second embodiment. [Figure 14] 10A and 10B are diagrams illustrating reduction of an allowable area based on a hollow area. [Figure 15] FIG. 10 is a diagram showing an example of the arrangement of a region determining device according to a third embodiment. [Figure 16] FIG. 10 is a diagram illustrating an expected application of the third embodiment. [Figure 17] FIG. 10 is a diagram showing an example in which the surrounding area is included in the allowable area. [Figure 18] 10 is a schematic flowchart of an area calculation process. DETAILED DESCRIPTION OF THE INVENTION

[0031] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0032] (First embodiment) Fig. 1 is a diagram showing an example of the configuration of a region determination device according to the first embodiment. The region determination device (information processing device) 1 of the example in Fig. 1 includes a position acquisition unit 11, a spatial information generation unit 12, a gravity direction acquisition unit 13, a region calculation unit 14, and an allowable region processing unit (output unit) 15. The region calculation unit 14 includes a classification unit 141, a region identification unit 142, and an allowable region determination unit 143.

[0033] It should be noted that the information processing system is not limited to the configuration shown in FIG. 1. Some of the components of the information processing device shown in FIG. 1 may be included in a device other than the region determination device 1, or may exist as an independent device. Furthermore, the components shown in FIG. 1 may be integrated within the region determination device 1, or may be further dispersed. Furthermore, components not shown or described may also be present in the region determination device 1. For example, one or more memories or storages that store information necessary for processing may be present in the region determination device 1.

[0034] The information processing system of this embodiment determines an allowable area for a target object. The target object is not particularly limited and may be a person, an animal, or a machine. The allowable area refers to an area in which the target object is allowed to be located. For example, the allowable area may be an area in which movement of the target object is allowed, or an area in which part of the target object can be moved.

[0035] FIG. 2 is a diagram showing an example of an allowable area. In the example of FIG. 2, a user 2 wearing a head-mounted display (HMD) is shown as the target object. An area 3 in which the user 2 can move or reach out without bumping into obstacles is shown as the allowable area. Note that the allowable area may be represented as a three-dimensional area, such as a combination of a dotted line 31 drawn on the floor and a wall 32 extending perpendicularly from the dotted line 31, as shown in FIG. 2. Alternatively, the allowable area may be represented as a two-dimensional area consisting of only the dotted line 31. In other words, the allowable area may be either a two-dimensional area or a three-dimensional area.

[0036] Conventionally, because it is difficult to automatically estimate an allowable area based on the actual surroundings without user input, the allowable area has been manually specified by the user. For example, the user has specified the allowable area by drawing a boundary line using a device such as a game controller. Alternatively, the allowable area has been set to a predetermined range, such as within a radius of several meters from the user. Therefore, the area determination device 1 of this embodiment automatically estimates an allowable area based on the actual surrounding environment, reducing the user's effort in specifying boundaries.

[0037] Other examples of the allowable area include a range in which a vehicle can move safely, and a range in which a fixed robot arm can move freely.

[0038] The processing of each component of the area determination device 1 will be described. In this embodiment, the position of the target object, spatial information about the periphery of the target object, and the direction of gravity are used as input information for determining the allowable area. Therefore, FIG. 1 shows a position acquisition unit 11 that acquires the position of the target object, a spatial information generation unit 12 that generates spatial information, and a gravity direction acquisition unit 13 that acquires the direction of gravity. In this embodiment, this information may be acquired by a conventional method, and the specific acquisition method is not particularly limited.

[0039] The position acquisition unit 11 acquires the position of a target object in three-dimensional space. For example, the position acquisition unit 11 may acquire an image of the target object and its surroundings, and estimate the assumed position of the target object in the three-dimensional space around the target object from the image. Alternatively, the position of the target object may be acquired from a satellite positioning system using an aeronautical satellite. In this case, the position of a reference satellite positioning system previously established in three-dimensional space is also acquired in advance so that the positional relationship between the target object and the three-dimensional space, such as the room in which the target object is located, can be determined.

[0040] The spatial information generation unit 12 generates spatial information of a three-dimensional space around the target object. The spatial information in this disclosure represents an object existing in the three-dimensional space using multiple planes. For example, a 3D mesh, an occupancy grid map, etc., correspond to the spatial information in this disclosure. FIG. 3 is a diagram showing an example of the spatial information. In FIG. 3, a 3D mesh 4 is shown as the spatial information. The 3D mesh 4 is composed of triangular unit planes (meshes) 41. Note that if a plane related to the target object is included in the spatial information, the plane is removed from the spatial information based on the position of the target object.

[0041] The method for generating the plane of spatial information is not particularly limited. For example, the spatial information generator 12 may acquire distance measurement information, such as a distance image indicating the distance to a peripheral object, from a distance measurement device such as a 3D sensor, and generate a 3D mesh from the distance measurement information using a method such as Kinect Fusion. The distance image can be acquired from a stereo camera, a ToF (Time of Flight) distance image sensor, or the like. Alternatively, the plane may be generated from point cloud data obtained by observing peripheral objects.

[0042] The generated spatial information may be used not only for estimating the allowable area but also for other processes, such as displaying the generated spatial information to a user wearing an HMD to present information about the surrounding environment.

[0043] The gravity direction acquisition unit 13 acquires the gravity direction. The gravity direction may be obtained using, for example, an IMU (Inertial Measurement Unit).

[0044] Note that devices for obtaining information used to generate input information for determining the allowable area, such as a camera, a ranging device, or an IMU, may be included in the same housing as the area determination device 1. For example, these devices and the area determination device 1 may be built into a wearable device such as the HMD shown in FIG. 2. In this case, the position of the target object coincides with the position at which the distance image from the 3D sensor was captured. Therefore, there is no need to prepare a camera to capture the target object and its surrounding environment in order to estimate the position of the target object. Even if such a camera is prepared, if the target object moves outside the camera's capture range, the allowable area cannot be updated. Therefore, it is preferable to attach a device for obtaining information used to generate input information to the target object, so that a new allowable area can be determined after the movement.

[0045] As described above, the position acquisition unit 11, the spatial information generation unit 12, and the gravity direction acquisition unit 13 may exist in a device external to the area determination device 1, and the area determination device 1 may acquire this information from the external device. In this case, when the area determination device 1 acquires an occupancy grid map relating to the periphery of the target object, the grid of the occupancy grid map is created parallel and perpendicular to the gravity direction, so it is not necessary to acquire the gravity direction.

[0046] The classification unit 141 classifies the multiple planes included in the space information into at least planes corresponding to the bottom surface and planes corresponding to obstacles.

[0047] Unlike a wall, the bottom surface is an obstacle, that is, an object that does not hinder the movement of a target object. For example, a floor, a stage, a road, etc., correspond to the bottom surface. In this embodiment, the bottom surface is assumed to be a plane perpendicular to the direction of gravity, that is, a horizontal plane. However, it does not have to be a strictly horizontal plane. If the angle between the normal to the plane and the direction of gravity is within a predetermined threshold, the plane can be considered to be a horizontal plane. Note that the threshold can be set appropriately depending on the specifications, such as the accuracy required of the area determination device 1.

[0048] Therefore, the classification unit 141 calculates the angle between the normal to each plane included in the spatial information, such as the unit plane 41 shown in Fig. 3, and the direction of gravity. Then, the classification unit 141 extracts, as horizontal planes, planes for which the calculated angle is smaller than a threshold value.

[0049] Among the extracted horizontal planes, some correspond to bottom surfaces and others to obstacles. Therefore, in order to classify these, in this embodiment, it is assumed that the number of horizontal planes corresponding to bottom surfaces is greater than the number of horizontal planes corresponding to obstacles, and that the bottom surfaces and the horizontal planes corresponding to obstacles have different heights. By assuming this, the heights of the extracted horizontal planes are calculated, and the horizontal plane with the largest number of horizontal planes having the same height can be considered to correspond to the bottom surface. Note that if the difference in height between the planes is equal to or less than a predetermined value, they can be considered to have the same height. Therefore, in other words, the extracted horizontal planes can be sorted into one of a plurality of groups with predetermined height ranges according to their height, and the plane belonging to the group with the largest number of sorted planes can be considered to correspond to the bottom surface. In this way, the planes corresponding to the bottom surface can be extracted.

[0050] Calculating the height of a plane boils down to the problem of finding the parameter d when the plane is expressed as ax+by+cz=d using the vector (a, b, c) that indicates the direction of gravity. The parameter d can be found using methods such as RANSAC (Random sample consensus). Furthermore, even if the direction of gravity is unknown, it is possible to estimate the plane from the 3D mesh itself by using methods such as Efficient RANSAC.

[0051] The height of the bottom surface may also be specified. In such a case, the horizontal plane at the specified height among the extracted horizontal planes may be considered to correspond to the bottom surface. For example, the height of the bottom surface may be specified using a marker placed on the bottom surface. However, it is preferable for the area determination device 1 to calculate the height of the bottom surface, as this eliminates the need to place a marker.

[0052] Furthermore, the classification unit 141 detects planes that correspond to obstacles from spatial information such as a 3D mesh. It is also possible to simply determine that planes that do not correspond to the bottom surface correspond to obstacles. Alternatively, conditions for determining an object as an obstacle may be set in advance, and planes that do not satisfy the conditions may not correspond to obstacles.

[0053] For example, suppose the size of a target object is 1 m and the height of a certain plane is 5 m from the bottom. In this case, since the target object does not hit the plane, it may be determined that the plane is not an obstacle to the target object. By adding such a determination, it is possible to prevent areas around objects that do not affect the movement of the target object from being excluded from the allowable area.

[0054] Furthermore, for example, if the target object is a vehicle, an object made up of multiple planes with a volume equal to or less than a predetermined value may be deemed not to affect the running of the vehicle, and the planes that make up the object may be determined not to be obstacles to the target object. In this way, a condition for determining whether a plane is an obstacle may be set.

[0055] The region specifying unit 142 specifies a bottom region corresponding to a plane corresponding to the bottom surface and an obstacle region corresponding to a plane corresponding to the obstacle. For example, a two-dimensional region may be obtained by projecting a three-dimensional shape consisting of the plane corresponding to the obstacle.

[0056] Fig. 4 is a diagram showing an example of a bottom surface area. Fig. 5 is a diagram showing an example of an obstacle area. In the examples of Fig. 4 and Fig. 5, the bottom surface area and the obstacle area are shown as bird's-eye views. In the example of Fig. 4, the black part indicates the bottom surface area, and in the example of Fig. 5, the black part indicates the obstacle area.

[0057] Note that the bottom surface area and the obstacle area may overlap. For example, for an object whose bottom surface is not obscured, such as a table, the bottom surface below the object is shown as the bottom surface area, and the plane of the object is also shown as the obstacle area. Therefore, overlapping areas may occur in overhead views such as those shown in Figures 4 and 5.

[0058] The allowable area determination unit 143 determines the allowable area based on the bottom area and the obstacle area. Fig. 6 is a diagram showing an example of the allowable area. In the example of Fig. 6, the black part indicates the allowable area.

[0059] The allowable area determination unit 143 may remove the overlapping portion between the bottom surface area and the obstacle area from the bottom surface area and determine the remaining area as the allowable area. Furthermore, to improve safety, the remaining area may be reduced. Furthermore, if the allowable area is not continuous but is divided into multiple areas, such as an outlying area, an area that does not include the position of the target object may be removed from the multiple areas. Furthermore, a portion of the bottom surface area may be removed. For example, the bottom surface area may include a portion narrower than the size of the target object. Since the target object cannot pass through such a portion, the portion may be removed. Furthermore, by removing the portion, the separate area that does not include the position of the target object may be removed. In this way, the allowable area determination unit 143 may determine the allowable area by removing the overlapping portion between the bottom surface area and the obstacle area from the bottom surface area and adjusting the remaining area. Furthermore, a deep neural network (DNN) that outputs an allowable area when a plane corresponding to the bottom surface and a plane corresponding to an obstacle are input may be generated in advance through learning, and the allowable area determination unit 143 may calculate the allowable area using the DNN.

[0060] The allowable area determination unit 143 may calculate a boundary line of the allowable area, such as the dotted line 31 in FIG. 2. For calculating the boundary line, a method such as Marching Squares may be used. Note that multiple boundaries may be calculated. In such cases, it is sufficient to select one boundary line by taking into consideration the length of the boundary line, the distance from the target object, and the like. The allowable area determination unit 143 may also calculate a virtual wall, such as wall 32 in FIG. 2, by extending the boundary line in the direction of gravity.

[0061] The allowable area processing unit 15 (output unit) performs processing using the determined allowable area. For example, it may simply output information about the allowable area. The information about the allowable area may be information that allows the target object to recognize the allowable area. For example, an image showing the allowable area and the boundary line of the allowable area, as shown in Figures 2 and 6, may be generated and output to a specified display. If the target object is a person, making the allowable area visible can reduce the risk of the target object crossing the allowable area.

[0062] It is also preferable to notify the user of the danger when the target object approaches the boundary of the allowable area. Therefore, the allowable area processing unit 15 may output a warning image or sound, that is, an alert, when the distance between the target object and the boundary of the allowable area is equal to or less than a predetermined value.

[0063] The allowable area processing unit 15 may also output instructions regarding the movement of the target object. For example, when the target object approaches the boundary of the allowable area, the instruction may be to move away from the boundary of the allowable area. Alternatively, the instruction content may be changed depending on the position of the target object. For example, if an instruction to move 1 m to the left is normally issued, if moving 1 m to the left would bring the object too close to the allowable area, the instruction may be adjusted to move 0.5 m to the left. In this way, the instruction may be given to keep the object's movement within the allowable area.

[0064] In addition, there may be cases where a user who has confirmed the estimated allowable range wishes to modify the allowable range. Therefore, the information processing device may accept modifications to the estimated allowable range and update the allowable range. This saves the user time and effort compared to when the user manually specifies the allowable range from the beginning.

[0065] Next, the flow of processing will be described. Fig. 7 is a schematic flowchart of the overall processing of the area determination device 1 according to this embodiment. This flow can be executed multiple times, triggered by the passage of time, the movement of the target object and surrounding objects, etc. This flow also shows a case where the position, weight direction, and spatial information of the target object are used.

[0066] The position estimation unit acquires the position of the target object (S101), and the gravity direction acquisition unit acquires the weight direction (S102). The spatial information generation unit 12 generates spatial information based on the position of the target object and the ranging information (S103). The area calculation unit 14 executes area calculation processing based on the position, weight direction, and spatial information of the target object (S104). The flow of the area calculation processing will be described later. Since the allowable area is determined by the area calculation processing, the allowable area processing unit 15 outputs information based on the allowable area (S105). As described above, information for causing the target object to recognize the allowable area may be output, or information regarding instructions to prevent the target object from exceeding the allowable area without causing the target object to recognize the allowable area may be output.

[0067] 8 is a schematic flowchart of the area calculation process according to the first embodiment. This flow corresponds to the process of S104 in the overall process described above.

[0068] The classification unit 141 calculates the angle between the normal of each plane in the spatial information and the direction of gravity, and extracts planes that are considered to be horizontal from each plane based on the angle (S201). The classification unit 141 also calculates the height of each plane (S202). Note that the height of planes that are considered to be horizontal is always calculated, but the height of planes that are not considered to be horizontal may be calculated as needed.

[0069] The classification unit 141 extracts planes that correspond to bottom surfaces from among the planes that are deemed to be horizontal, based on the heights of the planes that are deemed to be horizontal (S203). If the height of the bottom surface is specified, the planes at the specified height can be determined to correspond to bottom surfaces. If the height of the bottom surface is not specified, it is assumed that the height of the bottom surface is constant and that the most planes correspond to bottom surfaces among the planes that are deemed to be horizontal, and the planes that are deemed to be horizontal can be grouped according to their height, and the plane that belongs to the group with the most planes can be determined to correspond to bottom surfaces.

[0070] Furthermore, the classification unit 141 extracts planes that correspond to obstacles from planes other than the bottom plane based on conditions such as height (S204). Planes that do not correspond to the bottom plane may simply be determined to correspond to obstacles, or planes that do not satisfy conditions such as area or height may not be determined to correspond to obstacles.

[0071] The area specifying unit 142 calculates a bottom area from a plane corresponding to the bottom surface, and calculates an obstacle area from a plane corresponding to the obstacle (S205). Then, the allowable area determining unit 143 calculates an allowable area based on the bottom area and the obstacle area (S206), and this flow ends.

[0072] Note that the flowcharts of the present disclosure are merely examples, and each process does not necessarily have to be performed in the order described above. As long as the data necessary for the process can be obtained, the order may be changed, or the processes may be performed in parallel. For example, obtaining the gravity direction (S102) may be performed before the area calculation process, and the order of obtaining the object position (S101) and the like may be reversed.

[0073] As described above, according to this embodiment, the bottom surface can be automatically determined and the allowable area can be determined, thereby saving the user the trouble of specifying the area and the bottom surface.

[0074] Furthermore, the target object can be prevented from going beyond the allowable area by outputting an image showing the allowable area, an alert indicating that the target object is close to the allowable area, etc. It is also possible to give instructions to the target object so as not to go beyond the allowable area.

[0075] Furthermore, even if there is an obstacle, if the obstacle does not obstruct the movement of the target object because of its different height, it can be not regarded as an obstacle. This improves the accuracy of the allowable area. For example, in a system that determines the allowable area based on an aerial camera image, the area under an overpass or a light is designated as an unallowable area, but this embodiment can prevent such a situation. (Second embodiment) 9 is a diagram showing an example of the configuration of a region determining device 1 according to the second embodiment. In the example of FIG. 9, the region calculating unit 14 further includes an unknown region determining unit 144 and a hollow region identifying unit 145.

[0076] For convenience of explanation, the example in FIG. 9 includes both the unknown region determining section 144 and the hollow region identifying section 145, but it is also possible to include only one of them.

[0077] In the first embodiment, objects present in a space are represented as planes included in the spatial information, and the allowable area is calculated by dividing the planes into a bottom surface and other obstacles. However, due to camera noise or the like, it is possible that objects present in a space are not represented as planes included in the spatial information. Therefore, in the second embodiment, it is assumed that an occupancy grid map is used, and information included in the occupancy grid map is further utilized.

[0078] FIG. 10 shows an example of an occupancy grid map. An occupancy grid map is composed of cubic unit cells called voxels. Each voxel has a value related to the probability of an object being present in its region. For example, an object may not be accurately detected due to noise caused by camera vibration. Therefore, the probability of an object being present is expressed numerically. Furthermore, in the occupancy grid map, each voxel is classified based on the value and a predetermined threshold as "Occupied," meaning that it is occupied by an object; "Free," meaning that it is not occupied by an object; or "Unknown," meaning that it is unknown whether it is occupied by an object. For example, voxels with a probability of an object being present higher than a first threshold are classified as "Occupied," voxels with a probability of an object being present lower than a second threshold are classified as "Free," and voxels with a probability of an object being present lower than the first threshold but higher than the second threshold are classified as "Unknown." Therefore, it can be said that each voxel in the occupancy grid map has information of one of Occupied, Free, and Unknown. Note that in the example of Fig. 10, voxels with Free information are omitted.

[0079] As described above, spatial information represents objects existing in a three-dimensional space using multiple planes. Therefore, when spatial information is generated using an occupancy grid map, the spatial information is generated using occupied voxels. That is, in the first embodiment, occupied voxels are used, but free voxels and unknown voxels are not used. On the other hand, in the second embodiment, free voxels and unknown voxels are also used to calculate the allowable area.

[0080] In this description, it is assumed that the spatial information is also generated using an occupancy grid map, but the spatial information may be generated using a 3D mesh, and the occupancy grid map may be used to determine the unknown area and hollow area, which will be described later. Also, the 3D mesh may be used only for detecting the bottom surface.

[0081] For example, a two-dimensional region corresponding to a location where many Unknown voxels exist can be considered an unknown region, and the unknown region can be excluded from the allowable region. FIG. 11 is a diagram illustrating the calculation of an unknown region. The upper part of FIG. 11 shows a plane parallel to the width and height of the voxels. The lower part of FIG. 11 shows an unknown region parallel to the length and width of the voxels, generated based on the upper plane. Most of the voxels in the first, second, and tenth columns from the left of the upper plane have Unknown information. Therefore, in the two-dimensional region at the bottom, the portions corresponding to the first, second, and tenth columns from the left are considered to be unknown regions. By excluding the unknown region from the allowable region, it is possible to prevent obstacles that were not detected due to noise, for example, from being included in the allowable region.

[0082] As described above, the unknown region determining unit 144 may determine an unknown region based on information held by each voxel. The conditions may be set as appropriate. For example, as shown in the upper part of FIG. 11 , a location where a predetermined number or more of Unknown voxels are stacked vertically may be included in the unknown region. Furthermore, even if a predetermined number or more of Unknown voxels are stacked vertically, the location may not be determined to be an unknown region if a predetermined number or more of Free voxels are also stacked in the location.

[0083] In addition, there may be cases where there is a step on the bottom surface, such as under a floor or on stairs. That is, there may be a flat surface below the bottom surface where the target object is located. As described in the first embodiment, if a flat surface other than the bottom surface is considered an obstacle, it is assumed that the flat surface below the bottom surface is also considered an obstacle and is not included in the allowable area. However, if the flat surface below the bottom surface is not captured by the camera due to a large step, the area related to the step may not be determined to be an obstacle area. Therefore, it may be determined that the area where free voxels exist below the bottom surface is a hollow area and excluded from the allowable area.

[0084] FIG. 12 is a diagram illustrating the calculation of hollow regions. The upper part of FIG. 12 shows planes parallel to the width and height of voxels. The lower part of FIG. 12 shows hollow regions parallel to the width and height of voxels, generated based on the upper plane. The occupied voxels in the third row of the upper plane are the bottom plane. The occupied voxels whose height matches the bottom plane can be considered the bottom plane. The bottom plane height may be specified or, as described above, may be estimated by the classification unit. Furthermore, in the first to third columns from the left of the upper plane, there are free voxels below the bottom plane. Therefore, in the two-dimensional region below, the portion corresponding to the first to third columns from the left is considered to be a hollow region. By excluding the hollow region from the allowable region, it is possible to reduce the risk of, for example, a target object falling into a hole.

[0085] As described above, the hollow region identifying section 145 may identify a hollow region based on information held by voxels located below the bottom surface. Note that the conditions may be adjusted as appropriate.

[0086] The allowable area determination unit 143 may first calculate the allowable area based on the bottom area and obstacle area, and then expand or reduce the allowable area based on the unknown area and hollow area, or it may calculate all areas that affect the generation of the allowable area, such as the bottom area, obstacle area, unknown area, and hollow area, and then calculate the allowable area using logical operations such as union, difference, and intersection of these areas.

[0087] FIG. 13 is a schematic flowchart of the region calculation process according to the second embodiment. The processes from S201 to S205 are the same as those in the flowchart of the first embodiment. However, the process adds an unknown region calculation process (S301) by the unknown region determination unit 144 and a hollow region calculation process (S302) by the hollow region identification unit 145. The unknown region calculation process (S301) can be performed in parallel with the processes from S201 to S205. The hollow region calculation process (S302) can be performed in parallel with the processes from S201 to S205 if a bottom surface is specified in advance. However, if not specified, as in the example of FIG. 13, the bottom surface is identified by the classification unit and then the hollow region calculation process (S302) is performed. The allowable region determination unit 143 then calculates the allowable region based on the regions calculated in each process (S303).

[0088] The allowable area determination unit 143 may reduce the allowable area based on at least one of the unknown area and the hollow area. For example, the allowable area may be reduced to a point a certain distance away from the unknown area and the hollow area, assuming that the unknown area and the hollow area are dangerous.

[0089] FIG. 14 is a diagram illustrating the reduction of the allowable area based on a hollow area. The room shown in the image of FIG. 14(A) is built like a loft, and the floor on the front side does not continue all the way to the back. Therefore, if the floor on the front side is considered to be the bottom, the area surrounded by the frame line 51 has no bottom and is hollow. When a target object moves in such a space, there is a risk that the target object may fall, so it is preferable to reduce the allowable area.

[0090] 14(B), the boundary of the allowable area when not reduced is shown by a white frame line. The allowable area is smaller than the bottom area shown in black, but is set close to hollow area 52 corresponding to frame line 51. Therefore, if the target object accidentally goes outside the allowable area, there is a risk of it falling.

[0091] Therefore, the allowable area is reduced based on the distance from the boundary between an area where no objects are assumed to exist, such as a hollow area, and an area where objects are assumed to exist, such as the bottom. Figure 14(C) shows the allowable area when reduced based on the distance from the boundary, outlined in white. Compared to the allowable area in Figure 14(B), it can be seen that the boundary of the allowable area is not close to the border between the black and white areas. Therefore, the risk of falling is reduced compared to Figure 14(B).

[0092] The distance to be reduced may be determined appropriately depending on the application of the region determining device 1. The method of reduction may also be determined appropriately. For example, the allowable region may be reduced by dividing the two-dimensional region into unit lattices, calculating the shortest distance between the center of each unit lattice and the boundary of the region, and excluding unit lattices whose shortest distance is equal to or less than a threshold value.

[0093] As described above, in the second embodiment, information on the occupancy grid map is used to identify areas that should not be included in the allowable area, such as unknown areas and hollow areas, and the allowable area is determined so that these areas are not included in the allowable area. This makes it possible to obtain the effect of improving safety compared to the first embodiment.

[0094] (Third embodiment) Fig. 15 is a diagram showing an example of the configuration of a region determining device 1 according to the third embodiment. In the example of Fig. 15, the region calculation unit 14 further includes a surrounding region determination unit 146. Note that, in the example of Fig. 15, the surrounding region determination unit 146 is added to the first embodiment, but it may also be added to the second embodiment. In other words, the surrounding region determination unit 146 may be included in the region determining device 1 together with at least one of the unknown region determination unit 144 and the hollow region identification unit 145.

[0095] In the third embodiment, the peripheral area of ​​the target object is included in the allowable area. FIG. 16 is a diagram illustrating an expected application of the third embodiment. As shown on the left side of FIG. 16, an HMD with a camera is attached to the target object. In this case, the camera does not capture the target object, so an area very close to the target object may not be recognized as the allowable area. Even if a camera is not attached to the target object, the target object may be perceived as an obstacle, and the area around the target object may not be recognized as the allowable area. Therefore, the peripheral area determination unit 146 may calculate an area within a predetermined distance from the target object as the peripheral area, and the allowable area determination unit 143 may further calculate the allowable area based on the calculated peripheral area.

[0096] The surrounding area determination unit 146 extracts a portion of the bottom area calculated by the classification unit 141 based on the position of the target object and determines it as the surrounding area. The position of the target object may be acquired in the same manner as in the first embodiment. That is, the position acquisition unit 11 may directly acquire the position of the target object from an external source, or may acquire information regarding the position of the target object from an external source and calculate the position of the target object from the information. For example, if a camera is attached to the target object, the position acquisition unit may acquire the position of the camera and calculate the position of the target object from the camera position and a predetermined calculation method. For example, if the camera is attached so as to capture the traveling direction of the target object, it may be considered that the target object exists at a position about a predetermined distance from the camera in the direction opposite to the camera's viewing direction, as shown on the right side of FIG. 16. Such a calculation method may be determined in advance.

[0097] The size of the surrounding region may be determined as appropriate. For example, the region identification unit 142 may determine the surrounding region as a circle of a predetermined radius centered on the position of the target object, or a shape corresponding to the size of the target object. In other words, the shape of the surrounding region of the target object may be determined as appropriate depending on the purpose of the region determination device 1, etc.

[0098] For example, if the target object is a human, the shape of the surrounding area may be elliptical or the like depending on the camera angle. This is because the human body is generally wider from side to side than from front to back, and the more the human body bends, the wider it becomes from front to back. For example, if the surrounding area is an ellipse, the lengths of the minor axis and the major axis are prepared as parameters. Then, the angle between the vector in the front direction of the camera and the vector in the direction of gravity is calculated. Then, the length of the minor axis can be changed linearly with respect to the angle, and the lengths of each axis obtained can be used to calculate the surrounding area.

[0099] Furthermore, for example, if the target object is a living organism, the size of the surrounding area may be determined from a record of the organism's standard body shape. For example, when the area determination device 1 acquires data about the target object, the surrounding area determination unit 146 predicts the target object's physical characteristics from the data. For example, when data such as the target object's age, sex, and nationality is received, data on the target object's standard physique that matches that data may be acquired from a predetermined database, and the size of the surrounding area may be determined based on the standard physique.

[0100] Alternatively, the size of the surrounding area may be determined from the pose of the target object. For example, it is known that the pose of a human captured by a camera can be estimated using a known pose estimation technique. Therefore, the surrounding area can be calculated by projecting the estimated pose onto a two-dimensional plane and calculating a fitting circle or ellipse.

[0101] The surrounding area may be calculated based on images from a camera other than the camera capturing the obstacle, measurements by a sensor, etc. For example, a sensor may be attached to the target object to detect and track its movement. In such cases, the results of the sensor may be used to determine the size of the surrounding area.

[0102] Furthermore, the surrounding area may be calculated in real time or at regular intervals. When the surrounding area is calculated multiple times, the surrounding area determination unit may include the previously calculated surrounding area in the surrounding area calculated this time. That is, a new surrounding area may be generated based on the previous surrounding area, and the trajectory of the movement of the target object may be included in the surrounding area.

[0103] Figure 17 is a diagram showing an example in which the surrounding area is included in the allowable area 3. On the left side of Figure 17, the floor where user 2, the target object, is located is not included in allowable area 3, but on the right side of Figure 17, the floor where user 2 is located is included in allowable area 3. By doing so, when performing secondary processing using the calculated allowable area, problems resulting from the target object not being within the allowable area can be prevented.

[0104] 18 is a schematic flowchart of the area calculation process in the third embodiment. The processes from S201 to S205 are the same as those in the flowchart in the first embodiment. After the process of S205, the surrounding area specification unit executes the calculation process of the surrounding area (S401). Then, the allowable area determination unit 143 calculates the allowable area based on at least the surrounding area in addition to the bottom area and the obstacle area (S402). Naturally, the allowable area may also be calculated based on the unknown area and hollow area.

[0105] Note that there may be cases where an obstacle exists around the target object, or where the target object is erroneously detected as having an obstacle around it. Therefore, the area determination device 1 may request an instruction as to whether or not to include the area around the target object in the allowable area, and may determine whether or not to include the area around the target object in the allowable area based on the instruction returned in response to the request.

[0106] As described above, in the third embodiment, the peripheral area of ​​the target object is calculated so that the peripheral area is included in the allowable area, thereby improving the accuracy of secondary processing based on the allowable area.

[0107] The processing of each device in the embodiments of the present disclosure can be realized by software (programs) executed by a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), etc. Note that instead of executing all of the processing of the device by software, some of the processing may be executed by hardware such as a dedicated circuit.

[0108] The above-described embodiment shows an example for realizing the present disclosure, and the present disclosure can be implemented in various other forms. For example, various modifications, substitutions, omissions, or combinations thereof are possible without departing from the spirit of the present disclosure. Such modifications, substitutions, omissions, etc. are also included within the scope of the present disclosure, as well as within the scope of the inventions described in the claims and their equivalents.

[0109] The present disclosure can also be configured as follows. [1] a classification unit that classifies the plurality of planes included in the spatial information that represents an object existing in a three-dimensional space using the plurality of planes into at least a plane corresponding to a bottom surface and a plane corresponding to an obstacle; an area specifying unit that specifies a bottom surface area relating to a plane corresponding to the bottom surface and an obstacle area relating to a plane corresponding to the obstacle; an allowable area determination unit that calculates an allowable area in which a target object existing in the three-dimensional space is allowed to be located, based on the bottom surface area and the obstacle area; An information processing device comprising: [2] The classification unit Calculating the angle between each normal to the plurality of planes and the direction of gravity; selecting a plane to be regarded as a horizontal plane from the plurality of planes based on the angle; Selecting the bottom plane from the plane considered to be a horizontal plane; [1] The information processing device according to [1]. [3] The classification unit Calculating the height of the plane regarded as the horizontal plane in the three-dimensional space; classifying the planes regarded as horizontal planes into a plurality of groups based on the calculated heights; and selecting, as the bottom plane, a plane belonging to a group with the largest number of planes among the plurality of groups. [2] The information processing device according to [2]. [4] The classification unit calculating heights of the plurality of planes in the three-dimensional space; Based on the calculated height, the plurality of planes other than the bottom surface are classified into planes corresponding to the obstacles and planes not corresponding to the obstacles. [3] The information processing device according to [3]. [5] an output unit that outputs an image showing the allowable region; The information processing device according to any one of [1] to [4], further comprising: [6] an output unit that outputs an image or sound indicating a warning when the distance between the target object and the boundary of the allowable area is equal to or less than a predetermined value; The information processing device according to any one of [1] to [4], further comprising: [7] an output unit that outputs instructions regarding the movement of the target object, and adjusts the instructions so that the movement of the target object remains within the tolerance area; The information processing device according to any one of [1] to [4], further comprising: [8] a spatial information generating unit that generates the spatial information; The information processing device according to any one of [1] to [7], further comprising: [9] The spatial information generating unit generates the spatial information from distance measurement information indicating the distance to a peripheral object acquired by a distance measuring device installed on the target object. [8] The information processing device according to [8].

[10] the spatial information generation unit generates the spatial information from distance measurement information measured by a distance measurement device; removing information corresponding to the target object from the distance measurement information or the spatial information based on the position of the target object; [9] The information processing device according to [9].

[11] a distance measurement unit that measures the distance to a subject and generates distance measurement information; a spatial information generation unit that generates the spatial information from the distance measurement information; a gravity direction acquisition unit that acquires the gravity direction; an output unit that outputs information about the allowable region; The information processing device according to [3], further comprising:

[12] An unknown area determination unit that determines unknown areas based on the occupancy grid map Furthermore, the occupancy grid map represents the three-dimensional space using a plurality of three-dimensional unit grids; at least some of the unit cells have unknown information indicating that it is unknown whether or not they are occupied by an object; the unknown region determination unit determines an unknown region based on a position of a three-dimensional unit cell having the unknown information; the allowable region determination unit prevents the unknown region from being included in the allowable region. [1] An information processing device according to any one of [1] to

[11] .

[13] the unknown region determining unit includes, in the unknown region, a location where a predetermined number or more of three-dimensional unit cells having the unknown information are stacked in a vertical direction.

[12] The information processing device according to

[12] .

[14] the unknown region determination unit does not include in the unknown region a location where a predetermined number or more of three-dimensional unit cells having the unknown information are stacked in the vertical direction, but also a predetermined number or more of three-dimensional unit cells having unoccupied information indicating that the unit is not occupied by an object are stacked.

[12] The information processing device according to

[12] .

[15] A hollow area identifying unit that identifies hollow areas based on the occupancy grid map. Furthermore, the occupancy grid map represents the three-dimensional space using a plurality of three-dimensional unit grids; at least a portion of the plurality of unit cells has unoccupied information indicating that the unit cells are not occupied by an object; the hollow region identifying unit determines a hollow region based on a position of a three-dimensional unit cell having the unoccupied information; the allowable region determination unit determines whether the hollow region is included in the allowable region. [1] to

[14] .

[16] the hollow region identifying unit includes, in the hollow region, a location of a three-dimensional unit cell that has the unoccupied information and is located lower than a plane corresponding to the bottom surface;

[15] The information processing device according to

[15] .

[17] a peripheral area determination unit that calculates, based on at least the position of the target object, an area that is part of the bottom surface area and includes the position of the target object, as a peripheral area; Furthermore, the allowable area determination unit calculates the allowable area further based on the surrounding area; [1] An information processing device according to any one of [1] to

[16] .

[18] the surrounding area determination unit adjusts the shape or size of the surrounding area based on the shooting direction of the camera and the direction of gravity;

[17] The information processing device according to

[17] .

[19] the surrounding area determination unit recognizes a general size of the target object based on information about the attributes of the target object, and adjusts the shape or size of the surrounding area in accordance with the general size of the target object.

[17] or

[18] .

[20] the surrounding area determination unit adjusts the shape or size of the surrounding area according to the orientation of the target object. An information processing device according to any one of

[17] to

[19] . [twenty one] a step of classifying the plurality of planes included in the spatial information representing an object existing in a three-dimensional space using the plurality of planes into at least a plane corresponding to a bottom surface and a plane corresponding to an obstacle; Calculating a bottom surface area relating to a plane corresponding to the bottom surface and an obstacle area relating to a plane corresponding to the obstacle; calculating an allowable area in which a target object existing in the three-dimensional space is allowed to be located based on the bottom surface area and the obstacle area; An information processing method comprising: [twenty two] a step of classifying the plurality of planes included in the spatial information representing an object existing in a three-dimensional space using the plurality of planes into at least a plane corresponding to a bottom surface and a plane corresponding to an obstacle; Calculating a bottom surface area relating to a plane corresponding to the bottom surface and an obstacle area relating to a plane corresponding to the obstacle; calculating an allowable area in which a target object existing in the three-dimensional space is allowed to be located based on the bottom surface area and the obstacle area; A program executed by a computer, comprising: [Explanation of symbols]

[0110] 1 Area determination device (information processing device) 11 Position acquisition part 12 Spatial information generation section 13 Gravity direction acquisition part 14 Area calculation section 141 Classification Department 142 Area identification part 143 Allowable area determination unit 15. Tolerance area processing section (output section) 2 users 3 Examples of tolerance areas 31 Dotted Line 32 Virtual Wall 4. 3D Mesh 41 Unit Plane (Mesh) 51 Border 52 Hollow area

Claims

1. An information processing device that automatically identifies an allowable area within which a user wearing a head-mounted display can safely move, a classification unit that acquires spatial information that represents an object existing in a three-dimensional space in which the user exists using a plurality of planes, and classifies the plurality of planes included in the spatial information into at least a plane corresponding to a bottom surface and a plane corresponding to an obstacle; an area specifying unit that specifies a bottom surface area related to a plane corresponding to the bottom surface and an obstacle area related to a plane corresponding to the obstacle based on the classification result by the classification unit; a hollow region identifying unit that identifies, based on information included in an occupation grid map that represents the three-dimensional space using a plurality of three-dimensional unit grids, a region that is located lower than the bottom surface and is unoccupied by the three-dimensional unit grids as a hollow region; an allowable area determination unit that calculates an allowable area in which the user is allowed to be located by performing a logical operation based on the bottom area, the obstacle area, and the hollow area; An information processing device comprising:

2. The information processing device further includes a peripheral area identification unit that acquires the user's position in three-dimensional space and identifies an area occupied by the user as a peripheral area based on the position. The information processing device according to claim 1 .

3. The allowable area determination unit calculates the allowable area by adding the peripheral area to a logical operation with the bottom area, the obstacle area, and the hollow area. The information processing device according to claim 2 .

4. The surrounding area identification unit determines the surrounding area by extracting a part of the bottom surface area based on the position of the user.

4. The information processing device according to claim 2.

5. The surrounding area identification unit adjusts the shape or size of the surrounding area based on the shooting direction of the camera and the direction of gravity. The information processing device according to claim 4 .

6. The surrounding area identification unit recognizes a general size of the user based on information about the user's attributes, and adjusts the shape or size of the surrounding area according to the general size of the user. The information processing device according to claim 4 .

7. The surrounding area identification unit adjusts the shape or size of the surrounding area according to the user's posture. The information processing device according to claim 4 .

8. An image showing the allowable region is output to an output device. The information processing device according to claim 1 .

9. When the distance between the user and the boundary of the allowable area is equal to or less than a predetermined value, an image or sound indicating a warning is output from an output device. The information processing device according to claim 1 .

10. Generating information for causing an output device to output instructions regarding the user's movements, the instructions being adjusted so that the user's movements remain within the tolerance zone. The information processing device according to claim 1 .

11. A spatial information generation unit that generates the spatial information based on distance measurement information indicating a distance to a surrounding object obtained by a distance measuring device worn by the user. The information processing device according to claim 1 .

12. an unknown area determination unit that determines an unknown area based on the occupation grid map; Furthermore, at least a portion of the plurality of three-dimensional unit cells has unknown information indicating that it is unknown whether or not the unit cell is occupied by an object; the unknown region determination unit determines an unknown region based on a position of a three-dimensional unit cell having the unknown information; the allowable region determination unit prevents the unknown region from being included in the allowable region. The information processing device according to claim 1 .

13. The classification unit: Calculating the angle between each normal to the plurality of planes and the direction of gravity; selecting a plane to be regarded as a horizontal plane from the plurality of planes based on the angle; selecting a plane that corresponds to the bottom plane from the planes that are considered to be horizontal; The information processing device according to claim 1 .

14. An information processing device that automatically identifies an allowable area within which a user wearing a head-mounted display can safely move, comprising: acquiring spatial information that represents an object existing in a three-dimensional space in which the user is present using a plurality of planes; classifying the plurality of planes included in the acquired spatial information into at least planes corresponding to bottom surfaces and planes corresponding to obstacles; Identifying a bottom surface area relating to a plane corresponding to the bottom surface and an obstacle area relating to a plane corresponding to the obstacle based on a result of the classification; identifying, as a hollow region, a region that is located lower than the bottom surface and that is unoccupied by the three-dimensional unit cells, based on information included in an occupation grid map that represents the three-dimensional space using a plurality of three-dimensional unit cells; calculating an allowable area in which the user is allowed to be located by performing a logical operation based on the bottom area, the obstacle area, and the hollow area; An information processing method comprising:

15. The method further comprises the step of acquiring a position of the user in the three-dimensional space, and identifying an area occupied by the user as a surrounding area based on the position. The information processing method according to claim 14.

16. The step of calculating the allowable area includes calculating the allowable area by adding the peripheral area to a logical operation with the bottom area, the obstacle area, and the hollow area. The information processing method according to claim 15.

17. The step of identifying the surrounding area includes adjusting a shape or size of the surrounding area based on a shooting direction of a camera and a direction of gravity. The information processing method according to claim 15.

18. The step of identifying the surrounding area includes recognizing a general size of the user based on information about attributes of the user, and adjusting a shape or size of the surrounding area according to the general size of the user. The information processing method according to claim 15.

19. The step of identifying the peripheral area includes adjusting a shape or size of the peripheral area depending on the posture of the user. The information processing method according to claim 15.

20. The method further comprises the step of outputting an image showing the allowable area to an output device. The information processing method according to any one of claims 14 to 19.

Images