Information processing device, information processing system, and program

Abstract

To provide a technique for appropriately presenting an action of a person in a virtual reality space.SOLUTION: An information processing device includes: a data acquisition unit that obtains data indicating the position and orientation of a user in a virtual reality space; and a data processing / output unit that based on the data, generates visualized data visually indicating an action of the user in the virtual reality space and transmits it to an external terminal.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing system, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for generating a three-dimensional geometric model of a room, fittings, etc., and simulating the room using computer graphics technology (see, for example, Patent Documents 1 and 2).

[0003] In addition, home renovations and furniture arrangements are being simulated using virtual reality (VR) systems and proposals are being made to clients and residents (hereinafter referred to simply as "clients").

[0004] Traditionally, a sales representative would be present with the client at a dealership or house manufacturer's showroom, allowing the client to experience such living spaces.

[0005] When using the VR system, sales representatives can observe the client's behavior in the VR space and make proposals that address the client's potential needs, helping to increase sales. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2021-163254 [Patent Document 2] JP 2023-39470 A Summary of the Invention [Problem to be solved by the invention]

[0007] Recently, with the advancement of technologies such as server-side rendering, it has become possible for clients to experience living spaces like those described above in their own homes.

[0008] This means that sales representatives cannot be present during the client's experience, making it difficult for them to use simulations generated by the VR system to make proposals to the client.

[0009] Hereinafter, the client experiencing the VR system will be referred to as a person in the VR space. The person is a user of the VR system, and is therefore an example of a user according to the present disclosure.

[0010] In view of the above, the present disclosure aims to provide a technology that can appropriately present the actions of a person in a VR space. [Means for solving the problem]

[0011] An information processing device according to one embodiment of the present disclosure includes a data acquisition unit that acquires data indicating a user's position and orientation within a virtual reality space, and a data processing and output unit that generates visualization data that visually indicates the user's behavior in the virtual reality space based on the data, and transmits the visualization data to an external terminal. Effect of the Invention

[0012] According to one aspect of the present disclosure, the actions of a person in a VR space can be appropriately presented. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 shows an overview of the process according to the present disclosure. [Diagram 2] FIG. 1 is a block diagram showing a configuration example of a VR log analysis and behavior visualization system according to an embodiment of the present disclosure. [Figure 3A] FIG. 2 is a block diagram showing an example of a hardware configuration of the VR log analysis device according to the embodiment; [Figure 3B] FIG. 2 is a block diagram showing an example of a functional configuration of a VR log analysis device according to the embodiment; [Figure 4] FIG. 1 is a block diagram showing an example of a hardware configuration of a user device according to the embodiment; [Figure 5A] A diagram showing an example of floor plan data [Figure 5B] Diagram showing an example of a partition [Figure 5C] A diagram showing an example of log data [Figure 6] FIG. 2 is a diagram showing a coordinate system used in the embodiment. [Figure 7] Diagram to explain the colors assigned to the walls [Figure 8] A diagram showing the relationship between the 3D coordinates of the wall and the 2D coordinates of the voting area. [Figure 9A] A diagram showing an example of a display screen of visualized data based on human footprint analysis. [Figure 9B] A diagram showing an example of a display screen for visualized data based on human trajectory analysis. [Figure 9C] A diagram showing an example of a display screen for visualized data based on wall attention analysis of people [Figure 9D] A diagram showing an example of a display screen of visualized data based on a person's footprint analysis, trajectory analysis, and wall attention analysis. [Figure 9E] A diagram showing an example of a display screen for visualized data based on wall attention analysis of people [Figure 10] A flowchart showing an example of the operation of the VR log analysis and behavior visualization system according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] <Summary of this Disclosure> First, an overview of the processing according to the present disclosure will be described with reference to FIG.

[0015] Fig. 1 is a diagram showing an overview of processing according to the present disclosure. As shown in Fig. 1, a VR log analysis device 10, which is an example of an information processing device according to the present disclosure, receives as input floor plan data of a building such as a detached house or an apartment house in a VR space, shape coordinate data of a section in the building defined by the walls of the building, and log data indicating the position and orientation of a person who moves within the building represented by the above-mentioned floor plan data and shape coordinate data.

[0016] 1, the VR log analysis device 10 acquires a user input from a user of a user device 20 described later. In the present disclosure, the user of the user device 20 is assumed to be a sales representative of a house manufacturer or a housing sales store, but is not limited thereto.

[0017] The VR log analysis device 10 generates visualization data that visually shows the behavior of a person in the VR space using the floor plan data, shape coordinate data, and log data in accordance with user input from the user of the user device 20, and outputs the visualization data (e.g., transmits the visualization data to the user device 20).

[0018] The visualization data generated and output by the VR log analysis device 10 includes a visual representation of the person's position and orientation (e.g., shown by a circle and a single line, respectively), the trajectory the person moved along, and / or a visual representation of the location (e.g., a wall) that the person gazed upon (focused on) (these are superimposed on the floor plan data), as well as a visual representation of how frequently or to what extent the person gazed upon the location or the area that constitutes that location.

[0019] According to the above, visualization data visually showing the behavior of a person in the VR space is appropriately presented to the user of the user device 20. As a result, even if the user of the user device 20 is not in the same room as the person, the user can grasp the person's behavior (movements and areas of interest) as if he or she were in the same room, and can make effective suggestions to the person. For example, the user of the user device 20 can easily consider suggestions (remodeling, furniture purchases, etc.) that are in line with the person's preferences. Furthermore, by making such suggestions, the satisfaction of the person (customer) increases, and sales of dealers, house manufacturers, etc. also increase.

[0020] (Embodiment) Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. However, the scope of the present disclosure is not limited to the examples shown in the drawings. In the following description, the same reference numerals are used to designate components having the same functions and configurations, and the description thereof may be omitted.

[0021] <Configuration of VR Log Analysis and Behavior Visualization System> FIG. 2 is a block diagram showing a configuration example of a VR log analysis and behavior visualization system according to an embodiment of the present disclosure. The VR log analysis and behavior visualization system shown in FIG. 2 is an example of an information processing system according to the present disclosure.

[0022] As shown in FIG. 2, the VR log analysis and behavior visualization system 1 includes a VR log analysis device 10 and a user device 20. The VR log analysis device 10 and the user device 20 are communicably connected to each other via a network 30. The network 30 may be a LAN (Local Area Network), the Internet, an intranet, a VPN (Virtual Private Network), a mobile communication network, or a combination thereof.

[0023] The VR log analysis and behavior visualization system 1 analyzes (processes) log data (the position and orientation of a person) indicating the result of a person moving within a house (living space) arranged in a VR space, generates visualization data (visual data) visually showing the behavior of the person, and presents the generated visualization data to the user of the user device 20.

[0024] The VR log analysis device 10 may be installed in a cloud environment and implemented by a computer such as a server, for example.

[0025] The VR log analysis device 10 acquires floor plan data of a house in a VR space, shape coordinate data of sections within the house in the VR space, and log data (position and orientation) of a person in the VR space.

[0026] The VR log analysis device 10 receives a user input instructing the analysis of log data from the user device 20 via the network 30.

[0027] The VR log analysis device 10 executes analysis of the log data in response to a user input instructing analysis of the log data, and generates screen data including visualization data visually showing the actions of a person as a result of the analysis of the log data.

[0028] Specifically, the VR log analysis device 10 performs a person's footprint analysis, which specifies the position and orientation of a person on a floor of a house to be analyzed based on the log data, and generates screen data including visualization data visually showing the specified position and orientation. The VR log analysis device 10 also performs a person's trajectory analysis, which specifies the position of a person on a floor of a house to be analyzed based on the log data, and generates screen data including visualization data visually showing the trajectory of the person's movement on the floor based on the specified position. The VR log analysis device 10 also performs a person's wall attention analysis, which specifies the position and orientation of a person on a floor of a house to be analyzed based on the log data, and specifies the wall that the person gazes at (attention) and the area that constitutes the wall that the person gazes at (attention) based on the position of the wall on the floor and the specified position and orientation, and generates screen data including visualization data that is a visual representation for identifying the specified wall and area.

[0029] The VR log analysis device 10 transmits (outputs) screen data including the generated visualization data to the user device 20 via the network 30.

[0030] The user device 20 may be realized by a computer such as a tablet, a smartphone, or a personal computer (PC). The user device 20 has a configuration capable of connecting to the VR log analysis device 10 via a dedicated application or a web browser installed in the user device 20. For example, the following processing of the user device 20 may be performed via the dedicated application or the web browser.

[0031] The user device 20 receives a user input for instructing the analysis of log data from the user of the user device 20, and transmits the received user input to the VR log analysis device 10 via the network 30.

[0032] The user device 20 receives screen data including visualization data that visually shows the actions of a person, which is the analysis result of log data, from the VR log analysis device 10, and causes the display device (display unit) of the user device 20 to display the received visualization data.

[0033] The user device 20 may be referred to as a device, a user terminal, a terminal, etc., and is an external terminal for the VR log analysis device 10.

[0034] <Configuration of VR Log Analysis Device> FIG. 3A is a block diagram showing an example of the hardware configuration of the VR log analysis device according to the present embodiment. The VR log analysis device 10 may be realized by a computer (computer) such as a server or an information processing device.

[0035] For example, as shown in FIG. 3A, the VR log analysis device 10 may include a storage device 11, a processing device 12, a user interface (UI) device 13, a communication device 14, and a bus 15. The storage device 11, the processing device 12, the UI device 13, and the communication device 14 may be interconnected via the bus 15.

[0036] The program or instruction for realizing the functions and processes described above and below in the VR log analysis device 10 may be downloaded from some external device (for example, a server) via a network or the like. Also, such a program or instruction may be provided from a removable storage medium such as a CD-ROM (Compact Disc Read Only Memory) or a flash memory.

[0037] Storage device 11 is realized by a random access memory (RAM), a flash memory, a hard disk drive, etc., and stores installed programs or instructions as well as files, data, etc. used in executing the programs or instructions. Storage device 11 may include a non-transitory storage medium.

[0038] The processing device 12 may be realized by, for example, a general-purpose processor or controller (circuit), or may be realized by a dedicated processor or controller (circuit). When realized by a general-purpose processor, the processing device 12 may be realized by one or more central processing units (CPUs), graphics processing units (GPUs), processing circuitry, etc., which may be composed of one or more processor cores. In this case, the processing device 12 executes the functions and processing of the VR log analysis device 10 described above and below according to a program or instruction stored in the storage device 11, data such as parameters used to execute the program or instruction, etc. Such a program may be a program that causes the computer or processing device 12 to function as the VR log analysis device 10 according to an embodiment of the present disclosure when executed by the computer or processing device 12. In addition, such a program may be a program that causes the computer or processing device 12 to execute a VR log processing method according to an embodiment of the present disclosure when executed by the computer or processing device 12.

[0039] The UI device 13 may be composed of input devices such as a keyboard, a mouse, a camera, a microphone, etc., output devices such as a display, a speaker, a headset, a printer, etc., and input / output devices such as a haptic device, a touch panel, etc., and realizes an interface between a user of the VR log analysis device 10 and the VR log analysis device 10. For example, the user of the VR log analysis device 10 operates the VR log analysis device 10 by operating a graphical user interface (GUI) displayed on a display or a touch panel using a keyboard, a mouse, etc.

[0040] The communication device 14 may be realized by various communication circuits that execute communication processes with external devices and communication networks such as the Internet, an intranet, a LAN, and a VPN.

[0041] The above-described hardware configuration of the VR log analysis device 10 is merely an example, and the VR log analysis device 10 according to the embodiment of the present disclosure may be realized by another appropriate hardware configuration.

[0042] FIG. 3B is a block diagram showing an example of the functional configuration of the VR log analysis device according to the present embodiment.

[0043] 3B, the VR log analysis device 10 includes a data acquisition unit 101, a user input acquisition unit 102, a log data processing unit 103, and a visualization data output unit 104. The data acquisition unit 101, the user input acquisition unit 102, the log data processing unit 103, and the visualization data output unit 104 may be realized by the processing device 12 as hardware, software, or a combination of hardware and software.

[0044] The data acquisition unit 101 acquires floor plan data of a house in a VR space, shape coordinate data of a section in the house in the VR space, and log data of a person in the VR space.

[0045] The floor plan data is acquired as image data, and for a one-story house, one sheet of floor plan data (for the first floor) is acquired, for a two-story house, two sheets of floor plan data (for the first and second floors) are acquired, for a three-story house, three sheets of floor plan data (for the first to third floors) are acquired, etc. The data acquisition unit 101 may also acquire information on the origin position and scale as supplementary information of the floor plan data (these pieces of information may be individual for each floor).

[0046] FIG. 5A(a) shows floor plan data showing the first floor of a two-story house, and FIG. 5A(b) shows floor plan data showing the second floor of the same house.

[0047] The shape coordinate data is obtained as numerical data representing the three-dimensional vertex coordinates of each outline section (polygon) defined by the walls (and floor and ceiling). Note that the outline section is sometimes simply called a section.

[0048] Fig. 5B is floor plan data showing the first floor of the two-story house shown in Fig. 5A(a), and shows each outline section surrounded by a thick line. The living / dining / kitchen section shown in the center of Fig. 5B is approximated by an octagonal prism, and the other sections are approximated by square prisms.

[0049] The log data is data that records a person's simulation experience in a VR space, and is acquired as numerical data that indicates the person's position and orientation.

[0050] Fig. 5C shows an example of log data recorded at regular time intervals. In Fig. 5C, Time represents time, Xw, Yw, and Zw represent the position of a person (Xw: X coordinate, Yw: Y coordinate, and Zw: Z coordinate), and RotX and RotY represent the direction of the person (RotX: elevation angle, RotY: azimuth angle). RotZ is not used in this embodiment.

[0051] Now, with reference to FIG. 6, the coordinate system used in this embodiment will be described.

[0052] 6A, in the plan view data, a position on a plane is expressed by an Xp axis, the right direction of the paper being positive, and a Yp axis, the upward direction of the paper being positive. The origin position may be arbitrary.

[0053] As shown in FIG. 6B, in a three-dimensional space, the position of a person is expressed by (Xw, Yw, Zw). Here, the XwZw plane is the horizontal plane, and the Yw axis represents the height direction. The orientation of the person is expressed by (RotX, RotY, RotZ). Here, RotX represents the elevation angle, with the horizontal being 0 degrees and a range of ±90 degrees up and down, and RotY represents the azimuth angle, with the Zw direction being 0 degrees and the Xw positive direction being 90 degrees, ranging from 0 degrees to 360 degrees. RotZ represents the inclination of the person's neck, but as described above, it is not used in this embodiment.

[0054] The relationship between the plan coordinates (Xp, Yp) in the coordinate system shown in FIG. 6(a) and the three-dimensional coordinates (Xw, Yw, Zw) in the coordinate system shown in FIG. 6(b) is as follows. Xw=-Xp Zw=-Yp

[0055] It should be noted that although FIG. 6 illustrates an example of a left-handed coordinate system, the present disclosure is not limited to this example.

[0056] The floor plan data and shape coordinate data are also used in the home experiencing VR system, and the log data is a record of the position and orientation of a person in the VR space of the home experiencing VR system over time. The data acquisition unit 101 may acquire this data by importing it from the home experiencing VR system, or may acquire the data by having the VR log analysis device 10 read a file in which a person has recorded this data.

[0057] The data acquisition unit 101 stores the acquired plan view data, shape coordinate data, and log data in a storage area of ​​the storage device 11 that is accessible by the log data processing unit 103 and the visualization data output unit 104. Alternatively, the data acquisition unit 101 may output the acquired plan view data, shape coordinate data, and log data to the log data processing unit 103 and the visualization data output unit 104.

[0058] The user input acquisition unit 102 acquires user input by receiving a user input such as a user input instructing the start or end of the method according to the present disclosure, a user input specifying a floor of a house, etc. The user input instructing the start of the method according to the present disclosure also includes information specifying which person experienced which house in the home experience VR system. The user input acquisition unit 102 outputs the user input specifying the floor of the house to the log data processing unit 103. The following processing of the log data processing unit 103 and the visualization data output unit 104 may be executed every time the user input acquisition unit 102 acquires a user input specifying a floor of a house. Therefore, the user input specifying the floor of a house may be interpreted as a user input instructing the analysis of log data.

[0059] The log data processing unit 103 executes log data analysis (processing) using the floor plan data and / or shape coordinate data as appropriate in response to a user input instructing analysis of the log data output by the user input acquisition unit 102. The log data analysis includes, for example, a person's footprint analysis, a person's trajectory analysis, and a person's wall attention analysis.

[0060] For the person's footprint analysis, the log data processing unit 103 extracts from the log data the time when the person stayed on the floor specified by the user of the user device 20 and the person's position and orientation. The log data processing unit 103 outputs the extracted time and the person's position and orientation to the visualization data output unit 104. Since the floor and the Yw coordinate range are associated in advance, the log data processing unit 103 can identify the floor on which the person stayed from the person's Yw coordinate value.

[0061] Regarding the person's trajectory analysis, the log data processing unit 103 extracts, from the log data, the time when the person stayed on the floor specified by the user of the user device 20 and the person's position. The log data processing unit 103 outputs the extracted time and the person's position to the visualization data output unit 104.

[0062] Regarding the wall attention analysis of a person, the log data processing unit 103 extracts from the log data the time when the person stayed on the floor specified by the user of the user device 20, and the position and orientation of the person. Based on the position of the person (which may be read as the "point of view of the person") and the orientation of the person (which may be read as the "line of sight of the person"), the log data processing unit 103 identifies a wall at which the line of sight of the person intersects on the specified floor. The log data processing unit 103 also counts the number of times (frequency) the line of sight of the person intersects with each wall. Furthermore, the log data processing unit 103 calculates the coordinates of the intersection of the line of sight of the person and the wall, calculates the distance between the position of the person and the intersection, and calculates an index value representing the degree to which the person gazes at each area constituting the wall based on the calculated distance. The log data processing unit 103 outputs information identifying the wall (referred to as wall information; for example, coordinates representing the wall), the number of times the line of sight intersected with the wall, and the index value to the visualization data output unit 104. The wall attention analysis will be described in detail below.

[0063] Regarding the analysis of a person's footprints, when the log data processing unit 103 outputs the time and the person's position and orientation, the visualization data output unit 104 generates screen data including visualization data in which the position of the person at each time is plotted as a circle in the floor plan data of the floor specified by the user of the user device 20, and the orientation of the person at each time is drawn as a single line.

[0064] Regarding person trajectory analysis, when the log data processing unit 103 outputs the time and the person's position, the visualization data output unit 104 generates screen data including visualization data in which the positions of the person at adjacent times among the discretely represented times in the floor plan data of the floor specified by the user of the user device 20 are connected by straight lines.

[0065] Regarding the analysis of a person's attention to a wall, when the log data processing unit 103 outputs wall information and the number of times the line of sight intersects with the wall, the visualization data output unit 104 generates screen data including visualization data in which a rectangle is drawn in a color corresponding to the number of times the line of sight intersects near the wall in the plan data of the floor specified by the user of the user device 20. In addition, when the log data processing unit 103 outputs the wall information and the index value, the visualization data output unit 104 generates screen data including visualization data (heat map) in which a color corresponding to the index value is colored in a planar image showing the wall. The generation of these visualization data will be described in detail below.

[0066] The visualized data output unit 104 outputs all of the above-mentioned screen data including the generated visualized data. For example, the visualized data output unit 104 outputs (transmits) all of the screen data including the generated visualized data to the user device 20 via the communication device 14 in response to a user input instructing analysis of log data.

[0067] <User device configuration> 4 is a block diagram showing an example of a hardware configuration of a user device according to this embodiment. The user device 20 may be realized by any computer or information processing device, such as a tablet, a smartphone, or a PC (Personal Computer) (e.g., a notebook PC).

[0068] 4, the user device 20 may include a storage device 21, a processing device 22, a UI device 23, a communication device 24, and a bus 25. The storage device 21, the processing device 22, the UI device 23, and the communication device 24 may be connected to each other via the bus 25.

[0069] The programs or instructions for implementing the functions and processes of the user device 20 described above and below may be downloaded from some external device (e.g., a server) via a network, etc. Also, such programs or instructions may be provided from a removable storage medium such as a CD-ROM (Compact Disc Read Only Memory) or a flash memory.

[0070] Storage device 21 is realized by RAM, flash memory, a hard disk drive, etc., and stores installed programs or instructions as well as files, data, etc. used in executing the programs or instructions. Storage device 21 may include a non-transitory storage medium.

[0071] The processing device 22 may be realized, for example, by a general-purpose processor or controller (circuit), or may be realized by a dedicated processor or controller (circuit). When realized by a general-purpose processor, the processing device 22 may be realized by one or more CPUs, GPUs, processing circuits, etc., which may be composed of one or more processor cores. In this case, the processing device 22 executes the functions and processing of the user device 20 described above and below according to a program or instruction stored in the storage device 21, data such as parameters used to execute the program or instruction, and the like. Such a program may be a program that causes the computer or processing device 22 to function as the user device 20 according to the present disclosure when executed by the computer or processing device 22. In addition, such a program may be a program that causes the computer or processing device 22 to execute the VR log processing method according to the present disclosure when executed by the computer or processing device 22.

[0072] The UI device 23 may be composed of input devices such as a keyboard, a mouse, a camera, and a microphone, output devices such as a display, a speaker, a headset, and a printer, and input / output devices such as a haptic device and a touch panel, and realizes an interface between a user of the user device 20 and the user device 20. For example, the user of the user device 20 operates the user device 20 by operating a graphical user interface (GUI) displayed on a display or a touch panel using a keyboard, a mouse, and the like.

[0073] The communication device 24 may be realized by various communication circuits that execute communication processes with external devices and communication networks such as the Internet, an intranet, a LAN, and a VPN.

[0074] The above-described hardware configuration of the user device 20 is merely an example, and the user device 20 according to the present disclosure may be realized by any other appropriate hardware configuration.

[0075] The processing device 22 accepts a user input from the user of the user device 20 using the UI device 23. For a predetermined user input (for example, a user input specifying a floor), the processing device 22 transmits the accepted user input to the VR log analysis device 10 via the communication device 24.

[0076] The processing device 22 receives screen data including floor plan data and visualization data from the VR log analysis device 10 via the communication device 24.

[0077] The processing device 22 causes a display device (display unit) included in the UI device 23 to display the received screen data including the floor plan data and visualization data.

[0078] The above-mentioned user input may include the following (in the following, it is assumed that the user device 20 is a PC): - Enlarging the drawing (plan view data / visualization data) (e.g., enlarging one step at a time with left mouse click) - Zoom out the drawing (for example, zoom out one step at a time with a right mouse click) - Moving the drawing (e.g. by dragging the mouse) -Floor navigation (i.e. specifying a floor) (e.g. pressing the up / down arrow keys to move one floor at a time) - Show / hide footprints (for example, press F to toggle between showing and hiding) - Show / hide the trajectory (for example, press the L key to toggle between showing and hiding) - Wall attention display / hide (for example, press A key to toggle between display and hide) - Display of attention distribution for each wall (for example, by double-clicking the mouse on a rectangle)

[0079] The above-mentioned user inputs are merely examples. Even if the user device 20 is a tablet or a smartphone, the same (display) control is possible by operating these devices (for example, operating a touch-type display).

[0080] <Details of wall attention analysis and visualization data (heat map) generation> Next, details of an exemplary wall attention level analysis performed by the log data processing unit 103 and details of an exemplary visualization data (heat map) generation performed by the visualization data output unit 104 will be described.

[0081] First, the log data processing unit 103 extracts log data when a person stayed on a floor specified by the user of the user device 20, based on the Y coordinate Yw of the person.

[0082] Next, the log data processing unit 103 performs an inside / outside determination of all sections (closed polygons) existing on a floor specified by the user of the user device 20 with respect to the person's position (viewpoint) in the log data when the person was staying on that floor, and identifies the section in which the person was staying.

[0083] In one example, the inside / outside determination is performed in two dimensions of Xw-Zw. The log data processor 103 draws a half line from the position of the person in a specific direction (in this example, the negative Xw direction), and if the number of times the half line intersects with each side of the closed polygon is an even number, it determines that the person was outside the section, and if the number of times the half line intersects with each side of the closed polygon is an odd number, it determines that the person was inside the section.

[0084] Next, the log data processor 103 executes a determination of intersection between the line of sight of the person and a wall based on the position and orientation (line of sight) of the person and the vertex coordinates of the section, and identifies a wall that intersects with the line of sight of the person. The wall is assumed to be a rectangle or a square consisting of a horizontal side and a vertical side.

[0085] In one example, the intersection detection is simply performed in two dimensions, Xw-Zw. Let the position of the person be P=(x p ,0,z p ) and the line of sight vector is

number

number

[0086] The log data processing unit 103 outputs to the visualization data output unit 104 wall information about the wall identified in this manner and the number of times the line of sight intersects with the wall.

[0087] The visualization data output unit 104 assigns a color to the wall according to the number of times the wall and the line of sight intersect (called the number of intersections), as follows:

[0088] The maximum number of intersections (maximum frequency) on a floor specified by the user of the user device 20 is n max Let the number of wall intersections be n i (i is an integer in the range of 0 to (the number of walls existing on the floor)−1, and indicates one of the walls existing on the floor.) In one example, the visualization data output unit 104 calculates a relative frequency r=n i / n max Calculate n according to the value of r normalized from 0 to 1. i The visualization data output unit 104 assigns color R (red), G (green), and B (blue) values ​​to the wall corresponding to the intersection number. In one example, the brightness of the color components RGB is expressed in the range of 0 to 1, and as shown in the tone curve in the upper part of Fig. 7, the brightness of the color components RGB is given and associated with the relative frequency r, and conversion to a color table in which the relative frequency and color are associated with each other is performed, as shown in the lower part of Fig. 7. For example, the visualization data output unit 104 assigns colors to the walls according to the number of intersections, changing from blue to green to yellow to orange to red as the number of intersections increases.

[0089] Next, a procedure for generating a heat map will be described. It is assumed that the walls that intersect with the line of sight of the person have been specified as described above, and the number of times that the line of sight of the person intersects with the target wall is N.

[0090] The log data processing unit 103 calculates the intersection point between the person's line of sight and the wall as a coordinate value from a reference point of the wall (for example, the upper left corner of the wall) by a known intersection calculation method, and calculates the distance between the person's position (viewpoint) and the calculated intersection point. Here, the intersection coordinates are C i =(x wi ,y wi ,z wi ) and D iThe intersection point C with the person's viewpoint i Let i be the distance between the intersection and the wall, where i is an integer in the range of 0 to (N-1) and indicates the i-th intersection. Let the size of the wall be S wx ×S wy Let the coordinate of the reference point (upper left corner) of the wall be Q = (x wq ,y wq ,z wq )

[0091] Next, the log data processor 103 calculates the size S wx ×S wy A two-dimensional array of voting area V with the same aspect ratio (size of voting area V: S vx ×S vy ) and allocate all elements A(x v ,y v ) to 0 (clear it).

[0092] 8 shows the relationship between the three-dimensional coordinates of the wall and the two-dimensional coordinates of the voting region V. The three-dimensional wall and the two-dimensional voting region are similar, and the log data processing unit 103 calculates the intersection C by the following formula (2). i The position of the voting region corresponding to (x v ,y v ) is calculated.

number

[0093] In this example, the range of 30 degrees, which is the central visual field of a human visual field, is regarded as "gazing (attention)", and the angle at which the viewpoint is looking at the wall is approximated as directly ahead. Next, the log data processing unit 103 calculates the distance D i Based on this, the gaze radius r in the three-dimensional area is calculated using the following equation (3). wi The gaze radius r is calculated using the following formula (4). wi Let V be the radius r vi Convert to.

number

number

[0094] Next, the log data processor 103 adds the value of the attention density distribution according to the two-dimensional Gaussian distribution to each element of the voting region V (two-dimensional array).

[0095] The two-dimensional Gaussian distribution is obtained by the following equation (5).

number

number

number

number

[0096] Then, for each intersection i, the log data processing unit 103 calculates the total number of pixels (x a ,y a ) for intersection point C i The position of the voting region corresponding to (x v ,y v ) is calculated, and h(r) obtained for each pixel is added using equation (6).

number

[0097] The log data processing unit 103 calculates the wall information of the wall and all elements A(x v ,y v ) to the visualization data output unit 104. v ,y v ) is an example of the index value mentioned above.

[0098] The visualization data output unit 104 outputs the element A(x v ,y v ) is assigned to the area (pixels) that constitutes the wall corresponding to the element.

[0099] This color assignment may be the same as the above-mentioned example of assigning colors to walls according to the number of intersections. That is, in this color assignment, the "n i " is "A(x v ,y v )" and "n" in the above explanation max " is "A(x v ,y v ) may be replaced with "maximum of

[0100] In this way, when the entire wall is colored blue to red, blue may be assigned to areas where the person did not gaze at all, which may give the mistaken impression that the person gazed at the area to some extent. To avoid this, a threshold may be set for coloring, and the visualization data output unit 104 may color pixels having a value less than the threshold (a value normalized between 0 and 1) in gray. The threshold may be 0.01, for example. Furthermore, the visualization data output unit 104 may color pixels having a value equal to or greater than the threshold and less than a second threshold greater than the threshold (first threshold) by blending gray and blue. The second threshold may be 0.25, for example. For example, the visualization data output unit 104 may color pixels having a value of 0.25 100% blue, pixels having a value of 0.01 100% gray, and pixels having a value of 0.01 to 0.25 by linear interpolation so that the proportion of gray increases as the value decreases. In this way, the entire wall can be smoothly expressed, and the areas not being gazed at can be made clear, while areas with low gaze (attention) can also be clearly expressed.

[0101] <Example of a display screen for visualized data> Next, an example of a display screen on the user device 20 of the visualization data generated as described above will be described.

[0102] 9A is a diagram showing an example of a display screen of visualization data based on a person's footprint analysis. As shown in FIG. 9A, the position where the person stayed in the VR space is visually indicated by a circle 901, and the person's direction is visually indicated by a single line 902 protruding from the circle 901.

[0103] 9B is a diagram showing an example of a display screen for visualization data based on a person's trajectory analysis. As shown in FIG. 9B, the trajectory of a person's movement in the VR space is visually shown by a group of straight line segments 911.

[0104] FIG. 9C is a diagram showing an example of a display screen of visualization data based on the analysis of the wall attention degree of a person. As shown in FIG. 9C, the wall that the person gazed at in the VR space is visually indicated by the quadrilateral 921 in its vicinity.

[0105] FIG. 9D is a diagram showing an example of a display screen of visualization data based on the analysis of the footprint, trajectory, and wall attention degree of a person. As shown in FIG. 9D, the visualization data shown in FIGS. 9A to 9C are superimposed and displayed.

[0106] FIG. 9E is a diagram showing an example of a display screen of visualization data based on the analysis of the wall attention degree of a person. As shown in FIG. 9E, the area of the target wall that the person gazed at is displayed in color-coded. In the example of the display screen shown in FIG. 9E, the person has gazed at the target wall twice. Once, the person gazed at the left side of the target wall from a distance, and once, the person gazed at the center right side of the target wall from close by.

[0107] <Operation of VR Log Analysis and Behavior Visualization System> Next, with reference to FIG. 10, an example of the operation of the VR log analysis and behavior visualization system according to the present embodiment will be described.

[0108] FIG. 10 is a flowchart showing an example of the operation of the VR log analysis device 10 according to the present embodiment. Before the flow shown in FIG. 10 starts, it is assumed that the VR log analysis device 10 (data acquisition unit 101) has acquired floor plan data, shape coordinate data, and log data.

[0109] In step S11, the VR log analysis device 10 receives a user input designating the floor of the house from the user device 20. Step S11 may be executed by the user input acquisition unit 102 as described above.

[0110] In step S12, the VR log analysis device 10 executes the analysis of the footprint of the person. Step S12 may be executed by the log data processing unit 103 as described above.

[0111] In step S13, the VR log analysis device 10 executes a person trajectory analysis. Step S13 may be executed by the log data processing unit 103 as described above.

[0112] In step S14, the VR log analysis device 10 executes a wall attention analysis of a person. Step S14 may be executed by the log data processing unit 103 as described above.

[0113] In step S15, the VR log analysis device 10 generates screen data including visualization data visually showing the behavior of the person in the VR space based on the results of the analysis in steps S12 to S14. Step S15 may be executed by the visualization data output unit 104 as described above.

[0114] In step S16, the VR log analysis device 10 transmits the screen data generated in step S15 to the user device 20, and the flow ends. Step S16 may be executed by the visualization data output unit 104 as described above.

[0115] As a result of the flow shown in FIG. 10, the user device 20 causes the display unit of the user device 20 to display visualized data visually showing the actions of a person.

[0116] <Effects of the embodiment> As described above, the VR log analysis device 10 according to the present embodiment acquires data indicating the position and orientation of a person in the VR space, and generates visualization data visually indicating the person's behavior in the VR space based on the acquired data, and transmits the visualization data to the user device 20. As a result, the visualization data visually indicating the person's behavior in the VR space is appropriately presented to the user of the user device 20. Therefore, even if the user of the user device 20 is not present at the same time as the person, the user can grasp the person's behavior (traffic path and points of interest) as if he or she were present at the same time, and can make effective suggestions to the person. For example, the user of the user device 20 can easily consider suggestions (remodeling, furniture purchase, etc.) that are in line with the person's preferences. In addition, by making such suggestions, the satisfaction of the person (customer) increases, and sales of dealers, house manufacturers, etc. also increase.

[0117] <Modifications of the embodiment> In the above embodiment, the log data analysis (processing) is executed every time a user input specifying a floor of a house is acquired, but the present invention is not limited to this example. For example, the log data analysis (processing) may be executed for all floors of a house in response to a user input instructing the start of the method according to the present disclosure, and visualization data that is the analysis result may be output (transmitted) to the user device 20.

[0118] The functional units described and illustrated in the above embodiment may be divided into a plurality of functional units, or two or more functional units may be integrated into one functional unit. For example, the log data processing unit 103 and the visualization data output unit 104 may be integrated into one functional unit such as a data processing / output unit. The names of the functional units are merely examples and may be changed as appropriate.

[0119] The steps in the flowcharts described and illustrated in the above embodiments may be executed in a different order or in parallel, as appropriate.

[0120] <Summary of the embodiment> An information processing device according to one embodiment of the present disclosure includes a data acquisition unit that acquires data indicating a user's position and orientation within a virtual reality space, and a data processing and output unit that generates visualization data that visually indicates the user's behavior in the virtual reality space based on the data, and transmits the visualization data to an external terminal.

[0121] In one example, the data processing and output unit identifies a wall that the user is gazing at based on the data and the positions of walls in a house placed within the virtual reality space, and generates a visual representation associated with the identified wall as the visualization data.

[0122] In one example, the visual representation includes a color that corresponds to the number of times the user gazes at the identified wall.

[0123] In one example, the data processing and output unit identifies an area constituting the wall that the user is gazing at based on the data and the positions of walls in a house placed within the virtual reality space, and generates a visual representation associated with the identified area as the visualization data.

[0124] In one example, the visual representation includes a color that corresponds to the degree to which the user gazes at the identified region.

[0125] An information processing system according to one aspect of the present disclosure includes an information processing device and a terminal, wherein the information processing device acquires data indicating a user's position and orientation within a virtual reality space, and based on the data, generates visualization data that visually indicates the user's behavior in the virtual reality space and transmits it to the terminal, and the terminal receives the visualization data from the information processing device and displays it on a display unit.

[0126] A program according to one embodiment of the present disclosure is a program for causing a computer to execute a process including acquiring data indicating a user's position and orientation within a virtual reality space, and generating visualization data that visually indicates the user's behavior in the virtual reality space based on the data, and transmitting the visualization data to an external terminal.

[0127] Although the embodiments of the present disclosure have been described above with reference to the drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modified or amended examples within the scope of the claims. It is understood that such modified or amended examples also belong to the technical scope of the present disclosure. In addition, each component or each item in the embodiment (including modified examples) may be arbitrarily combined within the scope of the present disclosure.

[0128] The present disclosure can be realized in software, hardware, or software in conjunction with hardware. [Industrial Applicability]

[0129] One aspect of the present disclosure is useful as an information processing device that analyzes the behavior of a person in a VR space and generates visualization data that visually shows the behavior of the person. [Explanation of symbols]

[0130] 1 VR log analysis and behavior visualization system 10 VR log analysis device 11 Storage device 12 Processing equipment 13 User Interface (UI) Devices 14 Communication equipment 101 Data Acquisition Section 102 User input acquisition unit 103 Log data processing section 104 Visualization data output section 20 User Devices 21 Storage device 22 Processing equipment 23 User Interface (UI) Devices 24 Communication Equipment

Claims

1. A data acquisition unit that acquires data indicating a position and orientation of a user in a virtual reality space; a data processing and output unit that generates visualization data that visually indicates the user's behavior in the virtual reality space based on the data, and transmits the visualization data to an external terminal; An information processing device comprising:

2. The data processing and output unit identifies a wall that the user gazes at based on the position of a wall in the house arranged in the virtual reality space and the data, and generates, as the visualization data, a visual representation associated with the identified wall. The information processing device according to claim 1 .

3. the visual representation includes a color corresponding to the number of times the user gazes at the identified wall; The information processing device according to claim 2 .

4. The data processing and output unit identifies an area constituting the wall gazed upon by the user based on the position of a wall in the house arranged in the virtual reality space and the data, and generates a visual representation associated with the identified area as the visualization data. The information processing device according to claim 1 .

5. the visual representation includes a color corresponding to the degree to which the user gazes at the identified region. The information processing device according to claim 4.

6. An information processing system including an information processing device and a terminal, The information processing device includes: obtaining data indicative of a user's position and orientation within a virtual reality space; generating visualization data that visually indicates the user's behavior in the virtual reality space based on the data, and transmitting the visualization data to the terminal; The terminal includes: receiving the visualization data from the information processing device and displaying it on a display unit; Information processing system.

7. obtaining data indicative of a user's position and orientation within a virtual reality space; generating visualization data that visually indicates the user's behavior in the virtual reality space based on the data, and transmitting the visualization data to an external terminal; A program for causing a computer to execute a process including the steps of:

Images