Image output method, image output system, image output device and image output program

Abstract

To provide an image output method or the like capable of referring to an interior design result after selecting a theme, and also to provide an image output method or the like capable of providing a three-dimensional image with a sense of presence.SOLUTION: An image output method includes the steps of: outputting a plurality of classifications relating to an interior of a house; accepting the selection of classification; generating a virtual space which imitates a living space constituted by an interior component corresponding to the selected classification; acquiring a virtual space image viewed from a virtual viewpoint in the generated virtual space; and performing processing which outputs the acquired virtual space image by a computer.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to an image output method for outputting a virtual space image viewed from a virtual viewpoint within a virtual space. [Background technology]

[0002] A device for assisting interior design has been proposed (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2001-142924 Summary of the Invention [Problem to be solved by the invention]

[0004] In the known technology described in Patent Document 1, after selecting the interior theme, it is necessary to select the color and pattern of the interior materials and fixtures. Also, the output image is a two-dimensional image, which lacks realism.

[0005] The present invention has been made in light of these circumstances. Its purpose is to provide an image output method that allows users to refer to the results of interior design after selecting a theme. It also aims to provide an image output method that can provide realistic three-dimensional images. [Means for solving the problem]

[0006] An image output method according to one aspect of the present application is characterized in that a computer performs a process of outputting multiple classifications related to residential interiors, accepting a selection of a classification, generating a virtual space that simulates a living space composed of interior components according to the selected classification, acquiring a virtual space image viewed from a virtual viewpoint within the generated virtual space, and outputting the acquired virtual space image. [Effects of the Invention]

[0007] In one aspect of the present application, after selecting a category (theme), the interior design results can be viewed. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram showing an example of the configuration of a VR system. [Figure 2] FIG. 10 is an explanatory diagram showing an example of a theme DB. [Figure 3] FIG. 10 is an explanatory diagram showing an example of a recommended combination DB. [Figure 4] FIG. 10 is an explanatory diagram showing an example of a texture DB. [Figure 5] FIG. 10 is an explanatory diagram showing an example of a room DB. [Figure 6] FIG. 10 is an explanatory diagram illustrating an example of a vertex DB. [Figure 7] FIG. 10 is an explanatory diagram showing an example of an edge DB. [Figure 8] FIG. 10 is an explanatory diagram showing an example of a surface DB. [Figure 9] FIG. 2 is an explanatory diagram showing an example of an object DB. [Figure 10] 10 is a flowchart illustrating an example of a procedure of a main process. [Figure 11] 10 is a flowchart illustrating an example of a procedure for generating a walk-through screen. [Figure 12] 10 is a flowchart illustrating an example of a procedure for screen processing. [Figure 13] 10 is a flowchart illustrating an example of a procedure for color selection processing. [Figure 14] 10 is a flowchart illustrating an example of a procedure for environment switching processing. [Figure 15] 10 is a flowchart illustrating an example of a procedure for viewpoint height change processing. [Figure 16] 10 is a flowchart illustrating an example of a procedure for viewpoint switching processing. [Figure 17] FIG. 10 is an explanatory diagram showing an example of a room selection screen. [Figure 18] FIG. 10 is an explanatory diagram showing an example of a walk-through screen. [Figure 19]FIG. 10 is an explanatory diagram showing an example of a color selection menu screen. [Figure 20] FIG. 10 is an explanatory diagram showing an example of an environment switching screen. [Figure 21] FIG. 10 is an explanatory diagram showing an example of a viewpoint height change screen. [Figure 22] FIG. 10 is an explanatory diagram showing an example of a layout screen. [Figure 23] FIG. 10 is an explanatory diagram showing an example of a building material selection screen. [Figure 24] FIG. 10 is an explanatory diagram showing an example of an information screen. [Figure 25] FIG. 10 is an explanatory diagram showing an example of an information screen. [Figure 26] 10A and 10B are explanatory diagrams showing the state of opening and closing the storage. [Figure 27] FIG. 10 is an explanatory diagram showing another example of the configuration of the VR system. [Figure 28] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a management server. [Figure 29] FIG. 2 is a block diagram illustrating an example of the hardware configuration of a user terminal. [Figure 30] FIG. 10 is an explanatory diagram illustrating an example of a user DB. [Figure 31] FIG. 10 is an explanatory diagram illustrating an example of a case DB. [Figure 32] FIG. 10 is an explanatory diagram illustrating an example of a history DB. [Figure 33] 10 is a flowchart illustrating an example of a procedure for a review process. [Figure 34] 10 is a flowchart illustrating an example of a procedure for a reference process. [Figure 35] FIG. 10 is an explanatory diagram showing another example of the walk-through screen. [Figure 36] FIG. 10 is an explanatory diagram showing an example of a training data DB. [Figure 37] FIG. 10 is an explanatory diagram illustrating an example of a learning model. [Figure 38] 10 is a flowchart illustrating an example of a procedure for a learning model generation process. [Figure 39] 10 is a flowchart illustrating an example of a procedure for theme selection processing. [Figure 40]FIG. 2 is a block diagram illustrating an example of functional units included in the image processing apparatus. [Figure 41] 10 is a flowchart illustrating an example of a procedure for a proximity display process. [Figure 42] FIG. 10 is an explanatory diagram showing an example of proximity display. [Figure 43] FIG. 10 is an explanatory diagram showing another example of a history DB. [Figure 44] 10 is a flowchart illustrating an example of a procedure for image display processing. [Figure 45] FIG. 10 is an explanatory diagram showing an example of a bookmark DB. [Figure 46] FIG. 10 is an explanatory diagram showing an example of a persona DB. [Figure 47] 10 is a flowchart illustrating an example of a procedure for a recommendation mark assignment process. [Figure 48] FIG. 2 is an explanatory diagram showing an example of a recommended furniture DB. [Figure 49] 10 is a flowchart illustrating an example of a procedure for related information processing. [Figure 50] 10 is a flowchart illustrating an example of a procedure for creating a moving image. [Figure 51] FIG. 10 is an explanatory diagram showing an example of a travel route. [Figure 52] FIG. 10 is an explanatory diagram illustrating an example of a template DB. [Figure 53] 10 is a flowchart illustrating an example of a procedure for creating a presentation board. [Figure 54] FIG. 10 is an explanatory diagram showing an example of a template. [Figure 55] FIG. 10 is an explanatory diagram showing another example of the configuration of the VR system. DETAILED DESCRIPTION OF THE INVENTION

[0009] (Embodiment 1) Hereinafter, an embodiment will be described with reference to the drawings. FIG. 1 is an explanatory diagram showing an example of the configuration of a VR system. VR is an abbreviation for Virtual Reality. The VR system 100 includes an image processing device 1. The image processing device 1 includes a controller 2 and a display 3. The image processing device 1 is configured as a desktop PC (Personal Computer), a notebook PC, a tablet computer, or the like. In order to display video images as realistic as possible, the image processing device 1 may be configured as a computer equipped with a graphics processing means having high 3D graphics processing capabilities, a so-called gaming PC. The image processing device 1 may also be configured as a multi-computer consisting of multiple computers, a virtual machine virtually constructed by software, or a quantum computer. .

[0010] The controller 2 is, for example, a VR controller. A user uses the controller 2 to give instructions to the image processing device 1. The display 3 is a display device including a liquid crystal display panel or the like. The display 3 is preferably a high-definition display such as a 4K display or an 8K display. The display 3 may also be a head-mounted display or VR goggles.

[0011] The image processing device 1 includes a control unit 11, a main memory unit 12, an auxiliary memory unit 13, an input I / F 15, a display I / F 16, a communication unit 17, and a reading unit 18. The control unit 11, the main memory unit 12, the auxiliary memory unit 13, the input I / F 15, the display I / F 16, the communication unit 17, and the reading unit 18 are connected by a bus B.

[0012] The control unit 11 has one or more arithmetic processing devices such as a central processing unit (CPU), a micro-processing unit (MPU), a graphics processing unit (GPU), etc. The control unit 11 reads out and executes a control program 1P stored in the auxiliary storage unit 13 to realize various functional units.

[0013] The main memory unit 12 is a static random access memory (SRAM), a dynamic random access memory (DRAM), a flash memory, etc. The main memory unit 12 mainly stores data in the control unit 11. temporarily stores data necessary to execute arithmetic processing.

[0014] The auxiliary storage unit 13 is a hard disk or a solid state drive (SSD), and the control unit The image processing device 1 includes an auxiliary storage unit 11 that stores a control program 1P and various DBs (Databases) required for executing the processing. The auxiliary storage unit 13 stores a theme DB 131, a recommended combination DB 132, a texture DB 133, a room DB 134, a vertex DB 135, an edge DB 136, a face DB 137, and an object DB 138. The auxiliary storage unit 13 may be an external storage device connected to the image processing device 1.

[0015] Input I / F15 communicates using communication standards such as PS / 2 and Bluetooth (registered trademark). The input I / F 15 communicates with the controller 2. The input I / F 15 may also communicate with a keyboard or mouse (not shown).

[0016] The display I / F 16 is a display output port conforming to a communication standard such as HDMI (registered trademark) (High-Definition Multimedia Interface), DisplayPort, Thunderbolt, etc. The display I / F 16 outputs an image signal to the display 3.

[0017] The communication unit 17 communicates with other computers via the network. The control unit 11 may use the communication unit 17 to download the control program 1P from other computers via the network N and store it in the auxiliary storage unit 13.

[0018] The reading unit 18 reads the portable storage medium 1a including a CD (Compact Disc)-ROM and a DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 18 and store it in the auxiliary storage unit 13. The control unit 11 may also read the control program 1P from the semiconductor memory 1b.

[0019] A server computer with higher performance may perform part of the processing performed by the image processing device 1, such as rendering. The image processing device 1 receives an image created by rendering from the server computer and displays it on the display 3.

[0020] In the following description, interior includes interior decoration and furnishings. Interior components literally refer to the individual elements that make up an interior, such as the basic elements that make up a room, such as flooring, wall materials, and ceiling materials, fixtures and decorative materials such as doors and windows, furniture, system kitchens, curtains, and lighting fixtures. Wallpaper materials refer to interior finishing materials such as wall materials and ceiling materials, and finishing materials for built-in furniture. Built-in furniture refers to built-in furniture that is made to match the interior. In this embodiment, building materials refer to flooring materials and wallpaper materials.

[0021] Next, the database (DB) used in the VR system 100 will be described. Figure 2 is an explanatory diagram showing an example of a theme DB. The theme DB 131 stores information about interior themes. Theme is a word that succinctly expresses the idea used to create a sense of unity in the interior design. The theme may also be expressed as style or taste. The theme DB 131 includes a theme ID column, a theme name column, a summary column, and a description column. The theme ID column stores a theme ID that uniquely identifies a theme. The theme name column stores the theme name. The summary column stores a summary description of the theme. The description column stores a detailed description of the theme.

[0022] Figure 3 is an explanatory diagram showing an example of a recommended combination DB. The recommended combination DB 132 stores recommended combinations of flooring materials and cloth materials for each theme. The recommended combination DB 132 stores a theme ID column, a flooring material column, and a cloth material column. The theme ID column stores theme IDs. The flooring material column and cloth material column store IDs of flooring materials and cloth materials recommended for combination, for example, product models.

[0023] FIG. 4 is an explanatory diagram showing an example of a texture DB. The texture DB 133 stores texture images. A texture is a repeating pattern applied to flooring or wallpaper materials. A texture image is an image of the repeating texture pattern. The texture DB 133 includes a texture ID column, a model column, a group column, an image column, and an application column. The texture ID column stores a texture ID that can uniquely identify a texture. The model column stores the product model of the flooring or wallpaper material corresponding to the texture. The group column stores the group to which the texture belongs. The image column stores the texture image. The image column may store the file name of the texture image, and the file may be stored in the auxiliary memory unit 13. The application column stores the target to which the texture is applied. The texture image may include not only the patterns of flooring or wallpaper materials, but also general textures in computer graphics, that is, images that represent the patterns on the surface of objects.

[0024] FIG. 5 is an explanatory diagram showing an example of a room DB. The room DB 134 stores information about rooms expressed as virtual three-dimensional spaces. The room DB 134 includes a room ID column, a name column, a theme column, a floor column, a wall column, a ceiling column, a fixture column, and a virtual camera column. The room ID column stores a room ID that can uniquely identify a room. The name column stores the name of the room. The name can be, for example, "A type," "B type," or "C type." type," "C type," etc. The theme column stores the theme ID of the design theme to be applied at the time of initial display. The floor column stores information about the floors that make up the room. The floor information includes, for example, coordinate values ​​indicating the position in virtual three-dimensional space of the surface representing the floor, and the texture ID of the texture to be applied as the floor material. The wall column stores information about the walls that make up the room. The wall information includes, for example, coordinate values ​​indicating the position in virtual three-dimensional space of the surface representing the wall, and the texture ID of the texture to be applied as the wall material. The ceiling column stores information about the ceiling that makes up the room. The ceiling information includes, for example, coordinate values ​​indicating the position in virtual three-dimensional space of the surface representing the ceiling, and the texture ID of the texture to be applied as a finishing material. The ceiling column also stores information about the surfaces to represent a dropped ceiling. The fixture column stores identification information for the fixtures that make up the room. The virtual camera column stores the position, shooting direction, etc. of a virtual camera that is set in advance in the virtual three-dimensional space. An image of the virtual three-dimensional space captured by the virtual camera becomes an output image from the image processing device 1. The virtual camera corresponds to the virtual viewpoint of the user.

[0025] FIG. 6 is an explanatory diagram showing an example of the vertex DB. The vertex DB 135 stores vertices that represent a virtual three-dimensional space. The vertex DB 135 includes a vertex ID string, an X coordinate string, a Y coordinate string, and a Z coordinate string. The vertex ID string stores a vertex ID that can uniquely identify a vertex. The X coordinate string stores the X coordinate of the vertex. The Y coordinate string stores the Y coordinate of the vertex. The Z coordinate string stores the Z coordinate of the vertex.

[0026] FIG. 7 is an explanatory diagram showing an example of an edge DB. The edge DB 136 stores edges that represent a virtual three-dimensional space. The edge DB 136 includes an edge ID column, a start column, and an end column. The edge ID column stores an edge ID that can uniquely identify an edge. The start column stores the vertex ID of the vertex that indicates the start point of the edge. The end column stores the vertex ID of the vertex that indicates the end point of the edge. The edges are directed edges with a set direction.

[0027] FIG. 8 is an explanatory diagram showing an example of the face DB. The face DB 137 stores faces that represent a virtual three-dimensional space. The face DB 137 includes a face ID column, an edge 1 column, an edge 2 column, an edge 3 column, an edge 4 column, an edge 5 column, and an edge 6 column. The face ID column stores a face ID that can uniquely identify a face. The edge 1 column, edge 2 column, edge 3 column, edge 4 column, edge 5 column, and edge 6 column each store the edge IDs of the edges that make up the face. Because a face can be composed of at least three edges, the edge 4 column to edge 6 column may be set to NULL, indicating that there are no values. Since the edges have a predetermined direction, tracing the edges that make up the face will form a complete circle around the perimeter. In other words, the vertices that make up the face are components of two edges, and are the starting point of one edge and the end point of the other edge. The direction of the perimeter determines the front and back of the face. When looking at a face, the perimeter that runs counterclockwise is the front, and the perimeter that runs clockwise is the back.

[0028] FIG. 9 is an explanatory diagram showing an example of an object DB. The object DB 138 stores information about objects placed in a room. Objects include fixtures such as doors and windows. The object DB 138 includes an object ID column, a type column, a name column, a definition column, a cloth material column, a behavior column, and an icon column. The object ID column stores an object ID that can uniquely identify an object. The type column stores the type of object. The type may be, for example, fixture, furniture, built-in furniture, or other. The name column stores the name of the object. The definition column stores the definition of the 3D shape. For example, it stores the surface IDs of multiple surfaces that represent an object. The cloth material column stores information about surfaces to which cloth material is applied. The definition column includes information about the texture to be applied to surfaces that are not the target to apply cloth material. The behavior column stores methods (functions) for changing the state of an object. For example, open() is a method for opening a door or opening a closet door. close() is a method for closing a door or closing a closet. This is a method for closing the Z door, etc. The icon column stores information about the icons used for operations to be attached to objects. The icons are, for example, a change icon (selection icon) or an information icon. The change icon is an icon that opens a screen for changing the texture of the object. The information icon is an icon that opens an information screen about the object. If no icon is attached to the object, the icon column stores "none."

[0029] As shown above, when defining a virtual 3D space, first a vertex is defined. An edge is defined from two vertices. A face is defined from multiple edges. An object is defined from multiple faces. This is an example of a definition method called a surface model or surface modeling. The definition method is not limited to this. A surface model with a triangle as the smallest element may also be used, or a wireframe model, solid model, etc. may also be adopted.

[0030] Next, we will discuss the information processing performed by the image processing device 1. FIG. 10 is a flowchart showing an example of the procedure for the main processing. The main processing is the main processing performed by the image processing device 1. The control unit 11 of the image processing device 1 displays a top screen on the display 3 (step S1). The control unit 11 displays a room select screen on the display 3 (step S2). The user selects a room on the room select screen. Each room has an interior design with a different theme. The control unit 11 generates a walk-through screen according to the room selected by the user. (Step S3). When the user performs an operation on the walk-through screen, the control unit 11 determines whether the operation is a menu selection (Step S4). If the control unit 11 determines that the operation is not a menu selection (NO in Step S4), it executes processing on the walk-through screen (screen processing) (Step S5) and returns the processing to Step S4. If the control unit 11 determines that the operation is a menu selection (YES in Step S4), it determines whether or not a room selection has been selected (Step S6). If the control unit 11 determines that a room selection has been selected (YES in Step S6), it returns the processing to Step S2. If the control unit 11 determines that a room selection has not been selected (NO in Step S6), it determines whether or not a top screen has been selected (Step S7). If the control unit 11 determines that a top screen has been selected (YES in Step S7), it returns the processing to Step S1. If the control unit 11 determines that a top screen has not been selected (NO in Step S7), it determines whether or not a change in display settings has been selected (Step S8). If the control unit 11 determines that a change in display settings has been selected (YES in step S8), it updates the screen in accordance with the change in settings (step S9) and returns the process to step S4. If the control unit 11 determines that a change in display settings has not been selected (NO in step S8), it determines whether or not end has been selected (step S10). If the control unit 11 determines that end has not been selected (NO in step S10), it displays a screen in accordance with the selected menu (step S11) and returns the process to step S4. If the control unit 11 determines that end has been selected (YES in step S10), it ends the main process.

[0031] FIG. 11 is a flowchart showing an example of the procedure for the walk-through screen generation process. The walk-through screen generation process corresponds to step S3 in FIG. 10. The control unit 11 of the image processing device 1 acquires information about the room selected by the user on the room select screen from the room DB 134 (step S21). The control unit 11 acquires recommended combination information from the recommended combination DB 132 using the theme ID included in the room information (step S22). The control unit 11 acquires the texture of the floor material and the texture of the cloth material from the texture DB 133 based on the recommended combination information (step S23). The control unit 11 acquires information about objects included in the virtual three-dimensional space based on the contents of the floor row, wall row, ceiling row, and fittings row in the room DB 134 (step S24). The control unit 11 generates a virtual three-dimensional space based on the object information (step S25). At this time, texture images acquired from the texture DB 133 are pasted on the floor, wall, closet, etc. The control unit 11 acquires images of the virtual three-dimensional space based on the contents of the virtual camera sequence in the room DB 134 (step S26). The control unit 11 generates a walk-through screen including the acquired images (step S27). The control unit 11 outputs the generated walk-through screen (step S28) and returns the process to the caller.

[0032] FIG. 12 is a flowchart showing an example of the procedure for screen processing. The screen processing corresponds to step S5 in FIG. 10. The control unit 11 of the image processing device 1 detects an object at a position specified by the user on the walk-through screen (step S41). The control unit 11 determines whether the detected object is an icon (step S42). If the control unit 11 determines that the detected object is an icon (YES in step S42), it determines whether the icon is a change icon (step S43). If the control unit 11 determines that the icon is a change icon (YES in step S43), it displays a selection screen for changing the flooring material or wallpaper material (step S44). If the object is a floor surface, the control unit 11 displays a flooring material selection screen. If the object is a wall, closet, or the like, the control unit 11 displays a wallpaper material selection screen. The user selects a flooring material or wallpaper material, or selects "close." The control unit 11 determines whether the user's operation is an instruction to change the flooring material or floor material (step S45). If the control unit 11 determines that the instruction is to change the floor material or flooring material (YES in step S45), it changes the texture of the floor or wall or the like in accordance with the instruction (step S46). The control unit 11 then captures an image of the virtual camera after the change. The control unit 11 acquires the image (step S47). The control unit 11 updates the walk-through screen based on the acquired image (step S48). The control unit 11 outputs the updated walk-through screen (step S49) and returns the process to the caller. If the control unit 11 determines that the icon is not a change icon (NO in step S45), the control unit 11 returns the process to the caller. If the control unit 11 determines that the icon is not a change icon (NO in step S43), the control unit 11 acquires information for displaying an information screen (step S50). For example, if an information icon attached to furniture is selected, information about the furniture is acquired. The control unit 11 generates a screen for displaying the acquired information (step S51). The control unit 11 outputs the generated screen (step S52) and returns the process to the caller. If the control unit 11 determines that the detected object is not an icon (NO in step S42), the control unit 11 determines whether or not a viewpoint change has occurred (step S53). If the control unit 11 determines that a viewpoint change has occurred (YES in step S53), the control unit 11 performs a viewpoint change (step S54). The control unit 11 switches the virtual camera or changes the position of the virtual camera. The control unit 11 executes step S47 and subsequent steps based on the changed virtual camera. If the control unit 11 determines that the viewpoint has not been changed (NO in step S53), it returns the process to the caller. The viewpoint change in step S54 allows the user to freely change the viewpoint using the controller 2. The user feels as if they are walking around freely in a room expressed as a virtual three-dimensional space.

[0033] FIG. 13 is a flowchart showing an example of the procedure for color selection processing. The color selection processing is one of the processes corresponding to the screen update (step S9) in FIG. 10. The control unit 11 creates a menu screen (step S61). The menu screen displays multiple recommended combinations of flooring materials and wallpaper materials, and is created based on the recommended combination DB 132. It is desirable to display only combinations that correspond to the theme. The control unit 11 performs display settings for the currently selected combination on the menu screen so that the user can see it (step S62). The control unit 11 displays the created menu screen (step S63). The user selects the changed combination on the menu screen or selects "Back." The control unit 11 determines whether the user has selected another combination (step S64). If the control unit 11 determines that the user has selected another combination (YES in step S64), it changes the texture of the floor and walls according to the selected combination (step S65). The control unit 11 acquires an image after the texture change (step S66). The control unit 11 updates the screen based on the acquired image (step S67). The control unit 11 outputs the updated screen (step S68) and returns the process to step S64. If the control unit 11 determines that the user has not selected another combination (NO in step S64), it erases the menu screen (step S69) and ends the process.

[0034] FIG. 14 is a flowchart showing an example of the procedure for the environment switching process. The environment switching process is one of the processes corresponding to the screen update (step S9) in FIG. 10. The control unit 11 displays a menu (step S81). The menu may be, for example, "day / night switching," "open / close storage," or "display furniture." The user changes one of the menus or selects "back." The control unit 11 determines whether the day / night switching has been operated (step S82). If the control unit 11 determines that the day / night switching has been operated (YES in step S82), it changes the light source (virtual light source) (step S83). The day / night switching switches the time period in the virtual three-dimensional space between daytime and nighttime. Daytime is assumed to be a sunny day. In the daytime, the sun is used as the light source, and the lighting is expressed as if sunlight is pouring in through a window. In the nighttime, the lighting is expressed as if the light source is a light source installed on the ceiling or wall. The control unit 11 acquires an image after the light source has been changed (step S84). The control unit 11 updates the screen based on the acquired image (step S85). Note that the day / night switching may switch the view seen from the window. The control unit 11 outputs the updated screen (step S86) and returns the process to step S82. If the control unit 11 determines that the day / night switching has not been operated (NO in step S82), it determines whether the storage open / close operation has been performed (step S87). If the control unit 11 determines that the storage open / close operation has been performed (YES in step S87), it The control unit 11 changes the open / closed state of a door of a closet or the like (step S88). The control unit 11 changes the state by executing a method stored in the behavior column of the object DB 138. In addition to the closet, the drawers of a desk or dresser may also be opened or closed. The open state may be multi-stage, such as 25% open, 50% open, and 100% open. The control unit 11 executes step S84 and subsequent steps. If the control unit 11 determines that the storage open / close operation has not been performed (NO in step S87), it determines whether the furniture display has been operated (step S89). If the control unit 11 determines that the furniture display has been operated (YES in step S89), it changes whether the furniture is displayed or not (step S90). The control unit 11 executes step S84 and subsequent steps. If the control unit 11 determines that the furniture display has not been operated (NO in step S89), it erases the menu (step S91) and ends the process.

[0035] FIG. 15 is a flowchart showing an example of the procedure for viewpoint height change processing. The viewpoint height change processing is one of the processes corresponding to the screen update (step S9) in FIG. 10. The viewpoint height change processing is a process that changes only the height of the viewpoint without changing the position in a plane parallel to the floor. Setting the viewpoint height to a value that takes the user's height into consideration makes it possible to output a more realistic image. Furthermore, setting the viewpoint height low makes it possible to see the room from a child's perspective. The control unit 11 displays a height change menu (step S101). The menu displays multiple different heights, with the current setting selected. The user changes the height or selects "Back." The control unit 11 determines whether the height has been changed (step S102). If the control unit 11 determines that the height has been changed (YES in step S102), it changes the viewpoint (step S103). The control unit 11 changes the height of the virtual camera, which corresponds to the viewpoint. The control unit 11 acquires the changed image (step S104). The control unit 11 updates the screen based on the acquired image (step S105). The control unit 11 outputs the updated screen (step S106) and returns the process to step S102. If the control unit 11 determines that the height has not been changed (NO in step S102), it erases the menu (step S107) and ends the process.

[0036] FIG. 16 is a flowchart showing an example of the procedure for viewpoint switching processing. The viewpoint switching processing is one of the processes corresponding to the screen update (step S9) in FIG. 10. The viewpoint switching processing is a process for changing the position of the viewpoint in a plane parallel to the floor without changing the height of the viewpoint. The control unit 11 of the image processing device 1 displays a layout screen (step S121). The layout screen is a plan view showing the shape of a room and the positions of fixtures and fittings in the room. The layout screen shows the positions of multiple virtual cameras that can be specified as viewpoints using icons. The display mode of the currently selected virtual camera is made different from the display modes of the other virtual cameras. The user changes the virtual camera or selects "Close." The control unit 11 determines whether the selection state has changed (step S122). If the control unit 11 determines that the selection state has changed (YES in step S122), it changes the viewpoint (step S123). The control unit 11 changes the virtual camera corresponding to the viewpoint. The control unit 11 acquires an image after the change (step S124). The control unit 11 updates the screen based on the acquired image (step S125). The control unit 11 outputs the updated screen (step S126) and returns the process to step S122. If the control unit 11 determines that the selection state has not been changed (NO in step S122), it erases the layout screen (step S127) and ends the process.

[0037] Next, examples of screens used in the above process will be shown. Fig. 17 is an explanatory diagram showing an example of a room select screen. The room select screen d01 is a screen for selecting a room to display. The room select screen d01 includes a top button d011 and a plurality of room select buttons d012. When the top button d011 is selected, the screen transitions to the top screen (not shown). When the room select button d012 is selected, the screen transitions to a walk-through screen that displays the selected room. In the example shown in Fig. 17, each room has an interior design with a different theme, and the room select buttons d012 are used to select the name of the corresponding theme (Calm, Flow, A, etc.). The themes displayed are: The control unit 11 generates the theme based on the theme information stored in the theme DB 131. When setting for a specific apartment, the room select button d012 may be provided with an A-type, B-type, or the like indicating the type of room.

[0038] FIG. 18 is an explanatory diagram showing an example of a walk-through screen. The walk-through screen d02 includes a top button d021, a room select button d022, a color select button d023, an environment switch button d024, a viewpoint height switch button d025, a VR mode button d026, a print button d027, a floor plan button d028, a close button d029, a settings display d02A, an information icon d02B, and a change icon d02C. Selecting the top button d021 transitions to the top screen. Selecting the room select button d022 returns to the room select screen d01. Selecting the color select button d023 launches a color select process that allows you to change the flooring and wallpaper materials all at once. Selecting the environment switch button d024 launches an environment switch process that allows you to switch between day and night, open and close storage, and so on. Selecting the viewpoint height switch button d025 launches a viewpoint switch process that allows you to change the viewpoint height. When the VR mode button d026 is selected, an image for VR goggles is output from the display I / F 16. When the print button d027 is selected, a hard copy of the screen is sent to the printer. When the floor plan button d028 is selected, the viewpoint switching process is started and the layout screen is overlaid. When the close button d029 is selected, the top button d021 to the close button d029 are hidden. The settings display d02A displays various settings. The settings display d02A includes a theme d02A1, combination d02A2, viewpoint height d02A3, selected camera icon d02A4, and camera icon d02A5. The theme d02A1 displays the name of the selected theme. The combination d02A2 displays the type of flooring and wallpaper material. The viewpoint height d02A3 displays the height of the selected viewpoint. The selected camera icon d02A4 indicates the position of the virtual camera selected as the viewpoint on the room layout diagram. The camera icon d02A5 indicates the position of the selectable virtual camera on the room layout diagram. When the information icon d02B is selected, an information screen that displays information about the item to which the icon is attached is overlaid. When the change icon d02C is selected, screen processing is started and a selection screen for changing the texture attached to the item to which the icon is attached is overlaid.The information icon d02B and the change icon d02C are arranged by the control unit 11 according to the values ​​of the icon column in the object DB 138.

[0039] FIG. 19 is an explanatory diagram showing an example of a color selection menu screen. The color selection menu screen d03 (hereinafter simply referred to as the "menu screen d03") displays multiple recommended combinations (options) of flooring and wallpaper materials, and is displayed during the color selection process. The menu screen d03 includes thumbnails d031, combination types d032, selection rectangles d033, check marks d034, and a back button d035. The thumbnails d031 show room design images when each recommended combination of flooring and wallpaper materials is applied. The combination types d032 indicate the product models of the combined flooring and wallpaper materials. The selection rectangle d033 indicates the currently selected combination. The check mark d034 indicates the combination currently applied to the walk-through screen d02. Selecting the back button d035 clears the menu screen d03. It is possible to change the flooring and wallpaper materials by selecting a combination other than the currently selected combination from the multiple recommended combinations displayed on the menu screen d03. The control unit 11 performs the change process for changing the texture of flooring materials based on the surface data stored in the floor column of the room DB 134, and for changing the texture of walls based on the surface data stored in the wall column of the room DB 134. For fixtures and furniture, the control unit 11 checks whether there is a change by referencing the value of the cross material column of the object DB, and changes the texture of the target surface based on the value of the definition column.

[0040] FIG. 20 is an explanatory diagram showing an example of the environment switching screen. The environment switching screen d04 is an environment switching menu. The environment switching screen d04 is a screen for displaying the furniture and is displayed during the environment switching process. The environment switching screen d04 includes a furniture area d041, a day / night area d042, a storage area d043, a selection rectangle d044, and a back button d045. The furniture area d041 can switch between displaying and hiding furniture. When switching between displaying and hiding furniture, the control unit 11 determines whether each object placed in the room is furniture based on the value of the type column in the object DB 138, and switches between displaying and hiding objects determined to be furniture. The day / night area d042 can switch the lighting environment of the room between daytime and nighttime. The storage area d043 can open and close the doors of storage furniture such as a closet. The selection rectangle d044 indicates which of the furniture area d041, the day / night area d042, and the storage area d043 is the operation target (active). When the back button d045 is selected, the environment switching screen d04 is cleared.

[0041] FIG. 21 is an explanatory diagram showing an example of a viewpoint height change screen. The viewpoint height change screen d05 is a screen that displays selectable viewpoint heights and is displayed during the viewpoint height change process. The viewpoint height change screen d05 includes a selection rectangle d051, a check mark d052, and a back button d053. The selection rectangle d051 indicates the currently selected viewpoint height. The check mark d052 indicates the viewpoint height currently applied to the walk-through screen d02. When the back button d053 is selected, the viewpoint height change screen d05 is cleared. FIG. 21A is an example of a walk-through screen when the viewpoint height is 150 cm. FIG. 21B is an example of a walk-through screen when the viewpoint height is 60 cm. Note that the selectable viewpoint heights are values ​​common to multiple rooms and stored in the auxiliary storage unit 13. The maximum viewpoint height may be a value according to the ceiling height of the room, stored in the room DB 134, etc.

[0042] The viewpoint heights shown in FIG. 21 are an example. It is desirable to provide at least three viewpoint heights. The first viewpoint height is approximately 130 cm to 190 cm and is intended for an adult's viewpoint. The second viewpoint height is approximately 60 cm to 130 cm and is intended for a child's viewpoint. The third viewpoint height is approximately 10 cm to 60 cm and is a viewpoint from which the texture of materials, particularly flooring, can be more realistically confirmed. The third viewpoint height may be 50 cm or less, preferably 40 cm or less, or may be 15 cm or more, preferably 20 cm or more. Providing at least three viewpoint heights allows users to experience interior design according to their height and further allows users to experience the texture of materials in real time. The viewpoint heights for an adult, a child, and a viewpoint for checking materials may each be set at one or more viewpoint heights. Furthermore, the viewpoint heights may be continuously set using a track bar. In this case, it is desirable to display rough value ranges corresponding to the adult's viewpoint, the child's viewpoint, and the viewpoint for checking the material together with a track bar. Furthermore, when switching from the first viewpoint height or the second viewpoint height to the third viewpoint height, information about the flooring material, such as the name and model number, may be automatically displayed. When switching from the third viewpoint height to the first viewpoint height or the second viewpoint height, the display of information about the flooring material, such as the name and model number, may be automatically hidden.

[0043] FIG. 22 is an explanatory diagram showing an example of a layout screen. FIG. 22A is an example of a layout screen showing the position of the virtual camera before the change. FIG. 22B is an example of a layout screen showing the position of the virtual camera after the change. The layout screen d06 is a diagram showing the positions of the virtual camera that can be selected on the floor plan of the room and the position of the virtual camera that is currently selected as the viewpoint. The layout screen d06 includes a position mark d061, a viewpoint display d062, a field of view display d063, and a close button d064. The position mark d061 indicates the positions of the virtual cameras that can be selected. The viewpoint display d062 indicates the selected virtual camera. The position of the virtual camera can be changed by selecting the position mark d061 with a mouse click or the like. The field of view display d063 indicates the field of view corresponding to the display on the walk-through screen d02. Selecting the close button d064 closes the layout screen d06.

[0044] FIG. 23 is an explanatory diagram showing an example of a building material selection screen. The building material selection screen d07 is a screen for selecting a floor material or a cladding material. This is a selection screen for changing loss materials, and is displayed when a change icon is selected in screen processing. The building material selection screen d07 includes a building material display d071, a selection rectangle d072, a tone switch d073, a tone switch d074, and a close button d075. The building material display d071 shows the texture and type of each building material. If the selected change icon is attached to a floor, the building material display d071 displays the floor material. If the selected change icon is attached to a wall, closet, etc., the building material display d071 displays the wallpaper material. The selection rectangle d072 shows the currently applied building material. Selecting tone switch d073 or tone switch d074 switches the tone of the building material displayed in the building material display d071. The building materials are grouped according to color tone. The control unit 11 performs this grouping using the values ​​in the group column of the texture DB 133. The tones are, for example, white tones, light tones, gray tones, medium tones, dark tones, or deep tones. When the close button d075 is selected, the building material selection screen d07 is closed. Note that the building materials displayed in the building material display d071 may be different for each selected theme.

[0045] 24 and 25 are explanatory diagrams showing examples of information screens. Information screen d08 is a screen showing information about a commercial material, and is displayed when an information icon is selected in screen processing. FIG. 24 is an example of information screen d08 that is displayed when an information icon attached to a building material is selected. FIG. 25 is an example of information screen d08 that is displayed when an information icon attached to furniture or the like is selected. Information about the commercial material is stored in auxiliary memory unit 13, and is read and displayed by control unit 11 as appropriate.

[0046] Fig. 26 is an explanatory diagram showing the opening and closing of a storage furniture. Fig. 26A is a screen when the door of the storage furniture is closed. Fig. 26B is a screen when the door of the storage furniture is open. The opening and closing of the door of the storage furniture is performed by the environment switching process described above.

[0047] In this embodiment, multiple recommended combinations of flooring and wallpaper materials are defined for each theme, allowing for a variety of interior designs for each theme. Computer graphics using a virtual three-dimensional space allows the user to confirm the resulting design with a sense of reality. Furthermore, by changing the recommended combinations, the user can make detailed changes to the design without changing the theme.

[0048] The lighting environment of the room can be switched between daytime and nighttime, allowing users to see how the design is perceived in different lighting environments. The open / closed state of built-in furniture such as closets can be confirmed as images, allowing users to determine whether the position of the furniture is appropriate. The building materials can be changed on an interior component-by-component basis, allowing users to change parts of the design according to their preferences.

[0049] After the user selects a room on the room selection screen, an introduction screen introducing the theme adopted in the design of the selected room may be displayed before the walk-through screen is generated and displayed. This introduction screen allows the user to propose a spatial concept that matches their lifestyle.

[0050] In this embodiment, the purpose is to support interior design by real estate developers and architects who design houses, and interior designers who are commissioned by end users who purchase houses, but the present invention is not limited to this. It may also be used by building material manufacturers to realistically represent the installation image of a wide variety of products in living spaces.

[0051] (Embodiment 2) This embodiment relates to an embodiment in which the contents of an interior design set by a first user can be referenced by a second user by storing the operation history of the image processing device 1. In this embodiment, the first user is an interior designer and the second user is an end user, but this is not limiting. For example, the first user may be an interior designer and the second user may be a real estate developer or an architect, or the first user may be a real estate developer or an architect and the second user may be an end user.

[0052] 27 is an explanatory diagram showing another example of the configuration of a VR system. The VR system 100 includes an image processing device 1, a management server 4, and a user terminal 5. The image processing device 1, the management server 4, and the user terminal 5 are connected to each other via a network N so that they can communicate with each other.

[0053] The image processing device 1 is a desktop PC (Personal Computer) similar to the first embodiment. The image processing device 1 is configured as a laptop PC, a tablet computer, etc. The image processing device 1 is used by an interior designer or the like who designs interiors.

[0054] The management server 4 is configured with a server computer, a workstation, etc. The management server 4 may be configured with a multi-computer consisting of multiple computers, a virtual machine virtually constructed by software, or a quantum computer. The management server 4 manages end-user information and the operation history of the image processing device 1.

[0055] The user terminal 5 is configured as a notebook PC, a tablet computer, or the like. The user terminal 5 is used by an end user. Although only one image processing device 1 and one user terminal 5 are shown in FIG. 27, there may be multiple devices. A system including the management server 4 and the image processing device 1 is referred to as a first system, and a system including the management server 4 and the user terminal 5 is referred to as a second system.

[0056] 28 is a block diagram showing an example of the hardware configuration of the management server 4. The management server 4 includes a control unit 41, a main memory unit 42, an auxiliary memory unit 43, and a communication unit 45. The control unit 41, the main memory unit 42, the auxiliary memory unit 43, and the communication unit 45 are connected by a bus B.

[0057] The control unit 41 has one or more arithmetic processing units such as a CPU, an MPU, a GPU, etc. The control unit 41 functions as the management server 4 by reading and executing a control program 4P stored in the auxiliary storage unit 43.

[0058] The main memory unit 42 is an SRAM, a DRAM, a flash memory, etc. The main memory unit 42 mainly temporarily stores data required for the control unit 41 to execute arithmetic processing.

[0059] The auxiliary storage unit 43 is a hard disk or SSD, etc., and stores the control program 4P and various DBs required for the control unit 41 to execute processing. The auxiliary storage unit 43 stores a user DB 431, a case DB 432, a history DB 433, and a training data DB 434. The auxiliary storage unit 43 also stores a learning model 441. The auxiliary storage unit 43 may be an external storage device connected to the management server 4. The various DBs and the like stored in the auxiliary storage unit 43 may be stored in a database server or cloud storage different from the management server 4. Note that the training data DB 434 and the learning model 441 are not essential components in this embodiment.

[0060] The communication unit 45 communicates with the image processing device 1 and the user terminal 5 via the network. The control unit 41 may use the communication unit 45 to download the control program 4P from another computer via the network N and store it in the auxiliary storage unit 43.

[0061] 29 is a block diagram showing an example of the hardware configuration of the user terminal. The computer 100 includes a control unit 51, a main memory unit 52, an auxiliary memory unit 53, a communication unit 54, an input unit 55, and a display unit 56. Each component is connected by a bus B.

[0062] The control unit 51 has one or more arithmetic processing units such as a CPU, an MPU, a GPU, etc. The control unit 51 functions as the user terminal 5 by reading and executing a control program 5P stored in the auxiliary storage unit 53.

[0063] The main memory unit 52 is an SRAM, a DRAM, a flash memory, etc. The main memory unit 52 mainly temporarily stores data required for the control unit 51 to execute arithmetic processing.

[0064] The auxiliary storage unit 53 is a hard disk or an SSD, etc., and stores the control program 5P and various DBs required for the control unit 51 to execute processing. The various DBs, etc. stored in the auxiliary storage unit 53 may be stored in a database server or cloud storage different from the user terminal 5.

[0065] The communication unit 54 communicates with the management server 4 via the network. The control unit 51 may use the communication unit 54 to download the control program 5P from another computer via the network N and store it in the auxiliary storage unit 53.

[0066] The input unit 55 is a keyboard and a mouse. The display unit 56 includes a liquid crystal display panel or the like. The display unit 56 displays a virtual space image acquired from the virtual three-dimensional space. The display unit 56 may also be a touch panel display integrated with the input unit 55. The user terminal 5 may also display on an external display device.

[0067] Next, we will explain the databases managed by the management server 4. Fig. 30 is an explanatory diagram showing an example of a user DB. The user DB 431 stores information about end users. The user DB 431 includes a user ID column, an age column, a gender column, an occupation column, a number of family members column, and an annual income column. The user ID column stores a user ID that can uniquely identify an end user. The age column stores the age of the end user. The gender column stores the gender of the end user. The occupation column stores the occupation of the end user. The number of family members column stores the number of people living in a room including the end user. The annual income column stores the annual income of the end user. The user DB 431 may also store other attributes that are useful for interior design, such as hobbies and favorite colors.

[0068] The following data may also be stored in the user DB 431: information on the location and property selected by a real estate developer, architect, or end user, and information that reveals the end user's preferences and lifestyle. The former information can be used to link the location with the exterior of the property and make proposals. The latter information can be used to make more appealing proposals to the end user. Examples of the latter information include favorite travel destinations (countries, places to stay), lodgings stayed at while traveling, books and magazines (fashion magazines, etc.) that are usually read, hobbies, places to buy clothes, favorite brands, etc.

[0069] Figure 31 is an explanatory diagram showing an example of a case DB. The case DB 432 stores each interior design case. The case DB 432 includes a case ID column, a person in charge ID column, a user ID column, a name column, and a status column. The case ID column stores a case ID that can uniquely identify the case. The person in charge ID column stores the ID of the interior design person, interior designer, etc. The user ID column stores the ID of the end user. The name column stores the name of the case. The name is set appropriately by the person in charge. The status column stores the status of the case. Examples of statuses include under consideration, proposed, and closed. Under consideration is a state in which the design is being considered. Proposed is a state in which the end user approves the design content and is considering whether to place an order. Closed is a state in which the design has been solidified and the end user has signed a contract.

[0070] FIG. 32 is an explanatory diagram showing an example of a history DB. The history DB (history storage unit) 433 stores the operation history of the image processing device 1. By referring to the operation history, it is possible to understand the progress and results of the interior design. The history DB 433 includes a history ID column, a date and time column, a user ID column, a person in charge ID column, a theme ID column, a flooring column, a wallpaper material column, and a change column. The history ID column records a history ID that can uniquely identify the history. The date and time column stores the date and time when the history was stored. The user ID column stores the user ID of the end user. The person in charge ID column stores the ID of the person in charge of design. The theme ID column stores the theme ID of the selected theme. The flooring column stores the type of flooring material selected. The wallpaper material column stores the type of wallpaper selected. The change column stores the changes made when part of the design is changed. For example, in the initial setting, the wallpaper is uniform throughout the room, but the wallpaper and the exterior of the kitchen counter in the living room and kitchen may have different designs. In this case, the change column stores the surface ID indicating the kitchen wall, the object ID indicating the kitchen counter, and the type of cloth material after the change.

[0071] Next, the information processing performed by the VR system 100 will be described. FIG. 33 is a flowchart showing an example of the procedure of the review process. The review process is a process executed when a design engineer reviews an interior design. The engineer operates the image processing device 1 to log in to the VR system 100. The control unit 11 of the image processing device 1 sends a login request to the management server 4 (step S131). The control unit 41 of the management server 4 performs login processing (step S132). The login processing is, for example, authentication using an ID and password. The control unit 41 searches the case DB 432, creates a list of cases linked to the engineer, and sends it to the image processing device 1 (step S133). The control unit 11 of the image processing device 1 displays the received list on the display 3 (step S134). The engineer selects a case when reviewing the design of a case included in the list, or instructs "new" when reviewing a new case. At that time, the engineer selects whether the review is for a specific end user or for a review for which an end user has not been determined. The control unit 11 determines whether the case is new (step S135). If the control unit 11 determines that the case is new (YES in step S135), it determines whether or not to register the end user information (step S136). If the control unit 11 determines that the end user information is to be registered (YES in step S136), it registers the end user information (step S137). The control unit 11 transmits the end user information entered by the person in charge to the management server 4 and registers it in the user DB 431. YES in step S136 means that the person in charge has selected a review for a specific end user. If the control unit 11 determines that the end user information is not to be registered (NO in step S136), it proceeds to step S138. NO in step S136 means that the person in charge has selected a review for which the end user has not been confirmed. If the control unit 11 determines that the case is not new (NO in step S135), it transmits a history request for the selected case to the management server 4 (step S139). The control unit 41 of the management server 4 receives the request (step S140). The control unit 41 acquires the history of the case from the history DB 433 and transmits it to the image processing device 1 (step S141). The control unit 41 receives the history (step S142).The control unit 41 proceeds to step S138. The control unit 11 executes the main processing (step S138). This main processing is the same as the main processing in embodiment 1, and therefore a description thereof will be omitted. Note that if a history has been received, the room selection screen is not displayed, and instead the walk-through screen is displayed. After the main processing, the control unit 11 transmits the history to the management server 4 (step S143). The history includes a theme ID, a theme ID string, a flooring material type, a wallpaper material type, etc. The control unit 41 of the management server 4 receives the history (step S144). The control unit 41 updates the history DB 433 based on the received history (step S145). The control unit 41 transmits a notification of update completion to the image processing device 1 (step S146). The control unit 11 of the image processing device 1 receives the completion (step S147). The control unit 11 determines whether to end the processing (step S148). The control unit 11 determines whether to end the review processing (step S148). Control unit. If the control unit 11 determines not to end the review process (NO in step S148), the process returns to step S134. If the control unit 11 determines to end the review process (YES in step S148), the control unit 11 ends the review process.

[0072] FIG. 34 is a flowchart showing an example of the procedure of the reference process. The reference process is a process executed when an end user references the contents of an interior design. The end user performs a login operation to the VR system 100 using the user terminal 5. The control unit 51 of the user terminal 5 sends a login request to the management server 4 (step S161). The control unit 41 of the management server 4 receives the login request (step S162). The control unit 41 searches the history DB 433 using the user ID included in the login request and acquires the history (step S163). The control unit 41 transmits the acquired history to the user terminal 5 (step S164). The control unit 51 of the user terminal 5 receives the history (step S165). The control unit 51 executes main processing (step S166). This main processing is similar to the main processing in the first embodiment, and therefore a description thereof will be omitted. Note that a room select screen is not displayed here. Furthermore, a walk-through screen is generated and displayed using the received history. The end user checks the design details with the designer on the walk-through screen and changes the flooring and wallpaper materials as necessary. After the main processing ends, the control unit 51 sends the changes made by the end user as update information to the management server 4 (step S167). The control unit 41 of the management server 4 receives the update information (step S168). The control unit 41 updates the history in the history DB 433 based on the update information (step S169). The control unit 41 sends a notification of update completion to the user terminal 5 (step S170). After receiving the completion notification (step S171), the control unit 51 of the user terminal 5 ends the reference processing.

[0073] FIG. 35 is an explanatory diagram showing another example of the walk-through screen. The walk-through screen d09 is almost the same as the walk-through screen d02 shown in FIG. 18. The following explanation will mainly focus on the differences. The walk-through screen d09 includes a top button d091, a room select button d092, a color select button d093, an environment switch button d094, a viewpoint height switch button d095, a print button d096, a floor plan button d097, a close button d098, a settings display d099, and an information icon d09A. The functions of each button, etc. are the same as those of the walk-through screen d02, so a detailed explanation will be omitted.

[0074] The walk-through screen d09 shown in Figure 35 does not include the VR mode button and change icon that are present on the walk-through screen d02 shown in Figure 18. Because it cannot operate in VR mode, the end user cannot freely change the viewpoint, but can only select a preset viewpoint. Also, because the change icon is not included, the end user cannot change the cloth material individually. This allows the interior design created by the interior designer to be reliably communicated to the end user.

[0075] In this embodiment, the end user can refer to the contents of the interior design, and can take their time to review the design created by the designer.

[0076] In this embodiment, a designer uses the image processing device 1 to create a design, and an end user uses the user terminal 5 to refer to and review the design created by the designer, but this is not limited to this. The management server 4 may provide services or programs to the user terminal 5 so that the user terminal 5 can execute functions equivalent to those of the image processing device 1. This allows the end user to simulate an interior design on their own without the help of an interior designer, etc. If the user terminal 5 is provided with all the same functions as the image processing device 1, and there is a risk that the end user will be confused by the multiple functions, it is desirable to provide a simplified version to the user terminal 5.

[0077] (Embodiment 3) This embodiment relates to a form in which a theme to be proposed to an end user is output using a learning model. When the theme is output, it may be output so that it can be distinguished from other themes to be displayed. The interior designer can clearly recognize the theme. For example, a label may be attached to the theme to be displayed, or the theme may be displayed in a separate column from other themes. Alternatively, when the theme is output, it may be output so that it cannot be distinguished from other themes to be displayed. This allows the interior designer to select a theme without preconceptions. In this embodiment, training data is prepared in which themes selected by an end user are associated with the attributes of the end user. A learning model is generated based on the training data. New end user attributes are input into the generated learning model. A designer considers the interior design to propose based on the themes obtained from the learning model.

[0078] FIG. 36 is an explanatory diagram showing an example of a training data DB. The training data DB 434 stores themes selected by the end user in association with the attributes of the end user. The training data DB 434 includes a user ID column, an age column, a gender column, an occupation column, a number of family members column, an annual income column, and a label column. The user ID column stores the user ID of the end user. The age column stores the age of the end user. The gender column stores the gender of the end user. The occupation column stores the occupation of the end user. The number of family members column stores the number of family members including the end user. The annual income column stores the annual income of the family including the end user. The label column stores the theme selected by the end user.

[0079] 37 is an explanatory diagram showing an example of a learning model. The learning model 441 is a neural network that takes end user attributes as input and themes as output. The neural network is, for example, a CNN (Convolution Neural Network), and has an input layer that accepts end user input, an output layer that outputs themes, and an intermediate layer that extracts feature quantities of end user attributes.

[0080] The input layer has multiple neurons that accept multiple attribute values ​​of the end user as input and passes the input item values ​​to the intermediate layer. The intermediate layer has a configuration in which convolution layers that convolve each attribute value input from the input layer and pooling layers that map the values ​​convolved in the convolution layer are alternately connected, compressing the input information and finally extracting features. The output layer has multiple neurons that output the suitability of each theme to the end user and outputs the suitability for each theme based on the features output from the intermediate layer.

[0081] In this embodiment, the learning model 441 is described as being a CNN, but the learning model 441 is not limited to a CNN and may be a model constructed using other learning algorithms, such as a neural network other than a CNN, a Bayesian network, or a decision tree.

[0082] The management server 4 performs learning using training data. The training data is data in which attribute information of the end user is associated with a theme selected by the end user. The training data is stored in the training data DB 434 as described above.

[0083] The management server 4 inputs the attribute values ​​of the end user contained in the training data into the input layer, undergoes calculation processing in the middle layer, and obtains information on themes suitable for the end user from the output layer. There are the same number of output nodes as there are themes. Each output node is associated with a theme, and outputs a value indicating the end user's suitability for each theme. Here, the sum of all suitability scores is 1.

[0084] The management server 4 compares the fitness output from the output layer with the label included in the training data, i.e., the correct value, and adjusts the fitness in the intermediate layer so that the output value of the output node corresponding to the label approaches 1. The parameters used in the calculation process are optimized. The parameters include, for example, the weights (coupling coefficients) between neurons and the coefficients of the activation functions used in each neuron. There are no particular limitations on the method for optimizing the parameters, but for example, the management server 4 uses the backpropagation method to optimize various parameters. The management server 4 performs the above learning process using all the training data and generates a trained learning model 441.

[0085] 38 is a flowchart showing an example of the procedure for the learning model generation process. The control unit 41 of the management server 4 creates training data from the user DB 431 and the history DB 433 (step S191). The control unit 41 stores the training data in the training data DB 434. The control unit 41 selects one of the training data stored in the training data DB 434 (step S192). The control unit 41 performs learning (step S193). The learning process has already been explained, so it will be omitted here. The control unit 41 determines whether or not there is unprocessed training data (step S194). If the control unit 41 determines that there is unprocessed training data (YES in step S194), the control unit 41 returns the process to step S192. If the control unit 41 determines that there is no unprocessed training data (NO in step S194), the control unit 41 stores the generated learning model 441 in the auxiliary storage unit 43 (step S195) and ends the process.

[0086] 39 is a flowchart showing an example of the procedure for theme selection processing. The theme selection processing is processing for selecting a theme that suits an end user using the learning model 441. For example, when a designer proposes an interior design to a new end user, the designer considers a design based on the theme selected by the theme selection processing.

[0087] The designer acquires the attributes of the end user and sends them to the management server 4 via the image processing device 1. The control unit 41 of the management server 4 acquires the attributes of the end user (step S201). The control unit 41 inputs the acquired attributes into the learning model 441 and acquires the theme to be selected from the output of the learning model 441 (step S202). The control unit 41 outputs the theme (step S203) and ends the process. The selected theme is sent to the image processing device 1. The designer selects the theme sent from the image processing device 1 and considers the design.

[0088] Although the theme selection process is performed by the management server 4 in the above embodiment, it is not limited to this. It may also be performed by the image processing device 1 used by the designer. In this case, a copy of the learning model 441 is stored in the auxiliary storage unit 13 of the image processing device 1.

[0089] A theme is selected based on the end user's attributes, but a learning model that proposes recommended combinations may also be generated based on the selected theme and the end user's attributes. The learning model is generated for each theme. Training data including the end user's attributes and the recommended combinations selected by the end user is generated for each theme from the historical data. The end user's attributes are the input, and the recommended combinations are the correct answer data. End user attributes are input into the learning model for each theme, and learning is performed so that the output of the learning model becomes the recommended combination.

[0090] In this embodiment, the theme is selected using the learning model 441, which makes it possible to increase the accuracy of theme selection, which is an early stage of design.

[0091] 40 is a block diagram showing an example of functional units included in an image processing device. The image processing device 1 includes, as functional units, a classification output unit 11a, a reception unit 11b, a generation unit 11c, an acquisition unit 11d, and an image acquisition unit 11e. Each of these functional units is realized by the control unit 11 operating based on a control program 1P.

[0092] The classification output unit 11a outputs a plurality of classifications related to the interior of the house. The generation unit 11c generates a virtual space that simulates a living space composed of interior components according to the selected classification. The acquisition unit 11d acquires a virtual space image seen from a virtual viewpoint within the generated virtual space. The image acquisition unit 11e outputs the acquired virtual space image.

[0093] (Proximity display function) As described above, when the viewpoint is set to the third viewpoint height (approximately 10 cm to 60 cm), a display mode is achieved that allows the user to more realistically confirm the texture of flooring materials. Materials whose textures can be confirmed do not have to be limited to flooring materials. The following describes the proximity display function that allows the user to confirm the texture of materials other than flooring materials.

[0094] The close-up display function switches the texture image attached to the surface of an object in virtual space to a higher resolution image when the position of the virtual viewpoint enters a predetermined range based on the object. When the close-up display function is enabled, the texture image attached to an object that is not the target of close-up display may be low-resolution or medium-resolution. High, medium, and low resolutions may be determined appropriately taking into consideration the hardware performance of the user terminal 5, etc. As an example of settings, 71 to 96 dpi is low resolution, around 200 dpi is medium resolution, and 350 to 400 dpi is high resolution.

[0095] FIG. 41 is a flowchart showing an example of the procedure for the close-up display process. The close-up display process is executed after the viewpoint change in step S54 and image acquisition in step S47 in the screen processing shown in FIG. 12. The close-up display process is also executed after image acquisition in step S104 in the viewpoint height change process shown in FIG. 15. The control unit 11 of the image processing device 1 acquires the viewpoint of the virtual camera (step S211). The control unit 11 selects an object to be processed (step S212). The object is, for example, an object whose value in the cloth material column of the object DB 138 is other than "none." The control unit 11 calculates the distance between the viewpoint position and the object (step S213). The control unit 11 determines whether the calculated distance is equal to or less than a predetermined threshold (step S214). If the control unit 11 determines that the calculated distance is equal to or less than the predetermined threshold (YES in step S214), the control unit 11 changes the texture attached to the surface of the object to a high-resolution one (step S215). If the control unit 11 determines that the calculated distance exceeds a predetermined threshold (NO in step S214), it determines whether or not there is an unprocessed object (step S216). If the control unit 11 determines that there is an unprocessed object (YES in step S216), it returns the process to step S212 and performs processing on the unprocessed object. If the control unit 11 determines that there is no unprocessed object (NO in step S216), it ends the process.

[0096] Although the close-up display process is described as being executed after step S47 or step S104, this is not limiting. It may also be processed integrally with image acquisition. When the viewpoint position of the virtual camera is changed, the appearance of each object as seen from the viewpoint position is recalculated. During this calculation, the distance between the viewpoint position and the object is calculated, and if the calculated distance is equal to or less than a threshold, the texture is made high-resolution. This allows for efficient rendering processing.

[0097] Fig. 42 is an explanatory diagram showing an example of proximity display. Fig. 42 shows an example of proximity display for flooring materials. In the proximity display, information such as the model, color name, material, price, or manufacturer name of the building material being displayed may also be displayed.

[0098] In the above explanation, it is assumed that the texture of the object (building material such as flooring or wall material) to be displayed in close proximity is changed to high resolution on the walk-through screen shown in FIG. 18, but this is not limitative. A window separate from the walk-through screen may be opened, and only the object to be displayed in 3D may be displayed in that window. Furthermore, in that window, it is possible to rotate the object. It may also be possible to make it possible to zoom in and out, and to check how it looks when sunlight hits it in the daytime and how it looks when indoor lighting hits it at night.

[0099] Although the texture of an object is changed to a high-resolution one when the distance between the viewpoint of the virtual camera and the object falls below a predetermined distance, this is not limitative. The user may explicitly specify an object to be displayed in close proximity using a pointing device or the like.

[0100] The close-up display function allows users to check the texture of individual materials (building materials). When displayed in 3D in a separate window, users can check the material as if they were holding it in their hands.

[0101] (Expanded operation history) In the second embodiment, a form of storing the operation history of the first user has been described. The operation history is stored in the history DB 433 shown in FIG. 32, but the stored history may be expanded. In addition to the content explicitly shown in FIG. 32, the history DB 433 may store a history of changes to the viewpoint height, an operation history of switching between day and night, and the like. Furthermore, a virtual space image may be stored in association with each operation history. The operation history may be stored each time a user performs an operation, or may be stored at predetermined time intervals.

[0102] Next, we will discuss the use of extended operation history. Here, the following operation is assumed. The users appearing are a showroom guide and an interior coordinator (hereinafter also referred to as "coordinator"). It is assumed that the guide is familiar with operating the VR system 100, but the coordinator is not so familiar with it. The coordinator visits the showroom and designs using the VR system 100 with the assistance of the guide. Alternatively, the guide may operate the VR system 100 in place of the coordinator. The operation history here is stored in the history DB 433 as the coordinator's history, even if the guide operates it. Furthermore, if the coordinator is not registered in the VR system 100, user registration is performed and a user ID is assigned to the coordinator.

[0103] FIG. 43 is an explanatory diagram showing another example of a history DB. The history DB 433 includes a history ID column, a date and time column, a sequence number column, a user ID column, a person in charge ID column, a theme ID column, a floor material column, a cloth material column, a change column, a viewpoint position column, a viewpoint height column, a day / night column, an image column, and a case ID column. The history ID column stores an ID that can uniquely identify a history. The date and time column stores the date and time when the user logged in. The sequence number column indicates the sequence number of the history. The user ID column stores the ID of the user, in this case, the coordinator. The person in charge ID column stores the ID of the guide. The theme ID column, floor material column, and cloth material column are the same as the columns with the same names shown in FIG. 32, so their explanations are omitted. Note that when floor materials and cloth materials are changed collectively to floor materials and cloth materials of different colors by referring to the recommended combination DB 132, the history is stored in the floor material column and the cloth material column. The change column stores the change history of cloth materials (building materials such as floor materials and wall materials). The viewpoint position column stores the history of changes to the viewpoint position. The viewpoint position column stores plane coordinates (x coordinate, y coordinate). The viewpoint height column stores the viewpoint position height ( The column stores the change history of the virtual space (z coordinate). If a preset first viewpoint height, second viewpoint height, or third viewpoint height is selected, this is stored in the viewpoint height column. In other cases, the viewpoint height column stores the z coordinate value. The day / night column stores the time zone switching history. As mentioned above, during the day, the lighting is expressed as sunlight pouring in through the window, and at night, the lighting is expressed as using lighting installed on the ceiling or wall as the light source, so switching between day and night changes the overall color in the virtual space. The image column stores virtual space images for each history. The image column may store the file name of the image file instead of the actual image data. The project ID column stores an ID that identifies the design project. By associating a series of histories with the same project ID, it becomes possible to determine the series of histories when referencing the history DB 433.

[0104] Once the design in the showroom is completed, an ID for the work, a project ID, is issued. The project ID may be issued when the work begins. The operation history is associated with the user ID and project ID and stored in the history DB 433. A two-dimensional code is also issued to the coordinator. The two-dimensional code includes information such as the user ID, project ID, date, and a URL (Uniform Resource Locator) for accessing the VR system 100. The two-dimensional code and date are printed out together with the virtual space image and provided to the coordinator. This printout may also be provided to the end user who plans to purchase a house.

[0105] The coordinator can access the VR system 100 using a two-dimensional code. The VR system 100 provides the coordinator with a web application. The coordinator can use the VR system 100 using a web browser without installing a new application on his or her terminal.

[0106] Next, processing related to operation history will be described. FIG. 44 is a flowchart showing an example of the procedure for image display processing. The image display processing is processing in which the coordinator refers to the content of the design created in the showroom on the terminal (user terminal) that the coordinator is using. The coordinator receives the design in the showroom and has the camera (not shown) of the user terminal 5 read the two-dimensional code. The control unit 51 of the user terminal 5 analyzes the two-dimensional code and issues a login request to the VR system 100 using the URL obtained through the analysis (step S221). The control unit 41 of the management server 4 receives the login request (step S222). The login request includes the user ID, the case ID, the date, etc. The control unit 41 reads the history from the history DB 433 using the user ID, the case ID, the date, etc. included in the login request (step S223). The control unit 41 creates a list of the read history and sends it to the user terminal 5 (step S224). The control unit 51 of the user terminal 5 receives the list and displays it on the display unit 56 (step S225). It is desirable that the list include a virtual space image corresponding to each history as a thumbnail. The coordinator uses the input unit 55 to select one of the histories. The control unit 51 transmits the selection information to the management server 4 (step S226). The control unit 41 of the management server 4 receives the selection information (step S227). The control unit 41 acquires the virtual space image corresponding to the selected history and transmits it to the user terminal 5 (step S228). The control unit 51 of the user terminal 5 receives the image and displays it on the display unit 56 (step S229), and the process ends.

[0107] The images displayed in the image display process are snapshot images, and cannot be operated like the walk-through screen, but can be switched to the walk-through screen at the instruction of the coordinator. In this case, the virtual space to be displayed on the walk-through screen is set based on the history. Alternatively, the snapshot images stored as history may be used only as thumbnails when displaying a list, and the walk-through screen based on the history may be reproduced and displayed. Since the application running on the coordinator's terminal is a web application, the functions available on the walk-through screen may be limited.

[0108] When displaying each history, not only the virtual space image but also the model number of the building material selected in the virtual space may be displayed. Furthermore, by linking with catalog information, it may be possible to display the price, form, features, etc. of the building material.

[0109] By checking the history, the coordinator can check the progress of the design review in the showroom. Also, by using the virtual space images stored along with the history, the coordinator can create materials for end users. Furthermore, by using the walk-through screen that reflects the history, the coordinator can make changes to the design.

[0110] The history may store information about the building materials that have been displayed in proximity. When referring to the history, the building materials that are displayed in proximity are included in the list, or a separate list of the building materials that are displayed in proximity is created.

[0111] (Bookmark function) The VR system 100 automatically stores history, but the bookmark function stores the state of the virtual space at a certain point in time when explicitly instructed by the user. FIG. 45 is an explanatory diagram showing an example of a bookmark DB. The bookmark DB 435 includes a bookmark ID column, a project ID column, a name column, a date and time column, a user ID column, a person in charge ID column, a theme ID column, a flooring material column, a wallpaper material column, a change column, and an image column. In the bookmark DB 435, the date and time column through the change column each store data similar to the columns with the same names in the history DB 433, so their explanations are omitted. The bookmark ID column stores an ID that can uniquely identify a bookmark. The project ID column stores an ID for each design project handled by the coordinator. The name column stores the name of the bookmark. It is assumed that the user inputs a name, but if no input is made, a name generated by the management server 4 is stored. The image column stores the virtual space image at the time when bookmark storage is instructed. The case ID column and the image column are data columns that are also included in the extended history described above. As with the history DB 433, a viewpoint position column, a viewpoint height column, and a day / night column may be provided in the bookmark DB 435. The instruction to save a bookmark is given by the user selecting a bookmark button. For example, the bookmark button is added to the top button d021, room select button d022, etc., lined up on the left side of the walk-through screen d02 shown in FIG. 18. When the mouse is right-clicked on the walk-through screen d02, a pull-down menu may be displayed, and the instruction to save a bookmark may be given from the menu.

[0112] The process of calling up a bookmark is the same as the image display process shown in Fig. 44, so a detailed explanation will be omitted. By using the bookmark function, the coordinator can memorize the situation of the virtual space at a timing specified by the coordinator.

[0113] (Recommended display) We will now explain the recommended building materials display. There is a wide variety of building materials that can be selected on the building materials selection screen shown in Figure 23. Therefore, even a coordinator may be confused about which building materials to select. The recommended display is a display that helps in selecting building materials. A persona is set for each theme. Also, recommended building materials are set for each theme. Based on the attributes of the end user, the persona that is closest to the end user is determined. Recommended building materials are identified based on the theme that corresponds to the determined persona. The identified building materials are displayed with a mark or icon indicating that they are recommended.

[0114] Figure 46 is an explanatory diagram showing an example of a persona DB. The persona DB 139 stores persona information for each theme. The persona DB 139 includes a theme ID column, an age column, a gender column, an occupation column, a family composition column, and an annual income column. The theme ID column stores the theme ID of the theme corresponding to the persona. The age column stores the age of the persona. The gender column stores the gender of the persona. The occupation column stores the occupation of the persona. The family composition column stores the family composition of the persona. The annual income column stores the family composition of the persona.

[0115] FIG. 47 is a flowchart showing an example of the procedure for the recommended mark assignment process. The recommended mark assignment process is executed when the building material selection screen d07 is generated. The control unit 11 of the image processing device 1 determines whether or not the attributes of the end user have been acquired (step S241). If the control unit 11 determines that the attributes of the end user have been acquired (YES in step S241), it determines the persona that is most similar to the end user (step S242). The control unit 11 calculates the similarity between the end user and each persona using the attributes of the end user and the attributes of the persona. The calculation of the similarity can be performed using publicly known technology, so a description thereof will be omitted. The persona with the highest similarity is determined to be the persona that is most similar to the end user. Control unit The control unit 11 refers to the persona DB 139 and determines the theme corresponding to the persona (step S243). The control unit 11 refers to the recommended combination DB and identifies building materials corresponding to the determined theme (step S244). The control unit 11 assigns a recommended mark to the identified building materials (step S245) and ends the process. If the control unit 11 determines that the end user's attributes have not been acquired (NO in step S241), it determines whether a theme has been selected (step S246). If the control unit 11 determines that a theme has been selected (YES in step S246), it moves the process to step S244. If the control unit 11 determines that a theme has not been selected (NO in step S246), it ends the process.

[0116] The method of assigning recommended marks is not limited to the above-mentioned method. Marks may be assigned to building materials that have a high affinity with the selected theme. Building materials that have a high affinity with the theme are determined by analyzing the operation histories of multiple users. The correspondence between themes and building materials is then stored in advance in the auxiliary storage unit 13. Also, based on the operation history of the user (coordinator), marks may be assigned to building materials that the user is likely to like or that have been selected in the past. Furthermore, marks may be assigned to building materials that manufacturers, etc., are particularly interested in promoting.

[0117] (Display related information) The recommendation display displays recommendations for building materials based on the end user's attributes. The related information display displays information based on building materials, such as personas corresponding to building materials and information on furniture that is considered to be good to combine with building materials.

[0118] Figure 48 is an explanatory diagram showing an example of a recommended furniture DB. The recommended furniture DB 13A stores information about furniture that is considered to be good to combine with building materials. The recommended furniture DB 13A includes a model column, a product name column, a product number column, a color column, an image column, and a catalog column. The model column stores the model of the building material. The product name column stores the product name of the recommended furniture. Product names include sofas, dining tables, and beds. The product number column stores the product numbers of the recommended furniture. The color column stores the color of the furniture. The image column stores images of the recommended furniture. The image column may store the file names of the image files. The catalog column stores catalog information about the recommended furniture or the URL of a website where the catalog information is posted.

[0119] FIG. 49 is a flowchart showing an example of the procedure for related information processing. Related information processing is processing that is executed when the user performs an operation such as right-clicking the mouse on a building material on the walk-through screen. The control unit 11 of the image processing device 1 accepts a command through a predetermined operation (step S251). The control unit 11 identifies the object that the user has operated based on the position of the mouse pointer or the like, and determines whether or not it is a target for related information processing (step S252). If the control unit 11 determines that the identified object is a target for related information processing (YES in step S252), it displays a menu (step S253). The control unit 11 determines whether or not the user has completed menu selection (step S254). If the control unit 11 determines that the user has not completed menu selection (NO in step S254), it repeats the determination process. If the control unit 11 determines that the user has completed menu selection (YES in step S254), it determines whether or not the menu selected by the user is "Catalog" (step S255). If the control unit 11 determines that the menu selected by the user is "catalog" (YES in step S255), it displays catalog information of the specified object, for example, building materials (step S256), and ends the process. For example, catalog information is acquired from the auxiliary storage unit 13 or the manufacturer's homepage using the model of the building materials as a key. If the control unit 11 determines that the menu selected by the user is not "catalog" (NO in step S255), it determines whether the menu selected by the user is "persona" (step S257). If the control unit 11 determines that the menu selected by the user is "persona" (YES in step S257), it displays persona information (step S258), and ends the process. The control unit 11 acquires persona information in the following procedure. The control unit 11 estimates The control unit 11 refers to the recommended combination DB 132 to acquire a theme ID corresponding to the building material. Persona information corresponding to the acquired theme ID is acquired from the persona DB 139. If the control unit 11 determines that the menu selected by the user is not "persona" (NO in step S257), it determines whether the menu selected by the user is "recommended furniture" (step S259). If the control unit 11 determines that the menu selected by the user is "recommended furniture" (YES in step S259), it acquires information on recommended furniture corresponding to the building material from the recommended furniture DB 13A and displays it (step S260), and ends the processing. If the control unit 11 determines that the menu selected by the user is not "recommended furniture" (NO in step S259), it ends the processing. Furthermore, if the control unit 11 determines that the identified object is not a target for related information processing (NO in step S252), it ends the processing.

[0120] The related information display function displays catalog information for building materials and persona information corresponding to the materials, allowing users to easily select building materials that seem appropriate for the end user.In addition, information on suitable furniture to be combined with building materials is also available, making it possible to recommend furniture purchases to end users.

[0121] In the above explanation, the related information display function is executed by right-clicking the building material with the mouse, but this is not limited to this. It may also be called up by clicking the information icon d02B shown in FIG.

[0122] (Recording function) On the walk-through screen d02, by continuously changing the position of the virtual camera, it is possible to obtain a series of images that give the perception of walking around in a virtual space. The recording function is a function that stores these images to create a video.

[0123] 50 is a flowchart showing an example of the procedure for video creation processing. The control unit 11 of the image processing device 1 receives an instruction to start recording (step S271). The control unit 11 starts recording (step S272). The control unit 11 determines whether or not to end recording (step S273). If the control unit 11 determines not to end recording (NO in step S273), it repeats the determination. If the control unit 11 determines to end recording (YES in step S273), it stores the video file in the auxiliary storage unit 13 (step S274) and ends the process. The control unit 11 determines to end when, for example, the user instructs to end.

[0124] Note that instructions to start / end recording can be given, for example, by adding a record button to the top button d021, room select button d022, etc. lined up on the left side of the walk-through screen d02 shown in FIG. 18, and the user can start recording by selecting the record button. While recording is in progress, the record button is changed to an end button and displayed, and recording ends when the user selects the end button. While recording, a pause button may be displayed in addition to the end button, allowing the user to pause and resume recording. When the mouse is right-clicked on the walk-through screen d02, a pull-down menu may be displayed, and instructions to start / end recording may be given from the menu.

[0125] The user may manually move the viewpoint position of the virtual camera using a pointing device such as a 3D mouse, or the virtual camera may move at a predetermined speed along a pre-set route. In this case, the user must instruct the start of recording, but not the end of recording. Even if the user does not instruct the end of recording, recording will end once movement along a predetermined route is complete. Multiple routes may be set so that the user can select one. Different routes may be set for each theme. Also, the virtual camera may stop midway through movement, display a close-up of the building material specified by the user for a predetermined period of time, and then resume movement. Furthermore, the user may be able to specify the height of the viewpoint and the speed of movement when moving. Also, the operation history or bookmarks may be saved. A video may be created by connecting a plurality of virtual space images stored in the virtual space.

[0126] When creating a video from a predetermined route by connecting multiple virtual space images, the management server 4 may perform the video creation process. The user can create the video by issuing an instruction from the user terminal 5. At that time, if the data held by the management server 4 is insufficient, the missing data is obtained from the image processing device 1.

[0127] FIG. 51 is an explanatory diagram showing examples of travel routes. Three routes are shown in FIG. 51. Route 1 and Route 2 have the same starting position but different goal positions. Route 3 has a different starting position from Route 1 and Route 2 but the same goal position as Route 2. Note that the user may be able to change the pre-set routes. The user may also be able to create new routes. Different pre-set routes may be prepared for each theme.

[0128] (Presentation board creation function) The presentation board is used when a coordinator proposes his / her own coordination to an end user (customer). A plurality of pieces of information are collected on one screen and displayed on the display unit 56 of the user terminal 5.

[0129] Figure 52 is an explanatory diagram showing an example of a template DB. The template DB 437 stores templates for creating presentation boards. The template DB 437 includes a theme ID column, a pattern column, an area 1 column, an area 2 column, an area 3 column, and a template column. The theme ID column stores the theme ID of the theme corresponding to the template. The pattern column stores the pattern name of the template. Here, this is represented as a number. The area 1 column, area 2 column, and area 3 column store the data content to be embedded in each area. The image indicates a virtual space image. The description indicates an explanation of the coordination concept. The building materials indicate information on the building materials selected in the virtual space. The template column stores the definition of the template. The size, position, etc. of each area are stored.

[0130] 53 is a flowchart showing an example of the procedure for the presentation board creation process. The user inputs a request to create a presentation board to the user terminal 5. The control unit 51 of the user terminal 5 sends the request to create a presentation board to the management server 4 (step S281). The control unit 41 of the management server 4 receives the request to create a presentation board (step S282). The control unit 41 acquires the user's operation history from the history DB 433 (step S283). The control unit 41 creates a list of the history and sends it to the user terminal 5 (step S284). The control unit 51 of the user terminal 5 receives the list and displays it on the display unit 56 (step S285). The user selects the history to use via the input unit 55. The control unit 51 accepts the selection (step S286). The control unit 51 sends the selected information to the management server 4 (step S287). The control unit 41 of the management server 4 receives the selected information (step S288). The control unit 41 acquires a theme corresponding to the selected history and acquires a template corresponding to the theme from the template DB 437 (step S289). The control unit 41 embeds data included in the history, such as the virtual space image and information on building materials, into the template and creates a presentation board (step S290). The control unit 41 transmits the presentation board to the user terminal 5 (step S291). The control unit 51 of the user terminal 5 receives the presentation board (step S292). The control unit 51 displays the presentation board on the display unit 56 (step S293). The user inputs a concept explanation for the coordination, etc., using the input unit 55. The control unit 51 accepts the input (step S294). Note that the concept explanation, etc. may be input after step S286 and transmitted to the management server 4 in step S287. The control unit 51 embeds the accepted concept explanation into the presentation board and redisplays the presentation board on the display unit 56 (step S295). The user checks the displayed content and inputs the necessary commands. The commands include "edit" and "save and exit". The control unit 51 receives the command (step S296). The control unit 51 determines whether the received command indicates completion or not (step S297). If the control unit 51 determines that the command does not indicate completion (NO in step S297), it executes the command (step S298) and returns the process to step S295. Depending on the content of the command, the process may be returned to step 285. If the control unit 51 determines that the command indicates completion (YES in step S297), it stores the presentation board in the auxiliary storage unit 53 (step S299) and ends the process.

[0131] Figure 54 is an explanatory diagram showing example templates. Figure 54 shows three types of templates. All templates have three types of embedded information, and the information types are the same. What differs between the templates is the position of the display area and the balance of size. A virtual space image is embedded in area 521. An explanation of the coordinator's concept is embedded in area 522. An image of the selected building material is embedded in area 523. Information displayed in area 523 may include catalog information for the building material (model, price, form, features, etc.).

[0132] The presentation board creation function allows the coordinator to easily create a presentation board for the client. In addition, the VR system 100 collects the necessary information according to the template, saving the coordinator the trouble of collecting information.

[0133] (Centralized data management) In the above-described embodiment, the image processing device 1 or the management server 4 stores data for configuring the virtual three-dimensional space, user data, and user operation history data, but this is not limited to this. A database server for centralized management may be provided to facilitate the management, updating, and sharing of this data. Furthermore, a terminal for a salesperson who introduces the VR system 100 may also be included as part of the configuration of the VR system 100.

[0134] 55 is an explanatory diagram showing another example of the configuration of a VR system. The VR system 100 includes an image processing device 1, a management server 4, a user terminal 5, a database server 6, and a sales representative terminal 7. The image processing device 1, the management server 4, the user terminal 5, the database server 6, and the sales representative terminal 7 are connected to each other via a network N so that they can communicate with each other.

[0135] The configurations of the image processing device 1, management server 4, and user terminal 5 are the same as those in the second embodiment, and therefore detailed description thereof will be omitted.

[0136] The database server 6 is configured with a server computer, a workstation, etc. The database server 6 may be configured with a multicomputer consisting of multiple computers, a virtual machine virtually constructed by software, or a quantum computer. The database server 6 stores data for configuring the virtual three-dimensional space, user information, and user operation history. In the above-described embodiment, the data for configuring the virtual three-dimensional space is stored in the image processing device 1. The user information and user operation history are stored in the management server 4. Here, these data are stored in the database server 6.

[0137] The salesperson terminal 7 is configured as a notebook PC or a tablet computer, etc. The salesperson terminal 7 is a terminal used mainly by a salesperson who sells building materials. The salesperson sells the building materials by introducing the VR system 100 to the coordinator. The salesperson terminal 7 stores explanatory materials for the VR system 100, etc.

[0138] The image processing device 1 is a computer with higher specifications than the user terminal 5 and the salesperson terminal 7. The image processing device 1 is used by a showroom guide who is familiar with operating the VR system 100. The user terminal 5 is used by a coordinator.

[0139] The image processing device 1, sales representative terminal 7, and user terminal 5 provide different functions as follows: All functions of the VR system 100 are available on the image processing device 1. The sales representative terminal 7 can demonstrate the VR system 100. The sales representative terminal 7 allows users to select a theme and view virtual space images designed for each selected theme. However, the viewpoint position of the virtual camera can only be selected from multiple preset positions. A web application is provided on the user terminal 5. The user terminal 5 has a main function of creating a presentation board using the results of the design performed on the image processing device 1.

[0140] The image processing device 1, management server 4, and database server 6 constitute a showroom system as a subsystem. The salesperson system as a subsystem is constituted by the salesperson terminal 7, management server 4, and database server 6. The coordinator system as a subsystem is constituted by the user terminal 5, management server 4, and database server 6. Since the database server 6 centrally manages the data used in the VR system 100, it is possible for any of the showroom system, salesperson system, or coordinator system to share and cooperate with each other on data.

[0141] The technical features (constituent elements) described in each embodiment can be combined with each other, and by combining them, new technical features can be formed. The embodiments disclosed herein are to be considered as illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0142] 100 VR Systems 1. Image processing device 11 Control section 11a Classification output section 11b Reception 11c Generator 11d Acquisition Department 11e Image acquisition section 12 Main memory 13 Auxiliary storage 131 Theme DB 132 Recommended Combination DB 133 Texture DB 134 Room DB 135 Vertex DB 136 Edge DB 137 surface DB 138 Object DB 139 Persona DB 13A Recommended Furniture DB 15 Input I / F 16 Display I / F 17 Communications Department 18 Reading unit 1P control program 1a Portable storage media 1b semiconductor memory 2 Controller 3. Display 4 Management Server 41 Control Unit 42 Main memory 43 Auxiliary storage section 431 User DB 432 Case DB 433 History DB 434 Training Data DB 435 Bookmark DB 437 Template DB 441 Learning Model 45 Communications Department 4P control program 5. User terminal 51 Control section 52 Main memory 53 Auxiliary storage section 54 Communications Department 55 Input section 56 Display section 5P control program B Bus N Network

Claims

1. Output multiple classifications related to home interiors, Accept the classification selection, A virtual space is generated that simulates a living space composed of interior components according to the selected classification; Acquire a virtual space image seen from a virtual viewpoint within the generated virtual space, The acquired virtual space image is output to a screen that allows a user to walk through the virtual space. An image output method characterized in that the processing is executed by a computer.

2. When the virtual viewpoint is set within a predetermined distance from the interior component, the texture image of the interior component is switched; Displays information about the interior components being displayed 2. The image output method according to claim 1, wherein the processing is executed by the computer.

3. a first classification output unit that outputs a plurality of classifications related to the interior of a house; a reception unit that receives a selection of a category; a first generation unit that generates a virtual space that simulates a living space configured by interior components according to the selected classification; a setting unit that sets a virtual viewpoint in the generated virtual space in accordance with an instruction from the first user; an acquisition unit that acquires a virtual space image seen from the virtual viewpoint; a first image output unit that outputs the acquired virtual space image to a screen that allows a user to walk through the virtual space; a first system having An image output system comprising:

4. In addition to the first system, a classification acquisition unit that acquires a classification selected by a first user from a plurality of classifications related to the interior of a house; a second generation unit that generates a virtual space that simulates a living space configured by interior components according to the acquired classification; a selection receiving unit that receives, from a second user, a selection from a plurality of virtual viewpoints that are preset in the virtual space; a second acquisition unit that acquires a virtual space image seen from the received virtual viewpoint; a second image output unit that outputs the acquired virtual space image; A second system having 4. The image output system according to claim 3, further comprising:

5. a classification output unit that outputs a plurality of classifications related to the interior of the house; a reception unit that receives a selection of a category; a generator for generating a virtual space that simulates a living space configured by interior components according to the selected classification; an acquisition unit that acquires a virtual space image seen from a virtual viewpoint in the generated virtual space; an image output unit that outputs the acquired virtual space image to a screen that allows a user to walk through the virtual space; An image output device comprising:

6. Output multiple classifications related to home interiors, Accept the classification selection, A virtual space is generated that simulates a living space composed of interior components according to the selected classification; Acquire a virtual space image seen from a virtual viewpoint within the generated virtual space, The acquired virtual space image is output to a screen that allows a user to walk through the virtual space. An image output program that causes a computer to execute processing.

7. a first classification output unit that outputs a plurality of classifications related to the interior of a house; a first receiving unit that receives a selection of a category; a first generation unit that generates a virtual space that simulates a living space configured by interior components according to the selected classification; a setting unit that sets a virtual viewpoint in the generated virtual space in accordance with an instruction from a user; an acquisition unit that acquires a virtual space image seen from the virtual viewpoint; a first image output unit that outputs the acquired virtual space image to a screen that allows a user to walk through the virtual space; an ID acquisition unit that acquires identification information assigned to the coordinator; a storage unit that stores the acquired identification information and a virtual space image in association with each other; a result output unit that outputs the acquired identification information and the virtual space image; A showroom system with a second classification output unit that outputs a plurality of classifications related to the interior of the house; a second receiving unit that receives a selection of a category; a second generation unit that generates a virtual space that simulates a living space configured by interior components according to the selected classification; a second acquisition unit that acquires a virtual space image seen from a predetermined virtual viewpoint; a second image output unit that outputs the acquired virtual space image to a screen that allows a user to walk through the virtual space; A sales representative system having a third receiving unit that receives the identification information; a third image output unit that acquires the virtual space image corresponding to the received identification information from the storage unit and outputs the image on a screen that allows a user to walk through the virtual space; Coordinator system having An image output system comprising:

Images