Image output method, image output system, image output device, and image output program

Abstract

To provide an image output method or the like capable of referring to an interior design result after selecting a theme, and also to provide an image output method or the like capable of providing a three-dimensional image with a sense of presence.SOLUTION: An image output method includes the steps of: outputting a plurality of classifications relating to an interior of a house; accepting the selection of classification; generating a virtual space which imitates a living space constituted by an interior component corresponding to the selected classification; acquiring a virtual space image viewed from a virtual viewpoint in the generated virtual space; and performing processing which outputs the acquired virtual space image by a computer.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present invention relates to an image output method and the like for outputting a virtual space image viewed from a virtual viewpoint in a virtual space.

Background Art

[0002] Devices for supporting interior design have been proposed (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the known technique described in Patent Document 1, after selecting an indoor theme, it is necessary to select the color patterns of interior materials and architectural materials. Also, the output image is a two-dimensional image and lacks a sense of presence.

[0005] The present invention has been made in view of such a situation. Its object is to provide an image output method and the like in which the result of interior design can be referred to after selecting a theme. Also, to provide an image output method and the like capable of providing a three-dimensional image with a sense of presence.

Means for Solving the Problems

[0006] An image output method according to one aspect of the present application outputs a plurality of classifications related to the interior of a house, accepts a selection of a classification, generates a virtual space simulating a living space composed of interior components corresponding to the selected classification, obtains a virtual space image viewed from a virtual viewpoint in the generated virtual space, and is characterized in that a computer performs a process of outputting the obtained virtual space image.

Effects of the Invention

[0007] In one aspect of this application, after selecting a classification (theme), the results of the interior design can be viewed. [Brief explanation of the drawing]

[0008] [Figure 1] This is an explanatory diagram showing an example configuration of a VR system. [Figure 2] This is an explanatory diagram showing an example of a theme database. [Figure 3] This is an explanatory diagram showing an example of a recommended combination database. [Figure 4] This is an explanatory diagram illustrating an example of a texture database. [Figure 5] This is an explanatory diagram showing an example of a room database. [Figure 6] This is an explanatory diagram illustrating an example of a vertex database. [Figure 7] This is an explanatory diagram showing an example of an edge database. [Figure 8] This is an explanatory diagram showing an example of a surface database. [Figure 9] This is an explanatory diagram showing an example of an object database. [Figure 10] This is a flowchart showing an example of the main processing procedure. [Figure 11] This flowchart shows an example of the procedure for generating a walkthrough screen. [Figure 12] This is a flowchart showing an example of the screen processing procedure. [Figure 13] This flowchart shows an example of the color selection process. [Figure 14] This flowchart shows an example of the procedure for switching environments. [Figure 15] This flowchart shows an example of the procedure for changing the viewpoint height. [Figure 16] This flowchart shows an example of the procedure for switching viewpoints. [Figure 17] This is an explanatory diagram showing an example of the room selection screen. [Figure 18] This is an explanatory diagram showing an example of a walkthrough screen. [Figure 19]It is an explanatory diagram showing an example of a color selection menu screen. [Figure 20] It is an explanatory diagram showing an example of an environment switching screen. [Figure 21] It is an explanatory diagram showing an example of a viewpoint height change screen. [Figure 22] It is an explanatory diagram showing an example of a layout screen. [Figure 23] It is an explanatory diagram showing an example of a building material selection screen. [Figure 24] It is an explanatory diagram showing an example of an information screen. [Figure 25] It is an explanatory diagram showing an example of an information screen. [Figure 26] It is an explanatory diagram showing the state of storage opening and closing. [Figure 27] It is an explanatory diagram showing another configuration example of a VR system. [Figure 28] It is a block diagram showing an example of the hardware configuration of a management server. [Figure 29] It is a block diagram showing an example of the hardware configuration of a user terminal. [Figure 30] It is an explanatory diagram showing an example of a user DB. [Figure 31] It is an explanatory diagram showing an example of a case DB. [Figure 32] It is an explanatory diagram showing an example of a history DB. [Figure 33] It is a flowchart showing an example of the procedure of consideration processing. [Figure 34] It is a flowchart showing an example of the procedure of reference processing. [Figure 35] It is an explanatory diagram showing another example of a walkthrough screen. [Figure 36] It is an explanatory diagram showing an example of a training data DB. [Figure 37] It is an explanatory diagram showing an example of a learning model. [Figure 38] It is a flowchart showing an example of the procedure of learning model generation processing. [Figure 39] It is a flowchart showing an example of the procedure of theme selection processing. [Figure 40]This is a block diagram showing an example of a functional unit of an image processing device. [Figure 41] This flowchart shows an example of the procedure for proximity display processing. [Figure 42] This is an explanatory diagram showing an example of proximity display. [Figure 43] This is an explanatory diagram showing another example of a history database. [Figure 44] This flowchart shows an example of the image display processing procedure. [Figure 45] This is an explanatory diagram showing an example of a bookmark database. [Figure 46] This is an explanatory diagram showing an example of a persona database. [Figure 47] This flowchart shows an example of the procedure for assigning a "recommended" mark. [Figure 48] This is an explanatory diagram showing an example of a recommended furniture database. [Figure 49] This flowchart shows an example of the procedure for processing related information. [Figure 50] This is a flowchart showing an example of the video creation process. [Figure 51] This is an explanatory diagram showing an example of a travel route. [Figure 52] This is an explanatory diagram showing an example of a template database. [Figure 53] This flowchart shows an example of the steps involved in creating a presentation board. [Figure 54] This is an explanatory diagram showing an example template. [Figure 55] This is an explanatory diagram showing other configuration examples of the VR system. [Modes for carrying out the invention]

[0009] (Embodiment 1) The following embodiments will be described with reference to the drawings. Figure 1 is an explanatory diagram showing an example of the configuration of a VR system. VR is an abbreviation for Virtual Reality. The VR system 100 is an image processing device 1 This includes a controller 2 and a display 3. The image processing device 1 consists of a desktop PC (Personal Computer), a notebook PC, or a tablet computer, etc. To enable the display of the most realistic video images possible, the image processing unit 1 may be composed of a computer equipped with high-performance graphics processing capabilities, a so-called gaming PC. Alternatively, the image processing unit 1 may be composed of a multi-computer system consisting of multiple computers, a virtual machine virtually constructed by software, or a quantum computer. .

[0010] Controller 2 is, for example, a VR controller. The user uses Controller 2 to give instructions to the image processing device 1. Display 3 is a display device including a liquid crystal display panel or the like. Display 3 is preferably a high-definition display such as a 4K display or an 8K display. Alternatively, Display 3 may be a head-mounted display or VR goggles.

[0011] The image processing device 1 includes a control unit 11, a main memory unit 12, an auxiliary memory unit 13, an input interface 15, a display interface 16, a communication unit 17, and a reading unit 18. The control unit 11, main memory unit 12, auxiliary memory unit 13, input interface 15, display interface 16, communication unit 17, and reading unit 18 are connected by bus B.

[0012] The control unit 11 has one or more arithmetic processing units such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), and a GPU (Graphics Processing Unit). The control unit 11 realizes various functions by reading and executing a control program 1P stored in the auxiliary storage unit 13.

[0013] The main memory unit 12 includes SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), flash memory, etc. The main memory unit 12 is primarily controlled by the control unit 11 It temporarily stores the data necessary to perform the calculation.

[0014] The auxiliary storage unit 13 is a hard disk or SSD (Solid State Drive), and the control unit 11 stores the control program 1P and various DBs (Databases) necessary for processing. The auxiliary storage unit 13 stores the theme DB 131, recommended combination DB 132, texture DB 133, room DB 134, vertex DB 135, edge DB 136, face DB 137, and object DB 138. The auxiliary storage unit 13 may be an external storage device connected to the image processing device 1.

[0015] Input I / F15 is for communication via communication standards such as PS / 2 and Bluetooth (registered trademark). This is a port. Input I / F15 communicates with controller 2. Input I / F15 may also communicate with a keyboard or mouse (not shown in the diagram).

[0016] Display I / F16 is a display output port that uses communication standards such as HDMI (High-Definition Multimedia Interface), DisplayPort, and Thunderbolt. Display I / F16 outputs the image signal to Display 3.

[0017] The communication unit 17 communicates with other computers via the network. The control unit 11 may use the communication unit 17 to download a control program 1P from another computer via the network N and store it in the auxiliary storage unit 13.

[0018] The reading unit 18 reads a portable storage medium 1a, including CD (Compact Disc)-ROM and DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 18 and store it in the auxiliary storage unit 13. Alternatively, the control unit 11 may read the control program 1P from the semiconductor memory 1b.

[0019] Some of the processing performed by the image processing device 1, such as rendering, may be outsourced to a server computer with higher performance capabilities. The image processing device 1 receives the image created by rendering from the server computer and displays it on the display 3.

[0020] In the following description, "interior" includes interior finishes and furnishings. "Interior components" refers to the basic elements that make up a room, such as flooring, wall coverings, and ceiling coverings; as well as fixtures and decorative materials such as doors and windows; furniture; kitchen systems; curtains; lighting fixtures; and other individual elements that constitute the interior. "Wall covering" refers to interior finishing materials such as wall coverings and ceiling coverings, and finishing materials for custom-made furniture. "Custom-made furniture" refers to built-in furniture made to match the interior. In this embodiment, building materials refer to flooring and wall coverings.

[0021] Next, we will explain the database (DB) used in the VR system 100. Figure 2 is an explanatory diagram showing an example of a theme DB. The theme DB 131 stores information about interior themes. A theme is a word that succinctly represents the concept used to give a sense of unity to the interior design. Themes are sometimes expressed as style or taste. The theme DB 131 includes a theme ID column, a theme name column, a summary column, and a description column. The theme ID column stores a theme ID that uniquely identifies the theme. The theme name column stores the theme name. The summary column stores a summary description of the theme. The description column stores a detailed description of the theme.

[0022] Figure 3 is an explanatory diagram showing an example of a recommended combination database. The recommended combination database 132 stores recommended combinations of flooring and wallpaper materials for each theme. The recommended combination database 132 stores columns for theme ID, flooring material, and wallpaper material. The theme ID column stores the theme ID. The flooring material and wallpaper material columns store the IDs, such as product model numbers, of the flooring and wallpaper materials for which combinations are recommended, respectively.

[0023] Figure 4 is an explanatory diagram showing an example of a texture database. The texture database 133 stores texture images. A texture is a repeating pattern applied to flooring or fabric materials. A texture image is an image of the repeating pattern of the texture. The texture database 133 includes a texture ID column, a model column, a group column, an image column, and an application column. The texture ID column stores a texture ID that can uniquely identify the texture. The model column stores the product model of the flooring or fabric material corresponding to the texture. The group column stores the group to which the texture belongs. The image column stores the texture image. The file name of the texture image may be stored in the image column, and the file may be stored in the auxiliary storage unit 13. The application column stores the target to which the texture is applied. Furthermore, texture images may include not only patterns of flooring and wallpaper materials, but also general textures in computer graphics, i.e., images representing the surface patterns of objects.

[0024] Figure 5 is an explanatory diagram showing an example of a room database. Room DB 134 stores information about rooms represented as a virtual three-dimensional space. Room DB 134 includes columns for room ID, name, theme, floor, wall, ceiling, fixtures, and virtual camera. The room ID column stores a room ID that uniquely identifies the room. The name column stores the name of the room. The name may be, for example, "A type" or "B Examples include "type" and "C type". The theme column stores the theme ID of the design theme to be applied when the room is first displayed. The floor column stores information about the floor that makes up the room. The floor information includes, for example, coordinate values ​​indicating the placement of the surface representing the floor in a virtual 3D space, and the texture ID of the texture to be applied as flooring material. The wall column stores information about the walls that make up the room. The wall information includes, for example, coordinate values ​​indicating the placement of the surface representing the wall in a virtual 3D space, and the texture ID of the texture to be applied as wall material. The ceiling column stores information about the ceiling that makes up the room. The ceiling information includes, for example, coordinate values ​​indicating the placement of the surface representing the ceiling in a virtual 3D space, and the texture ID of the texture to be applied as a finishing material. The ceiling column also stores information about the surface used to represent a dropped ceiling. The joinery column stores identification information for the joinery that makes up the room. The virtual camera column stores the position, shooting direction, etc., of the virtual camera that is pre-set in the virtual 3D space. The image of the virtual 3D space captured by the virtual camera becomes the output image from the image processing device 1. The virtual camera corresponds to the user's virtual viewpoint.

[0025] Figure 6 is an explanatory diagram illustrating an example of a vertex database. Vertex DB 135 stores vertices representing a virtual three-dimensional space. Vertex DB 135 includes a vertex ID column, an X-coordinate column, a Y-coordinate column, and a Z-coordinate column. The vertex ID column stores a vertex ID that uniquely identifies the vertex. The X-coordinate column stores the X-coordinate of the vertex. The Y-coordinate column stores the Y-coordinate of the vertex. The Z-coordinate column stores the Z-coordinate of the vertex.

[0026] Figure 7 is an explanatory diagram showing an example of an edge database. Edge DB136 stores edges that represent a virtual 3D space. Edge DB136 includes an edge ID column, a start column, and an end column. The edge ID column stores an edge ID that uniquely identifies the edge. The start column stores the vertex ID of the vertex that indicates the starting point of the edge. The end column stores the vertex ID of the vertex that indicates the ending point of the edge. Edges are directed edges with a defined direction.

[0027] Figure 8 is an explanatory diagram showing an example of a face database. Face DB 137 stores faces that represent a virtual three-dimensional space. Face DB 137 includes a face ID column, edge 1 column, edge 2 column, edge 3 column, edge 4 column, edge 5 column, and edge 6 column. The face ID column stores a face ID that can uniquely identify a face. Edge 1 column, edge 2 column, edge 3 column, edge 4 column, edge 5 column, and edge 6 column each store the edge IDs of the edges that make up the face. Since a face can be made up of at least three edges, edge 4 to edge 6 columns may be NULL, indicating that there are no values. Since the direction of the edges is defined, the face is configured so that tracing the edges that make up the face completes one full circuit of the outer perimeter. That is, the vertices that make up the face are components of two edges, being the starting point of one edge and the ending point of the other edge. The front and back of the face are defined by the direction of the outer perimeter. When viewing the face, the front is when the outer perimeter is counterclockwise, and the back is when the outer perimeter is clockwise.

[0028] Figure 9 is an explanatory diagram showing an example of an object database. Object DB 138 stores information about objects placed in a room. Objects include fixtures such as doors and windows. Object DB 138 includes columns for object ID, type, name, definition, fabric material, behavior, and icon. The object ID column stores an object ID that uniquely identifies the object. The type column stores the type of object. Types include, for example, fixtures, furniture, built-in furniture, or others. The name column stores the name of the object. The definition column stores the definition of the 3D shape. For example, it stores the face IDs of multiple faces that represent the object. The fabric material column stores information about the surface to which the fabric material is applied. Information about the texture to be applied to surfaces that are not the target of the fabric material is included in the definition column. The behavior column stores methods (functions) for changing the state of the object. For example, open() is a method for opening doors or closet doors. close() is a method for closing doors or closet doors. This is a method for closing doors, etc. The icon column stores information about the operation icons attached to objects. Icons can be, for example, change icons (selection icons) or information icons. A change icon opens a screen for changing the texture of an object. An information icon opens an information screen about an object. If no icon is attached to an object, the icon column stores "none".

[0029] As shown above, when defining a virtual 3D space, we first define vertices. We define edges from two vertices. We define faces from multiple edges. We define objects from multiple faces. This is one example of a definition method called a surface model or surface modeling. The definition method is not limited to this. A surface model with triangles as the minimum element is also acceptable, as are wireframe models, solid models, etc.

[0030] Next, we will describe the information processing performed by the image processing device 1. Figure 10 is a flowchart showing an example of the main processing procedure. The main processing is the primary processing performed by the image processing device 1. The control unit 11 of the image processing device 1 displays the top screen on the display 3 (step S1). The control unit 11 displays the room selection screen on the display 3 (step S2). The user selects a room on the room selection screen. Each room has a different interior design theme. The control unit 11 generates a walkthrough screen corresponding to the room selected by the user. (Step S3). When the user performs an operation on the walkthrough screen, the control unit 11 determines whether the operation is a menu selection (Step S4). If the control unit 11 determines that it is not a menu selection (NO in Step S4), it performs processing on the walkthrough screen (screen processing) (Step S5) and returns the process to Step S4. If the control unit 11 determines that it is a menu selection (YES in Step S4), it determines whether room selection has been selected (Step S6). If the control unit 11 determines that room selection has been selected (YES in Step S6), it returns the process to Step S2. If the control unit 11 determines that room selection has not been selected (NO in Step S6), it determines whether the top screen has been selected (Step S7). If the control unit 11 determines that the top screen has been selected (YES in Step S7), it returns the process to Step S1. If the control unit 11 determines that the top screen has not been selected (NO in Step S7), it determines whether display settings change has been selected (Step S8). If the control unit 11 determines that changing the display settings has been selected (YES in step S8), it updates the screen according to the change in settings (step S9) and returns the process to step S4. If the control unit 11 determines that changing the display settings has not been selected (NO in step S8), it determines whether termination has been selected (step S10). If the control unit 11 determines that termination has not been selected (NO in step S10), it displays the screen according to the selected menu (step S11) and returns the process to step S4. If the control unit 11 determines that termination has been selected (YES in step S10), it terminates the main process.

[0031] Figure 11 is a flowchart showing an example of the walkthrough screen generation process. The walkthrough screen generation process corresponds to step S3 in Figure 10. The control unit 11 of the image processing device 1 obtains information about the room selected by the user on the room selection screen from the room DB 134 (step S21). The control unit 11 uses the theme ID included in the room information to obtain recommended combination information from the recommended combination DB 132 (step S22). Based on the recommended combination information, the control unit 11 obtains the texture of the flooring material and the texture of the cross material from the texture DB 133 (step S23). Based on the contents of the floor, wall, ceiling, and fittings columns in the room DB 134, the control unit 11 obtains information about the objects included in the virtual 3D space (step S24). Based on the object information, the control unit 11 generates the virtual 3D space (step S25). At this time, texture images obtained from the texture DB 133 are pasted onto the floor, walls, closets, etc. The control unit 11 acquires images of the virtual 3D space based on the contents of the virtual camera column in the room DB 134 (step S26). The control unit 11 generates a walkthrough screen including the acquired images (step S27). The control unit 11 outputs the generated walkthrough screen (step S28) and returns the process to the caller.

[0032] Figure 12 is a flowchart showing an example of the screen processing procedure. The screen processing corresponds to step S5 in Figure 10. The control unit 11 of the image processing device 1 detects an object at a location specified by the user within the walkthrough screen (step S41). The control unit 11 determines whether the detected object is an icon or not (step S42). If the control unit 11 determines that the detected object is an icon (YES in step S42), it determines whether the icon is a change icon or not (step S43). If the control unit 11 determines that the icon is a change icon (YES in step S43), it displays a selection screen for changing the flooring material or wallpaper material (step S44). If the object is a floor surface, the control unit 11 displays a flooring material selection screen. If the object is a wall, closet, etc., the control unit 11 displays a wallpaper material selection screen. The user selects a flooring material or wallpaper material, or selects "Close". The control unit 11 determines whether the user's operation is an instruction to change the flooring material or wallpaper material (step S45). If the control unit 11 determines that there is an instruction to change the flooring material (YES in step S45), it changes the texture of the floor or wall, etc., according to the instruction (step S46). The control unit 11 takes the image of the virtual camera after the change. The control unit 11 gains an advantage (step S47). Based on the acquired image, the control unit 11 updates the walkthrough screen (step S48). The control unit 11 outputs the updated walkthrough screen (step S49) and returns the process to the caller. If the control unit 11 determines that it is not an instruction to change the flooring material (NO in step S45), it returns the process to the caller. If the control unit 11 determines that the icon is not a change icon (NO in step S43), it acquires information to display the information screen (step S50). For example, if an information icon attached to furniture is selected, it acquires information about the furniture. The control unit 11 generates a screen to display the acquired information (step S51). The control unit 11 outputs the generated screen (step S52) and returns the process to the caller. If the control unit 11 determines that the detected object is not an icon (NO in step S42), it determines whether or not it is a viewpoint change (step S53). If the control unit 11 determines that it is a viewpoint change (YES in step S53), it performs a viewpoint change (step S54). The control unit 11 switches or changes the position of the virtual camera. Based on the changed virtual camera, the control unit 11 executes steps S47 onwards. If the control unit 11 determines that it is not a viewpoint change (NO in step S53), it returns the process to the caller. The viewpoint change in step S54 allows the user to freely change the viewpoint using the controller 2. The user feels as if they are freely walking around inside a room represented as a virtual 3D space.

[0033] Figure 13 is a flowchart showing an example of the color selection process. The color selection process is one of the processes corresponding to the screen update (step S9) in Figure 10. The control unit 11 creates a menu screen (step S61). The menu screen displays multiple recommended combinations of flooring and wallpaper materials and is created based on the recommended combination DB 132. It is desirable that only combinations corresponding to the theme be displayed. On the menu screen, the control unit 11 sets the display of the selection status so that the user can see which combination is currently selected (step S62). The control unit 11 displays the generated menu screen (step S63). The user selects the combination they changed on the menu screen or selects "Back". The control unit 11 determines whether the user has selected another combination (step S64). If the control unit 11 determines that the user has selected another combination (YES in step S64), it changes the texture of the floor and wall according to the selected combination (step S65). The control unit 11 acquires the image after the texture change (step S66). The control unit 11 updates the screen based on the acquired image (step S67). The control unit 11 outputs the updated screen (step S68) and returns to step S64. If the control unit 11 determines that the user has not selected any other combination (NO in step S64), it clears the menu screen (step S69) and terminates the process.

[0034] Figure 14 is a flowchart showing an example of the environment switching process procedure. The environment switching process is one of the processes corresponding to the screen update (step S9) in Figure 10. The control unit 11 displays a menu (step S81). The menu may include options such as "Day / Night Switching," "Storage Open / Close," and "Furniture Display." The user changes one of the menu items or selects "Back." The control unit 11 determines whether or not the day / night switching operation has been performed (step S82). If the control unit 11 determines that the day / night switching operation has been performed (YES in step S82), it changes the light source (virtual light source) (step S83). Day / night switching switches the time of day in the virtual 3D space between daytime and nighttime. Daytime assumes clear skies. In the case of daytime, the sun is used as the light source, and the lighting is represented as if sunlight is pouring in through a window. In the case of nighttime, the lighting is represented as if lights installed on the ceiling or walls are the light source. The control unit 11 acquires the image after the light source change (step S84). The control unit 11 updates the screen based on the acquired image (step S85). Note that the view from the window may also be switched when switching between day and night. The control unit 11 outputs the updated screen (step S86) and returns to step S82. If the control unit 11 determines that the day / night switch has not been operated (NO in step S82), it determines whether the storage opening / closing operation has been performed (step S87). If the control unit 11 determines that the storage opening / closing operation has been performed (YES in step S87), the closet The control unit 11 changes the open / closed state of doors such as closets (step S88). The control unit 11 changes the state by executing a method stored in the behavior column of the object DB 138. In addition to closets, drawers of desks and chests of drawers may also be opened and closed. The open state may have multiple stages, such as 25% open, 50% open, and 100% open. The control unit 11 executes steps S84 onwards. If the control unit 11 determines that the storage opening / closing has not been operated (NO in step S87), it determines whether the furniture display has been operated (step S89). If the control unit 11 determines that the furniture display has been operated (YES in step S89), it changes whether the furniture is displayed or hidden (step S90). The control unit 11 executes steps S84 onwards. If the control unit 11 determines that the furniture display has not been operated (NO in step S89), it clears the menu (step S91) and terminates the process.

[0035] Figure 15 is a flowchart illustrating an example of the viewpoint height change procedure. The viewpoint height change procedure is one of the processes corresponding to the screen update (step S9) in Figure 10. The viewpoint height change procedure does not change the position in a plane parallel to the floor, but only changes the viewpoint height. By setting the viewpoint height to a value that takes into account the user's height, it becomes possible to output a more realistic image. Also, by setting the viewpoint height lower, it becomes possible to view the room from a child's perspective. The control unit 11 displays a height change menu (step S101). The menu displays several different heights, with the current setting selected. The user can change the height or select "Back". The control unit 11 determines whether the height has been changed or not (step S102). If the control unit 11 determines that the height has been changed (YES in step S102), it changes the viewpoint (step S103). The control unit 11 changes the height of the virtual camera corresponding to the viewpoint. The control unit 11 acquires the image after the change (step S104). The control unit 11 updates the screen based on the acquired image (step S105). The control unit 11 outputs the updated screen (step S106) and returns the process to step S102. If the control unit 11 determines that the height has not been changed (NO in step S102), it clears the menu (step S107) and terminates the process.

[0036] Figure 16 is a flowchart showing an example of the viewpoint switching procedure. The viewpoint switching procedure is one of the processes corresponding to the screen update (step S9) in Figure 10. The viewpoint switching procedure does not change the height of the viewpoint, but changes the position in a plane parallel to the floor. The control unit 11 of the image processing device 1 displays the layout screen (step S121). The layout screen is a plan view showing the shape of the room and the positions of fixtures and other items in the room. The layout screen shows the positions of multiple virtual cameras that can be specified as viewpoints using icons. The display mode of the currently selected virtual camera is different from the display mode of other virtual cameras. The user can change the virtual camera or select "Close". The control unit 11 determines whether the selection state has changed or not (step S122). If the control unit 11 determines that the selection state has changed (YES in step S122), it changes the viewpoint (step S123). The control unit 11 changes the virtual camera corresponding to the viewpoint. The control unit 11 acquires the image after the change (step S124). The control unit 11 updates the screen based on the acquired image (step S125). The control unit 11 outputs the updated screen (step S126) and returns the process to step S122. If the control unit 11 determines that the selection state has not changed (NO in step S122), it clears the layout screen (step S127) and terminates the process.

[0037] Next, we will illustrate the screens used in the above process. Figure 17 is an explanatory diagram showing an example of a room selection screen. The room selection screen d01 is the screen for selecting the room to display. The room selection screen d01 includes a top button d011 and multiple room selection buttons d012. Selecting the top button d011 transitions to the top screen (not shown). Selecting a room selection button d012 transitions to a walkthrough screen that displays the selected room. In the example shown in Figure 17, each room has an interior design with a different theme, and the room selection button d012 is the name of the corresponding theme (Calm, Flow, A The following are included: dvnaced, Wa Story, Tasty. The themes to display are: The control unit 11 generates the theme based on the theme information stored in the theme DB 131. Note that when setting up for a specific apartment building, the room select button d012 may be labeled with A-type, B-type, etc., indicating the room type.

[0038] Figure 18 is an explanatory diagram showing an example of a walkthrough screen. The walkthrough screen d02 includes a top button d021, a room select button d022, a color select button d023, an environment switching button d024, a viewpoint height switching button d025, a VR mode button d026, a print button d027, a floor plan button d028, a close button d029, a settings display d02A, an information icon d02B, and a change icon d02C. Selecting the top button d021 transitions to the top screen. Selecting the room select button d022 returns to the room select screen d01. Selecting the color select button d023 activates a color select process that allows you to change the flooring and wallpaper materials at once. Selecting the environment switching button d024 activates an environment switching process that allows you to switch between day and night modes and open and close storage, etc. Selecting the viewpoint height switching button d025 activates a viewpoint switching process that allows you to change the viewpoint height. Selecting the VR mode button d026 outputs an image for VR goggles from the display interface 16. The print button d027 sends a hard copy of the screen to the printer. Selecting the floor plan button d028 activates the viewpoint switching process and displays the layout screen as an overlay. Selecting the close button d029 hides the close button d029 from the top button d021. The settings display d02A displays various settings. The settings display d02A includes themes d02A1, combinations d02A2, viewpoint height d02A3, selected camera icon d02A4, and camera icon d02A5. Themes d02A1 displays the name of the selected theme. Combinations d02A2 displays the types of flooring and wallpaper materials. Viewpoint height d02A3 displays the height of the selected viewpoint. Selected camera icon d02A4 indicates the position of the virtual camera selected as the viewpoint on the room layout diagram. Camera icon d02A5 indicates the positions of selectable virtual cameras on the room layout diagram. Selecting the information icon d02B displays an information screen overlay showing information about the item with the icon. Selecting the change icon d02C triggers screen processing and displays an overlay selection screen for changing the texture applied to the item with the icon.The information icon d02B and the change icon d02C are positioned by the control unit 11 according to the values ​​in the icon column of the object DB138.

[0039] Figure 19 is an explanatory diagram showing an example of the color select menu screen. The color select menu screen d03 (hereinafter simply referred to as "menu screen d03") is a screen that displays multiple recommended combinations (options) of flooring and wallpaper materials, and is displayed during the color select process. The menu screen d03 includes thumbnails d031, combination type d032, selection rectangle d033, check mark d034, and a back button d035. Thumbnail d031 shows the room design image when each recommended combination of flooring and wallpaper material is applied. Combination type d032 shows the product type of the flooring and wallpaper material being combined. Selection rectangle d033 shows the currently selected combination. Check mark d034 shows the combination currently applied to the walkthrough screen d02. Selecting the back button d035 clears the menu screen d03. It is possible to change the flooring and wallpaper materials by selecting a combination other than the currently selected combination from the multiple recommended combinations displayed on the menu screen d03. For changes to the texture of flooring materials, the control unit 11 performs the modification based on the surface data stored in the floor column of the room DB 134. For changes to the texture of walls, the control unit 11 performs the modification based on the surface data stored in the wall column of the room DB 134. For fixtures and furniture, the control unit 11 checks whether a change is needed by referring to the value in the cloth material column of the object DB, and then changes the texture of the target surface based on the value in the definition column.

[0040] Figure 20 is an explanatory diagram showing an example of the environment switching screen. Environment switching screen d04 is the environment switching menu This screen displays the environment switching process. The environment switching screen d04 includes the furniture area d041, the day / night area d042, the storage area d043, the selection rectangle d044, and the back button d045. The furniture area d041 allows the display and hiding of furniture to be toggled. When switching the display and hiding of furniture, the control unit 11 determines whether each object placed in the room is furniture or not based on the value in the type column of the object DB 138, and toggles the display and hiding of objects that are determined to be furniture. The day / night area d042 allows the room's lighting environment to be switched between daytime and nighttime. The storage area d043 allows the doors of storage furniture such as closets to be opened and closed. The selection rectangle d044 indicates which of the furniture area d041, day / night area d042, or storage area d043 is the target of operation (active). Selecting the back button d045 clears the environment switching screen d04.

[0041] Figure 21 is an explanatory diagram showing an example of a viewpoint height change screen. The viewpoint height change screen d05 is a screen that displays the selectable viewpoint heights and is displayed during the viewpoint height change process. The viewpoint height change screen d05 includes a selection rectangle d051, a check mark d052, and a back button d053. The selection rectangle d051 indicates the currently selected viewpoint height. The check mark d052 indicates the viewpoint height currently applied to the walkthrough screen d02. Selecting the back button d053 clears the viewpoint height change screen d05. Figure 21A is an example of a walkthrough screen when the viewpoint height is 150cm. Figure 21B is an example of a walkthrough screen when the viewpoint height is 60cm. Note that the selectable viewpoint heights are stored in the auxiliary storage unit 13 as values ​​common to multiple rooms. The maximum viewpoint height may be stored in the room DB 134, etc., as a value corresponding to the ceiling height of the room.

[0042] The viewpoint heights shown in Figure 21 are just examples. It is desirable to have at least three viewpoint heights available. The first viewpoint height is approximately 130cm to 190cm, assuming an adult's perspective. The second viewpoint height is approximately 60cm to 130cm, assuming a child's perspective. The third viewpoint height is approximately 10cm to 60cm, a viewpoint that allows for a more realistic examination of materials, especially the texture of flooring. The third viewpoint height may be 50cm or less, preferably 40cm or less, and may be 15cm or more, preferably 20cm or more. By providing at least three viewpoint heights, it becomes possible to experience interior design according to height, and furthermore, to experience the texture of materials in real time. There may be one or more viewpoint heights each for adult perspective, child perspective, and perspective for examining materials. In addition, the viewpoint height may be set continuously using a trackbar. In this case, it is desirable to display approximate value ranges corresponding to the adult's perspective, the child's perspective, and the perspective for checking the material, along with trackbars. Additionally, when switching from the first or second viewpoint height to the third viewpoint height, floor material information such as name and model number may be automatically displayed. Conversely, when switching from the third viewpoint height back to the first or second viewpoint, the display of floor material information such as name and model number may automatically disappear.

[0043] Figure 22 is an explanatory diagram showing an example of a layout screen. Figure 22A is an example of a layout screen showing the virtual camera position before the change. Figure 22B is an example of a layout screen showing the virtual camera position after the change. Layout screen d06 is a diagram showing the selectable virtual camera positions on the room floor plan and the position of the virtual camera currently selected as the viewpoint. Layout screen d06 includes a position marker d061, viewpoint display d062, field of view display d063, and a close button d064. The position marker d061 indicates the selectable virtual camera positions. The viewpoint display d062 indicates the selected virtual camera. The virtual camera position can be changed by selecting the position marker d061 with a mouse click, etc. The field of view display d063 shows the field of view corresponding to the display on walkthrough screen d02. Selecting the close button d064 closes layout screen d06.

[0044] Figure 23 is an explanatory diagram showing an example of a building materials selection screen. Building materials selection screen d07 is for flooring or This is a selection screen for changing the building materials, and is displayed when the change icon is selected during screen processing. The building materials selection screen d07 includes a building materials display d071, a selection rectangle d072, a tone switch d073, a tone switch d074, and a close button d075. The building materials display d071 shows the texture and type of each building material. If the selected change icon is attached to the floor, the building materials display d071 displays the flooring material. If the selected change icon is attached to the wall, closet, etc., the building materials display d071 displays the wallpaper material. The selection rectangle d072 shows the currently applied building materials. Selecting tone switch d073 or tone switch d074 switches the tone of the building materials displayed in the building materials display d071. Building materials are grouped according to their color tone. The control unit 11 performs this grouping using the values ​​in the group column of the texture DB 133. Tones include, for example, white tone, light tone, gray tone, medium tone, dark tone, or deep tone. Selecting the close button d075 will close the building materials selection screen d07. Note that the building materials displayed in the building materials display d071 may differ depending on the selected theme.

[0045] Figures 24 and 25 are explanatory diagrams showing examples of information screens. Information screen d08 is a screen that displays information about a product and is displayed when an information icon is selected during screen processing. Figure 24 is an example of information screen d08 displayed when an information icon attached to a building material is selected. Figure 25 is an example of information screen d08 displayed when an information icon attached to furniture, etc., is selected. Information about the product is stored in the auxiliary storage unit 13 and read and displayed by the control unit 11 as appropriate.

[0046] Figure 26 is an explanatory diagram showing the opening and closing of storage units. Figure 26A shows the screen with the storage unit doors closed. Figure 26B shows the screen with the storage unit doors open. The opening and closing of the storage unit doors is performed by the environment switching process described above.

[0047] In this embodiment, multiple recommended combinations of flooring and wallpaper materials are defined for each theme, allowing for diverse interior designs for each theme. Furthermore, the results of the design can be realistically confirmed using computer graphics in a virtual 3D space. Additionally, by changing the recommended combinations, the design can be finely modified without changing the theme.

[0048] The room's lighting environment can be switched between daytime and nighttime, allowing users to see how the design appears in different lighting conditions. The open and closed states of built-in furniture doors, such as closet doors, can be viewed as images, enabling users to determine whether the furniture's placement is appropriate. Since building materials can be changed on an individual interior component basis, parts of the design can be modified according to the end user's preferences.

[0049] After the user selects a room on the room selection screen, before generating and displaying the walkthrough screen, an introductory screen introducing the theme adopted in the design of the selected room may be displayed. This introductory screen makes it possible to propose a spatial concept that suits the user's lifestyle.

[0050] In this embodiment, the intention is to support the interior design of rooms carried out by real estate developers, architects, and interior designers who are commissioned by end users who purchase houses, but it is not limited to that. It may also be used by building material manufacturers to realistically represent the installation image of their wide variety of products in living spaces.

[0051] (Embodiment 2) This embodiment relates to a configuration in which a second user can refer to the contents of an interior design set by a first user by storing the operation history of the image processing device 1. In this embodiment, the interior designer is the first user and the end user is the second user, but it is not limited to this. For example, the interior designer may be the first user and a real estate developer or architect may be the second user, or the real estate developer or architect may be the first user and the end user may be the second user.

[0052] Figure 27 is an explanatory diagram showing another example configuration of the VR system. The VR system 100 includes an image processing device 1, a management server 4, and a user terminal 5. The image processing device 1, the management server 4, and the user terminal 5 are connected to each other via a network N, enabling them to communicate with one another.

[0053] The image processing device 1 is a desktop PC (Personal Computer), similar to Embodiment 1. It consists of a PC or tablet computer, etc. Image processing device 1 is used by interior designers, etc. who perform interior design work.

[0054] The management server 4 consists of a server computer, workstation, etc. The management server 4 may also consist of a multicomputer system with multiple computers, a virtual machine virtually constructed by software, or a quantum computer. The management server 4 manages end-user information and the operation history of the image processing device 1.

[0055] User terminal 5 consists of a notebook PC or tablet computer, etc. User terminal 5 is used by end users. In Figure 27, only one image processing device 1 and one user terminal 5 are shown, but there may be multiple units. The system including the management server 4 and the image processing device 1 is called the first system, and the system including the management server 4 and user terminal 5 is called the second system.

[0056] Figure 28 is a block diagram showing an example of the hardware configuration of a management server. The management server 4 includes a control unit 41, a main memory unit 42, an auxiliary memory unit 43, and a communication unit 45. The control unit 41, the main memory unit 42, the auxiliary memory unit 43, and the communication unit 45 are connected by bus B.

[0057] The control unit 41 has one or more arithmetic processing units such as CPUs, MPUs, and GPUs. The control unit 41 functions as a management server 4 by reading and executing the control program 4P stored in the auxiliary storage unit 43.

[0058] The main memory unit 42 is an SRAM, DRAM, flash memory, etc. The main memory unit 42 primarily temporarily stores data necessary for the control unit 41 to perform arithmetic processing.

[0059] The auxiliary storage unit 43 is a hard disk or SSD, and stores the control program 4P and various databases necessary for the control unit 41 to execute processing. The auxiliary storage unit 43 stores the user database 431, case database 432, history database 433, and training data database 434. The auxiliary storage unit 43 also stores the learning model 441. The auxiliary storage unit 43 may be an external storage device connected to the management server 4. The various databases stored in the auxiliary storage unit 43 may be stored on a database server or cloud storage different from the management server 4. Note that the training data database 434 and the learning model 441 are not essential components in this embodiment.

[0060] The communication unit 45 communicates with the image processing device 1 and the user terminal 5 via the network. The control unit 41 may use the communication unit 45 to download a control program 4P from another computer via the network N and store it in the auxiliary storage unit 43.

[0061] Figure 29 is a block diagram showing an example of the hardware configuration of a user terminal. User terminal 5 is controlled It includes a main memory unit 51, a main memory unit 52, an auxiliary memory unit 53, a communication unit 54, an input unit 55, and a display unit 56. Each component is connected by bus B.

[0062] The control unit 51 has one or more arithmetic processing units such as CPUs, MPUs, and GPUs. The control unit 51 functions as a user terminal 5 by reading and executing a control program 5P stored in the auxiliary storage unit 53.

[0063] The main memory unit 52 is an SRAM, DRAM, flash memory, etc. The main memory unit 52 primarily temporarily stores data necessary for the control unit 51 to perform arithmetic processing.

[0064] The auxiliary storage unit 53 is a hard disk or SSD, and stores the control program 5P and various databases necessary for the control unit 51 to execute processing. The various databases stored in the auxiliary storage unit 53 may be stored on a database server or cloud storage separate from the user terminal 5.

[0065] The communication unit 54 communicates with the management server 4 via the network. The control unit 51 may use the communication unit 54 to download the control program 5P from another computer via the network N and store it in the auxiliary storage unit 53.

[0066] The input unit 55 is a keyboard or mouse. The display unit 56 includes a liquid crystal display panel or the like. The display unit 56 displays virtual space images acquired from a virtual three-dimensional space. Alternatively, the display unit 56 may be a touch panel display integrated with the input unit 55. The user terminal 5 may also display information on an external display device.

[0067] Next, we will explain the database managed by the management server 4. Figure 30 is an explanatory diagram showing an example of a user database. User database 431 stores information about end users. User database 431 includes columns for user ID, age, gender, occupation, number of family members, and annual income. The user ID column stores a user ID that can uniquely identify the end user. The age column stores the end user's age. The gender column stores the end user's gender. The occupation column stores the end user's occupation. The number of family members column stores the number of people living in the same room as the end user. The annual income column stores the end user's annual income. User database 431 may also store other attributes that can be used as reference for interior design, such as hobbies and favorite colors.

[0068] Furthermore, the following data may be stored in User DB431: information on locations and properties selected by real estate developers, architects, or end-users, as well as information that reveals the end-user's preferences and lifestyle. The former information can be used to propose locations and property exteriors in conjunction. The latter information can be used to make more appealing proposals to end-users. Examples of the latter information include favorite travel destinations (countries, accommodations), accommodations stayed at during travel, books and magazines (fashion magazines, etc.) that they usually read, hobbies, places where they buy clothes, favorite brands, etc.

[0069] Figure 31 is an explanatory diagram showing an example of a case database. Case DB 432 stores each interior design project. Case DB 432 includes columns for Case ID, Person in Charge ID, User ID, Name, and Status. The Case ID column stores a case ID that uniquely identifies the project. The Person in Charge ID column stores the ID of the person in charge of interior design, such as the interior designer. The User ID column stores the ID of the end user. The Name column stores the name of the project. The name is set by the person in charge as appropriate. The Status column stores the status of the project. Statuses include, for example, Under Consideration, Proposal in Progress, and Contracted. Under Consideration means the design is being considered. Proposal in Progress means the end user is considering whether to approve the design and place an order. Contracted means the design has been finalized and the end user has signed a contract.

[0070] Figure 32 is an explanatory diagram showing an example of a history database. The history database (history storage unit) 433 stores the operation history of the image processing device 1. By referring to the operation history, it is possible to understand the progress and results of the interior design. The history database 433 includes columns for history ID, date and time, user ID, person in charge ID, theme ID, flooring material, wallpaper material, and change. The history ID column records a history ID that can uniquely identify the history. The date and time column stores the date and time when the history was stored. The user ID column stores the user ID of the end user. The person in charge ID column stores the ID of the person in charge of design. The theme ID column stores the theme ID of the selected theme. The flooring material column stores the model number of the selected flooring material. The wallpaper material column stores the model number of the selected wallpaper material. The change column stores the details of the changes when a part of the design is changed. For example, in the initial settings, the wallpaper is unified throughout the room, but in the living room / kitchen, the wallpaper in the kitchen and the exterior of the back kitchen counter may be different designs. In this case, the change column stores the surface ID representing the kitchen wall, the object ID representing the kitchen counter, and the type of the changed wallpaper material.

[0071] Next, we will explain the information processing performed by the VR system 100. Figure 33 is a flowchart showing an example of the procedure for the review process. The review process is performed when a design person reviews interior designs. The person logs into the VR system 100 by operating the image processing device 1. The control unit 11 of the image processing device 1 sends a login request to the management server 4 (step S131). The control unit 41 of the management server 4 performs the login process (step S132). The login process is, for example, authentication using an ID and password. The control unit 41 searches the case DB 432, creates a list of cases associated with the person in charge, and sends it to the image processing device 1 (step S133). The control unit 11 of the image processing device 1 displays the received list on the display 3 (step S134). If the person in charge is reviewing a case included in the list, they select the case; if they are reviewing a new case, they indicate "new." At that time, they select whether it is a review for a specific end user or a review for which the end user has not yet been determined. The control unit 11 determines whether it is a new case or not (step S135). If the control unit 11 determines that it is a new case (YES in step S135), it determines whether or not to register the end user's information (step S136). If the control unit 11 determines to register the end user's information (YES in step S136), it registers the end user's information (step S137). The control unit 11 sends the end user information entered by the person in charge to the management server 4 and registers it in the user DB 431. A YES in step S136 occurs when the person in charge has selected to consider a specific end user. If the control unit 11 determines not to register the end user's information (NO in step S136), it proceeds to step S138. A NO in step S136 occurs when the person in charge has selected a consideration for which the end user has not been determined. If the control unit 11 determines that it is not a new case (NO in step S135), it sends a history request for the selected case to the management server 4 (step S139). The control unit 41 of the management server 4 receives the request (step S140). The control unit 41 retrieves the history of the case from the history DB 433 and sends it to the image processing device 1 (step S141). The control unit 41 receives the history (step S142).The control unit 41 proceeds to step S138. The control unit 11 executes the main process (step S138). This main process is the same as the main process in Embodiment 1, so its explanation is omitted. If history has been received, the room select screen will not be displayed and the walkthrough screen will be displayed. After the main process, the control unit 11 sends the history to the management server 4 (step S143). The history includes the theme ID, theme ID column, floor material type, cross material type, etc. The control unit 41 of the management server 4 receives the history (step S144). The control unit 41 updates the history DB 433 based on the received history (step S145). The control unit 41 sends a notification of update completion to the image processing device 1 (step S146). The control unit 11 of the image processing device 1 receives completion (step S147). The control unit 11 determines whether or not to terminate the process (step S148). The control unit 11 determines whether or not to terminate the review process (step S148). Control unit. If control unit 11 determines that it does not want to terminate the review process (NO in step S148), it returns to step S134. If control unit 11 determines that it wants to terminate the review process (YES in step S148), it terminates the review process.

[0072] Figure 34 is a flowchart illustrating an example of the reference processing procedure. The reference processing is performed when an end user refers to the contents of the interior design. The end user uses the user terminal 5 to log in to the VR system 100. The control unit 51 of the user terminal 5 sends a login request to the management server 4 (step S161). The control unit 41 of the management server 4 receives the login request (step S162). The control unit 41 uses the user ID included in the login request to search the history DB 433 and retrieve the history (step S163). The control unit 41 sends the retrieved history to the user terminal 5 (step S164). The control unit 51 of the user terminal 5 receives the history (step S165). The control unit 51 executes the main processing (step S166). This main processing is the same as the main processing in Embodiment 1, so its explanation is omitted. Note that the room selection screen is not displayed here. Also, a walkthrough screen is generated and displayed using the received history. The end user reviews the design content on a walkthrough screen, and changes the flooring and wallpaper materials as needed. After the main processing is complete, the control unit 51 sends the changes made by the end user as update information to the management server 4 (step S167). The control unit 41 of the management server 4 receives the update information (step S168). Based on the update information, the control unit 41 updates the history in the history DB 433 (step S169). The control unit 41 sends a message to the user terminal 5 indicating that the update is complete (step S170). After receiving the completion message, the control unit 51 of the user terminal 5 terminates the reference processing (step S171).

[0073] Figure 35 is an explanatory diagram showing another example of a walkthrough screen. Walkthrough screen d09 is almost identical to walkthrough screen d02 shown in Figure 18. The following explanation will mainly focus on the differences. Walkthrough screen d09 includes a top button d091, a room select button d092, a color select button d093, an environment switching button d094, a viewpoint height switching button d095, a print button d096, a floor plan button d097, a close button d098, a settings display d099, and an information icon d09A. The functions of each button, etc., are the same as in walkthrough screen d02, so the explanation will be omitted.

[0074] In the walkthrough screen d09 shown in Figure 35, the VR mode button and change icon found in the walkthrough screen d02 shown in Figure 18 are not included. Because it cannot operate in VR mode, end users cannot freely change their viewpoint and can only select from pre-set viewpoints. Furthermore, because the change icon is not included, end users cannot individually change the fabric material. This ensures that the interior design set by the interior designer is reliably communicated to the end user.

[0075] In this embodiment, the content of the interior design becomes accessible to the end user. The end user can take their time to review the design created by the designer.

[0076] In this embodiment, a designer uses the image processing device 1 to create a design, and an end user uses the user terminal 5 to refer to and review the design created by the designer; however, this is not limited to this configuration. The management server 4 may provide services or programs to the user terminal 5 so that it can perform functions equivalent to those of the image processing device 1. This would enable the end user to simulate interior design themselves without the need for assistance from an interior designer or similar professional. If providing the user terminal 5 with all the same functions as the image processing device 1 would confuse the end user due to its numerous features, it is desirable to provide the user terminal 5 with a simplified version of the functions.

[0077] (Embodiment 3) This embodiment relates to a form that outputs themes to be proposed to end users using a learning model. When outputting themes, the output may be made in a way that distinguishes them from other themes to be displayed. Interior designers can clearly recognize the themes. For example, themes to be displayed may be marked, or themes may be displayed in separate fields from other themes. Alternatively, when outputting themes, the output may be made in a way that makes it impossible to distinguish them from other themes to be displayed. Interior designers can select themes without preconceived notions. In this embodiment, training data is prepared that associates themes selected by the end user with the attributes of the end user. A learning model is generated based on the training data. New end user attributes are input into the generated learning model. Based on the themes obtained from the learning model, the designer considers the interior design to propose.

[0078] Figure 36 is an explanatory diagram showing an example of a training data database. The training data database 434 stores the themes selected by the end user, associating them with the attributes of that end user. The training data database 434 includes columns for User ID, Age, Gender, Occupation, Family Size, Annual Income, and Label. The User ID column stores the end user's User ID. The Age column stores the end user's age. The Gender column stores the end user's gender. The Occupation column stores the end user's occupation. The Family Size column stores the number of family members including the end user. The Annual Income column stores the annual income of the family including the end user. The Label column stores the themes selected by the end user.

[0079] Figure 37 is an explanatory diagram illustrating an example of a learning model. Learning model 441 is a neural network that takes end-user attributes as input and outputs a theme. The neural network is, for example, a CNN (Convolutional Neural Network) and has an input layer that accepts end-user input, an output layer that outputs a theme, and an intermediate layer that extracts feature quantities of end-user attributes.

[0080] The input layer has multiple neurons that accept multiple attribute values ​​of the end user as input, and passes the input item values ​​to the hidden layer. The hidden layer has a configuration in which a convolution layer that convolves each attribute value input from the input layer and a pooling layer that maps the values ​​convolved in the convolution layer are alternately connected, compressing the input information and ultimately extracting features. The output layer has multiple neurons that output the degree of fit for the end user for each theme, and outputs the degree of fit for each theme based on the features output from the hidden layer.

[0081] In this embodiment, the learning model 441 is described as a CNN, but the learning model 441 is not limited to a CNN and may be a model constructed using other learning algorithms such as neural networks other than CNNs, Bayesian networks, or decision trees.

[0082] Management server 4 performs training using training data. The training data is data that associates end-user attribute information with the themes selected by the end-users. As mentioned above, the training data is stored in training data DB434.

[0083] The management server 4 inputs the attribute values ​​of end users included in the training data into the input layer, performs calculations in the hidden layer, and obtains information about themes suitable for the end users from the output layer. The number of output nodes is equal to the number of themes. Each output node is associated with a theme and outputs a value indicating the degree of fit for each theme. Here, the sum of the degrees of fit is 1.

[0084] The management server 4 compares the goodness of fit output from the output layer with the labels included in the training data, i.e., the correct values, and adjusts the hidden layers so that the output value of the output node corresponding to the label approaches 1. The parameters used in the computation process are optimized. These parameters include, for example, the weights (connection coefficients) between neurons and the coefficients of the activation function used in each neuron. The method of parameter optimization is not particularly limited, but for example, the management server 4 uses backpropagation to optimize various parameters. The management server 4 performs the above learning process using all the training data and generates a trained model 441.

[0085] Figure 38 is a flowchart showing an example of the procedure for generating a learning model. The control unit 41 of the management server 4 creates training data from the user DB 431 and the history DB 433 (step S191). The control unit 41 stores the training data in the training data DB 434. The control unit 41 selects one of the training data stored in the training data DB 434 (step S192). The control unit 41 performs learning (step S193). Learning has already been explained, so it will be omitted here. The control unit 41 determines whether or not there is any unprocessed training data (step S194). If the control unit 41 determines that there is any unprocessed training data (YES in step S194), it returns to step S192. If the control unit 41 determines that there is no unprocessed training data (NO in step S194), it stores the generated learning model 441 in the auxiliary storage unit 43 (step S195) and terminates the process.

[0086] Figure 39 is a flowchart illustrating an example of the theme selection process. The theme selection process uses the learning model 441 to select a theme suitable for the end user. For example, when a designer proposes an interior design to a new end user, they consider a design based on the theme selected through the theme selection process.

[0087] The designer obtains the end user's attributes and sends them to the management server 4 via the image processing device 1. The control unit 41 of the management server 4 obtains the end user's attributes (step S201). The control unit 41 inputs the obtained attributes into the learning model 441 and obtains the theme to be selected from the output of the learning model 441 (step S202). The control unit 41 outputs the theme (step S203) and terminates the process. The selected theme is sent to the image processing device 1. The designer selects the theme sent from the image processing device 1 and considers the design.

[0088] The theme selection process is to be performed by the management server 4, but it is not limited to that. It may also be performed by the image processing device 1 used by the designer. In this case, a copy of the learning model 441 is stored in the auxiliary storage unit 13 of the image processing device 1.

[0089] Themes are selected based on end-user attributes, but a learning model may be further generated to suggest recommended combinations based on the selected theme and end-user attributes. This learning model is generated for each theme. Training data is generated from historical data for each theme, including end-user attributes and the recommended combinations selected by the end-users. End-user attributes are the input, and recommended combinations are the ground truth data. End-user attributes are input into the learning model for each theme, and training is performed so that the output of the learning model becomes the recommended combination.

[0090] In this embodiment, the theme is selected by the learning model 441, which makes it possible to increase the accuracy of theme selection, which is an early stage in the design process.

[0091] Figure 40 is a block diagram showing an example of the functional units of an image processing device. The image processing device 1 comprises a classification output unit 11a, a reception unit 11b, a generation unit 11c, an acquisition unit 11d, and an image acquisition unit 11e as functional units. Each of these functional units is realized by the control unit 11 operating based on the control program 1P.

[0092] The classification output unit 11a outputs multiple classifications related to residential interiors. The reception unit 11b The system accepts the selection of a category. The generation unit 11c generates a virtual space that mimics a living space composed of interior components according to the selected category. The acquisition unit 11d acquires a virtual space image viewed from a virtual viewpoint within the generated virtual space. The image acquisition unit 11e outputs the acquired virtual space image.

[0093] (Proximity display function) As mentioned above, when the viewpoint is set to a third viewpoint height (approximately 10cm to 60cm), the display mode allows users to more realistically perceive the texture of the flooring material. The materials whose texture can be confirmed do not have to be limited to flooring. The following describes a proximity display function that allows users to confirm the texture of materials other than flooring.

[0094] The proximity display function switches the texture image applied to the surface of an object in the virtual space to a higher resolution image when the virtual viewpoint's position falls within a predetermined range relative to that object. When the proximity display function is enabled, texture images applied to objects that are not subject to proximity display may be low-resolution or medium-resolution. High resolution, medium resolution, and low resolution should be determined appropriately considering the hardware performance of the user terminal 5. As an example of the settings, 71-96 dpi can be set to low resolution, around 200 dpi to medium resolution, and 350-400 dpi to high resolution.

[0095] Figure 41 is a flowchart showing an example of the procedure for proximity display processing. Proximity display processing is performed in the screen processing shown in Figure 12, after the viewpoint change in step S54 and after image acquisition in step S47. Proximity display processing is also performed in the viewpoint height change processing shown in Figure 15, after image acquisition in step S104. The control unit 11 of the image processing device 1 acquires the viewpoint of the virtual camera (step S211). The control unit 11 selects the object to be processed (step S212). The object is, for example, an object whose value in the cloth material column of the object DB 138 is anything other than "none". The control unit 11 calculates the distance between the viewpoint position and the object (step S213). The control unit 11 determines whether the calculated distance is below a predetermined threshold (step S214). If the control unit 11 determines that the calculated distance is below a predetermined threshold (YES in step S214), it changes the texture attached to the surface of the object to a high-resolution one (step S215). If the control unit 11 determines that the calculated distance exceeds a predetermined threshold (NO in step S214), it determines whether or not there are any unprocessed objects (step S216). If the control unit 11 determines that there are unprocessed objects (YES in step S216), it returns to step S212 and processes the unprocessed objects. If the control unit 11 determines that there are no unprocessed objects (NO in step S216), it terminates the process.

[0096] The proximity display processing is performed after step S47 or step S104, but is not limited to that. It may be processed integrally with image acquisition. As the viewpoint position of the virtual camera changes, the appearance of each object from the viewpoint position is recalculated. During this calculation, the distance between the viewpoint position and the object is calculated, and if the calculated distance is below a threshold, the texture is set to high resolution. This allows for efficient rendering.

[0097] Figure 42 is an explanatory diagram showing an example of close-up display. Figure 42 shows an example of close-up display of flooring material. In close-up display, information such as the model number, color name, material, price, or manufacturer name of the building material being displayed may also be shown.

[0098] The above explanation assumes that the texture of the object to be displayed in close proximity (building materials such as flooring or wall coverings) is changed to high resolution in the walkthrough screen shown in Figure 18, but it is not limited to this. A separate window may be opened from the walkthrough screen, and only the target object may be displayed in 3D in that window. Furthermore, the rotation of the object may be changed in that window. The display may also be able to zoom in and zoom out. Furthermore, it may be possible to check how the display looks in daylight and under indoor lighting at night.

[0099] The system is not limited to changing the texture of an object to a high-resolution one when the distance between the virtual camera's viewpoint and the object falls below a predetermined distance. The user may explicitly specify objects to be displayed in close proximity using a pointing device or the like.

[0100] The proximity display function allows users to examine the texture of individual materials (building materials). When displayed in 3D in a separate window, users can examine the materials as if they were holding them in their hands.

[0101] (Expansion of operation history) Embodiment 2 describes a configuration for storing the operation history of the first user. The operation history is stored in the history DB 433 shown in Figure 32, but the history to be stored may be expanded. In addition to the contents explicitly shown in Figure 32, the history of changes in viewpoint height, day / night switching operation history, etc., may also be stored in the history DB 433. Furthermore, a virtual space image may be stored in association with each operation history. The operation history may be stored each time the user performs an operation, or it may be stored at predetermined intervals.

[0102] Next, we will discuss the use of the extended operation history. Here, we assume the following operation: The users involved are a showroom guide and an interior coordinator (hereinafter also referred to as "coordinator"). The guide is familiar with operating the VR system 100, but the coordinator is not so familiar with it. The coordinator visits the showroom and uses the VR system 100 to create a design with the assistance of the guide. Alternatively, the guide may operate the VR system 100 on behalf of the coordinator. In this case, even if the guide operates the system, the operation history is stored in the history DB 433 as the coordinator's history. Also, if the coordinator is not registered with the VR system 100, user registration is performed and the coordinator is assigned a user ID.

[0103] Figure 43 is an explanatory diagram showing another example of a history database. History DB 433 includes columns for History ID, Date and Time, Sequence Number, User ID, Staff ID, Theme ID, Flooring Material, Wallpaper Material, Change, Viewpoint Position, Viewpoint Height, Day / Night, Image, and Case ID. The History ID column stores an ID that uniquely identifies the history. The Date and Time column stores the date and time the user logged in. The Sequence Number column indicates the sequence number of the history. The User ID column stores the ID of the user, in this case the coordinator. The Staff ID column stores the ID of the guide. The Theme ID, Flooring Material, and Wallpaper Material columns are the same as the columns with the same names shown in Figure 32, so their explanation is omitted. Note that if the flooring material and wallpaper material are changed to different colors all at once by referring to Recommended Combination DB 132, the history is stored in the Flooring Material column and Wallpaper Material column. The Change column stores the change history of wallpaper material (building materials such as flooring and wall coverings). The viewpoint position column stores the history of changes in viewpoint position. The viewpoint position column stores planar coordinates (x coordinate, y coordinate). The viewpoint height column stores the viewpoint height ( The column stores the change history of the z-coordinate. If a pre-set first, second, or third viewpoint height is selected, this is recorded in the viewpoint height column. Otherwise, the viewpoint height column stores the z-coordinate value. The day / night column stores the time period switching history. As mentioned above, during the day, the lighting is represented as if sunlight is pouring in through a window, and at night, the lighting is represented as if lights installed on the ceiling or walls are the light source. Switching between day and night changes the overall color of the virtual space. The image column stores the virtual space image for each history. The image column may store the file name of the image file instead of the actual image data. The case ID column stores an ID that identifies the design case. By associating a series of histories with the same case ID, it becomes possible to determine the series of histories when referring to the history DB433.

[0104] Once the design is complete in the showroom, an ID for the work and a project ID are issued. The project ID may also be issued at the start of the work. The operation history is stored in the history DB433, associated with the user ID and project ID. A QR code is also issued to the coordinator. The QR code includes information such as the user ID, project ID, date, and a URL (Uniform Resource Locator) for accessing the VR system 100. The coordinator is provided with a printout of the QR code, date, and virtual space image. This printout may also be provided to the end user who plans to purchase the house.

[0105] The coordinator can access the VR system 100 using a QR code. The VR system 100 provides the coordinator with a web application. The coordinator can use the VR system 100 using a web browser without installing any new applications on their device.

[0106] Next, we will explain the processing related to the operation history. Figure 44 is a flowchart showing an example of the image display processing procedure. The image display processing is the process by which the coordinator views the content designed in the showroom on the terminal (user terminal) that they are using. The coordinator has the camera (not shown) of the user terminal 5 read the two-dimensional code received in the showroom. The control unit 51 of the user terminal 5 analyzes the two-dimensional code and uses the URL obtained from the analysis to make a login request to the VR system 100 (step S221). The control unit 41 of the management server 4 receives the login request (step S222). The login request includes the user ID, case ID, date, etc. The control unit 41 uses the user ID, case ID, date, etc. included in the login request to read the history from the history DB 433 (step S223). The control unit 41 creates a list of the read history and sends it to the user terminal 5 (step S224). The control unit 51 of the user terminal 5 receives the list and displays it on the display unit 56 (step S225). It is desirable that the list include a virtual space image corresponding to each history entry as a thumbnail. The coordinator selects one of the history entries using the input unit 55. The control unit 51 sends the selection information to the management server 4 (step S226). The control unit 41 of the management server 4 receives the selection information (step S227). The control unit 41 acquires the virtual space image corresponding to the selected history entry and sends it to the user terminal 5 (step S228). The control unit 51 of the user terminal 5 receives the image, displays it on the display unit 56 (step S229), and terminates the process.

[0107] The images displayed during the image display process are snapshot images, and cannot be operated like a walkthrough screen. However, the system can switch to a walkthrough screen upon instruction from the coordinator. In this case, the virtual space to be displayed on the walkthrough screen is configured based on the history. Alternatively, the snapshot images stored as history may be used only as thumbnails in the list view, and a walkthrough screen based on the history may be recreated and displayed. Since the application running on the coordinator's terminal is a web application, the functions that can be performed on the walkthrough screen may be limited.

[0108] In displaying each history, not only the virtual space image may be shown, but also the model number of the building material selected in the virtual space. Furthermore, by linking with catalog information, it may be possible to display the price, form, characteristics, etc. of the building material.

[0109] By reviewing the history, the coordinator can track the progress of design considerations in the showroom. Furthermore, using the virtual space images stored along with the history, the coordinator can create materials for end-users. Additionally, using a walkthrough screen that reflects the history, the coordinator can make design changes.

[0110] Furthermore, the history may be configured to store information about building materials that were displayed in proximity. When referring to the history, include the building materials displayed in close proximity in the list, or create a separate list of building materials displayed in close proximity.

[0111] (Bookmark function) The VR system 100 automatically stores the history, but the bookmark function stores the state of the virtual space at a specific point in time when explicitly instructed by the user. Figure 45 is an explanatory diagram showing an example of the bookmark database. The bookmark database 435 includes columns for bookmark ID, case ID, name, date and time, user ID, assigned ID, theme ID, flooring material, cross material, change, and image. In the bookmark database 435, the columns from date and time to change store the same data as the columns with the same names in the history database 433, so their explanation is omitted. The bookmark ID column stores an ID that can uniquely identify the bookmark. The case ID column stores the ID for each design case handled by the coordinator. The name column stores the name of the bookmark. The name is expected to be entered by the user, but if not entered, the one generated by the management server 4 is stored. The image column stores the image of the virtual space when bookmark storage was instructed. The Case ID column and the Image column are data columns that are also included in the extended history described above. Similar to the History DB433, the Viewpoint Position column, Viewpoint Height column, and Day / Night column may be added to the Bookmark DB435. Note that the instruction to save a bookmark is made by the user selecting the bookmark button. For example, the bookmark button can be added among the top button d021, room select button d022, etc., lined up on the left side of the walkthrough screen d02 shown in Figure 18. On the walkthrough screen d02, a pull-down menu may be displayed when the mouse is right-clicked, and the instruction to save a bookmark may be made from this menu.

[0112] The process for retrieving bookmarks is the same as the image display process shown in Figure 44, so we will omit the explanation. By using the bookmark function, the coordinator can remember the state of the virtual space at a time of their choosing.

[0113] (Recommended) This section explains the recommended building materials display. As shown in Figure 23, there is a wide variety of building materials available for selection on the building materials selection screen. Therefore, even coordinators may have difficulty choosing building materials. The recommended display is designed to help in the selection of building materials. A persona is set for each theme. In addition, recommended building materials are set for each theme. Based on the attributes of the end user, the persona that most closely resembles the end user is determined. Based on the determined persona and the corresponding theme, recommended building materials are identified. The identified building materials are displayed with a mark or icon indicating that they are recommended.

[0114] Figure 46 is an explanatory diagram showing an example of a persona database. Persona DB 139 stores persona information for each theme. Persona DB 139 includes columns for theme ID, age, gender, occupation, family structure, and annual income. The theme ID column stores the theme ID of the theme corresponding to the persona. The age column stores the persona's age. The gender column stores the persona's gender. The occupation column stores the persona's occupation. The family structure column stores the persona's family structure. The annual income column stores the persona's family structure.

[0115] Figure 47 is a flowchart showing an example of the procedure for assigning recommendation marks. The recommendation mark assignment process is performed when the building materials selection screen d07 is generated. The control unit 11 of the image processing device 1 determines whether or not the end user's attributes have been acquired (step S241). If the control unit 11 determines that the end user's attributes have been acquired (YES in step S241), it determines the persona that most closely resembles the end user (step S242). The control unit 11 uses the end user's attributes and the persona's attributes to calculate the similarity between the end user and each persona. The calculation of similarity is possible by publicly known technology, so the explanation is omitted. The persona with the highest similarity is considered the persona that most closely resembles the end user. 11 refers to the persona DB139 to determine the theme corresponding to the persona (step S243). The control unit 11 refers to the recommended combination DB to identify the building materials corresponding to the determined theme (step S244). The control unit 11 assigns a recommendation mark to the identified building materials (step S245) and terminates the process. If the control unit 11 determines that the end user's attributes have not been obtained (NO in step S241), it determines whether a theme has been selected or not (step S246). If the control unit 11 determines that a theme has been selected (YES in step S246), it moves the process to step S244. If the control unit 11 determines that a theme has not been selected (NO in step S246), it terminates the process.

[0116] The method for assigning recommendation marks is not limited to the method described above. Marks may also be assigned to building materials that have a high affinity with the selected theme. The determination of building materials that have a high affinity with a theme is made by analyzing the operation history of multiple users, etc. The correspondence between themes and building materials is then stored in the auxiliary storage unit 13 in advance. Furthermore, based on the user's (coordinator's) operation history, marks may be assigned to building materials that the user is likely to like or that they have selected in the past. In addition, marks may be assigned to building materials that manufacturers, etc., particularly want to promote.

[0117] (Related information displayed) The recommendation section displays recommendations for building materials based on the end user's attributes. The related information section displays information starting from the building materials. This includes information such as the persona corresponding to the building materials and information on furniture that would be a good match with the building materials.

[0118] Figure 48 is an explanatory diagram showing an example of a recommended furniture database. Recommended furniture database 13A stores information on furniture that is considered to be a good combination with building materials. Recommended furniture database 13A includes columns for model number, product name, product number, color, image, and catalog. The model number column stores the model number of the building material. The product name column stores the product name of the recommended furniture. Product names include items such as sofa, dining table, or bed. The product number column stores the product number of the recommended furniture. The color column stores the color of the furniture. The image column stores images of the recommended furniture. The image column may also store the file name of the image file. The catalog column stores catalog information for the recommended furniture or the URL of the website where the catalog information is posted.

[0119] Figure 49 is a flowchart showing an example of the procedure for related information processing. Related information processing is a process that is executed when a user performs an operation such as right-clicking on a building material with the mouse on the walkthrough screen. The control unit 11 of the image processing device 1 receives a command from a predetermined operation (step S251). The control unit 11 identifies the object that the user has operated on from the position of the mouse pointer, etc., and determines whether or not it is the target of related information processing (step S252). If the control unit 11 determines that the identified object is the target of related information processing (YES in step S252), it displays a menu (step S253). The control unit 11 determines whether or not the user has completed selecting a menu (step S254). If the control unit 11 determines that the user has not completed selecting a menu (NO in step S254), it repeats the determination process. If the control unit 11 determines that the user has completed selecting a menu (YES in step S254), it determines whether or not the menu selected by the user is "Catalog" (step S255). If the control unit 11 determines that the menu selected by the user is "Catalog" (YES in step S255), it displays catalog information for the specified object, such as building materials (step S256), and terminates the process. For example, catalog information is obtained from the auxiliary storage unit 13 or the manufacturer's website using the model number of the building materials as the key. If the control unit 11 determines that the menu selected by the user is not "Catalog" (NO in step S255), it determines whether the menu selected by the user is "Persona" (step S257). If the control unit 11 determines that the menu selected by the user is "Persona" (YES in step S257), it displays persona information (step S258), and terminates the process. The control unit 11 obtains persona information in the following procedure. The control unit 11 obtains a theme ID corresponding to the building materials by referring to the recommended combination DB132. Persona information corresponding to the obtained theme ID is obtained from the persona DB139. If the control unit 11 determines that the menu selected by the user is not a "persona" (NO in step S257), it determines whether the menu selected by the user is a "recommended furniture" (step S259). If the control unit 11 determines that the menu selected by the user is a "recommended furniture" (YES in step S259), it obtains and displays information on recommended furniture corresponding to the building materials from the recommended furniture DB13A (step S260) and terminates the process. If the control unit 11 determines that the menu selected by the user is not a "recommended furniture" (NO in step S259), it terminates the process. Also, if the control unit 11 determines that the identified object is not subject to related information processing (NO in step S252), it terminates the process.

[0120] The related information display function shows catalog information for building materials and persona information corresponding to those materials, making it easy for users to select building materials that are appropriate for the end user. Furthermore, information on suitable furniture to combine with the building materials is also available, making it possible to recommend furniture purchases to the end user.

[0121] In the explanation above, it was stated that the related information display function is performed by right-clicking on the building material with the mouse, but this is not the only way. It can also be invoked by clicking the information icon d02B shown in Figure 18.

[0122] (Recording function) In the walkthrough screen d02, by continuously changing the position of the virtual camera, it is possible to obtain a series of images that give the user the impression of walking around in a virtual space. The recording function creates a video by storing these images.

[0123] Figure 50 is a flowchart showing an example of the procedure for video creation. The control unit 11 of the image processing device 1 receives an instruction to start recording (step S271). The control unit 11 starts recording (step S272). The control unit 11 determines whether or not to end the recording (step S273). If the control unit 11 determines not to end the recording (NO in step S273), it repeats the determination. If the control unit 11 determines to end the recording (YES in step S273), it stores the video file in the auxiliary storage unit 13 (step S274) and ends the process. The control unit 11 determines to end the process for example when the user instructs it to end.

[0124] Furthermore, to start / stop recording, for example, a recording button can be added to the top button d021, room select button d022, etc., located on the left side of the walkthrough screen d02 shown in Figure 18, and recording will start when the user selects the recording button. During recording, the recording button will be displayed as an end button, and recording will end when the user selects the end button. In addition to the end button, a pause button may also be displayed during recording to allow pausing and resuming recording. On the walkthrough screen d02, a pull-down menu may be displayed when the mouse is right-clicked, and the start and end of recording may be instructed from this menu.

[0125] The user may manually move the virtual camera's viewpoint using a pointing device such as a 3D mouse, or they may move along a pre-set route at a predetermined speed. In this case, the user must initiate recording to start, but not to end it. Recording will end automatically once the predetermined route is completed, even if the user does not initiate it. Multiple routes may be set up for the user to select from. Different routes may be set for each theme. Furthermore, the system may pause during movement, display a close-up view of a building material specified by the user for a predetermined time, and then resume movement. In addition, the user may be able to specify the viewpoint height and movement speed during movement. This can also be used as an operation history or bookmark. A video may be created by stitching together multiple virtual space images that have been stored.

[0126] When creating a video from a predetermined route, the video creation process may be performed by the management server 4 by stitching together multiple virtual space images. The user can create the video by giving instructions from the user terminal 5. In this case, if there is any missing data in the management server 4, the missing data is obtained from the image processing device 1.

[0127] Figure 51 is an explanatory diagram showing examples of travel routes. Figure 51 shows three routes. Routes 1 and 2 have the same starting point but different ending points. Route 3 has a different starting point than Routes 1 and 2, but the ending point is the same as Route 2. Users may be allowed to change pre-set routes. Users may also be allowed to create new routes. In addition, different routes may be provided for each theme.

[0128] (Presentation board creation function) The presentation board is used when a coordinator proposes their coordination to an end-user (customer). Multiple pieces of information are consolidated on a single screen and displayed on the display unit 56 of the user terminal 5.

[0129] Figure 52 is an explanatory diagram showing an example of a template database. Template DB 437 stores templates for creating presentation boards. Template DB 437 includes columns for Theme ID, Pattern, Area 1, Area 2, Area 3, and Template. The Theme ID column stores the theme ID of the theme corresponding to the template. The Pattern column stores the pattern name of the template. Here, it is referred to as a number. Area 1, Area 2, and Area 3 stores the data content to be embedded in each area. The Image shows a virtual space image. The Description shows a description of the coordination concept. The Building Materials column shows information about the building materials selected in the virtual space. The Template column stores the definition of the template. It stores the size, position, etc. of each area.

[0130] Figure 53 is a flowchart illustrating an example of the procedure for creating a presentation board. The user inputs a request to create a presentation board into the user terminal 5. The control unit 51 of the user terminal 5 sends the presentation board creation request to the management server 4 (step S281). The control unit 41 of the management server 4 receives the presentation board creation request (step S282). The control unit 41 retrieves the user's operation history from the history DB 433 (step S283). The control unit 41 creates a list of history entries and sends it to the user terminal 5 (step S284). The control unit 51 of the user terminal 5 receives the list and displays it on the display unit 56 (step S285). The user selects the history entries to use using the input unit 55. The control unit 51 accepts the selection (step S286). The control unit 51 sends the selection information to the management server 4 (step S287). The control unit 41 of the management server 4 receives the selection information (step S288). The control unit 41 acquires a theme corresponding to the selected history and retrieves a template corresponding to the theme from the template DB 437 (step S289). The control unit 41 embeds data such as virtual space images and building material information included in the history into the template and creates a presentation board (step S290). The control unit 41 sends the presentation board to the user terminal 5 (step S291). The control unit 51 of the user terminal 5 receives the presentation board (step S292). The control unit 51 displays the presentation board on the display unit 56 (step S293). The user inputs a coordination concept explanation, etc., using the input unit 55. The control unit 51 accepts the input (step S294). Note that the input of the concept explanation, etc., may be done after step S286 and sent to the management server 4 in step S287. The control unit 51 embeds the received concept explanation into the presentation board and redisplays the presentation board on the display unit 56 (step S295). The user confirms the displayed content and inputs the necessary commands. Commands include "Edit" and "Remember and End". The command is "Finished," etc. The control unit 51 accepts the command (step S296). The control unit 51 determines whether the received command is finished or not (step S297). If the control unit 51 determines that the command is not finished (NO in step S297), it executes the command (step S298) and returns the process to step S295. Depending on the content of the command, the process may also return to step 285. If the control unit 51 determines that the command is finished (YES in step S297), it stores the presentation board in the auxiliary storage unit 53 (step S299) and terminates the process.

[0131] Figure 54 is an explanatory diagram showing examples of templates. Figure 54 shows three types of templates. Each template has three types of embedded information, and the information types are the same. The difference between the templates lies in the position and size balance of the display areas. A virtual space image is embedded in area 521. A coordinator's concept explanation is embedded in area 522. An image of the selected building material is embedded in area 523. Information displayed in area 523 may also include catalog information of the building material (model number, price, form, features, etc.).

[0132] The presentation board creation function allows coordinators to easily create presentation boards for clients. Furthermore, the VR system 100 collects the necessary information according to the template, saving coordinators the time and effort of gathering information themselves.

[0133] (Centralized data management) In the above-described embodiment, the image processing device 1 or the management server 4 stores data for constructing the virtual 3D space, user data, and user operation history data, but it is not limited to this. To facilitate the management, updating, and sharing of this data, a centrally managed database server may be provided. Furthermore, a terminal for sales personnel to introduce the VR system 100 may also be included as part of the configuration of the VR system 100.

[0134] Figure 55 is an explanatory diagram showing another configuration example of the VR system. The VR system 100 includes an image processing device 1, a management server 4, a user terminal 5, a database server 6, and a salesperson terminal 7. The image processing device 1, management server 4, user terminal 5, database server 6, and salesperson terminal 7 are connected to each other via a network N, enabling them to communicate with one another.

[0135] The configuration of the image processing device 1, the management server 4, and the user terminal 5 is the same as in Embodiment 2, so a detailed explanation will be omitted.

[0136] The database server 6 consists of a server computer, workstation, etc. The database server 6 may also consist of a multicomputer system with multiple computers, a virtual machine virtually constructed by software, or a quantum computer. The database server 6 stores data for constructing a virtual 3D space, user information, and user operation history. In the above embodiment, the image processing device 1 stores the data for constructing the virtual 3D space. The management server 4 stores user information and user operation history. Here, the database server 6 stores this data.

[0137] Salesperson terminal 7 consists of a notebook PC or tablet computer. Salesperson terminal 7 is primarily used by salespeople who promote building materials. Salespeople promote building materials by introducing the VR system 100 to coordinators. Salesperson terminal 7 stores explanatory materials for the VR system 100.

[0138] Image processing device 1 will be a computer with higher specifications than user terminal 5 and salesperson terminal 7. Image processing device 1 will be used by showroom guides who are familiar with operating the VR system 100. User terminal 5 will be used by coordinators.

[0139] The image processing device 1, salesperson terminal 7, and user terminal 5 offer different functionalities as follows: All functions of the VR system 100 are available on the image processing device 1. The salesperson terminal 7 allows demonstrations of the VR system 100. On the salesperson terminal 7, users can select a theme and view virtual space images designed for each selected theme. However, the viewpoint position of the virtual camera can only be selected from multiple pre-set positions. The user terminal 5 is provided with a web application. The user terminal 5 primarily features the ability to create presentation boards using the design results from the image processing device 1.

[0140] The image processing device 1, management server 4, and database server 6 constitute the showroom system as a subsystem. The salesperson terminal 7, management server 4, and database server 6 constitute the salesperson system as a subsystem. The user terminal 5, management server 4, and database server 6 constitute the coordinator system as a subsystem. Because the database server 6 centrally manages the data used by the VR system 100, it is possible to share and link data between any of the systems, such as the showroom system, salesperson system, or coordinator system.

[0141] The technical features (constituent elements) described in each embodiment are combinable with each other, and by combining them, new technical features can be formed. The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims, not in the sense described above, and all modifications within the sense and scope equivalent to the claims are intended. [Explanation of Symbols]

[0142] 100 VR systems 1 Image Processing Device 11 Control Unit 11a Classification output section 11b Reception Desk 11c Generator 11d Acquisition Department 11e Image acquisition section 12 Main memory 13 Auxiliary storage 131 Theme Database 132 Recommended combination DB 133 Texture DB 134 Room Database 135 Vertex DB 136-sided DB 137 surface DB 138 Object DB 139 Persona DB 13A Recommended Furniture Database 15 Input Interfaces 16 Display I / F 17 Communications Department 18 Reading section 1P Control Program 1a Portable storage medium 1b Semiconductor memory 2 Controllers 3 displays 4. Management Server 41 Control Unit 42 Main memory 43 Auxiliary storage section 431 User DB 432 Case DB 433 History DB 434 Training Data DB 435 Bookmark DB 437 Template DB 441 Learning Models 45 Communications Department 4P control program 5. User terminals 51 Control Unit 52 Main memory 53 Auxiliary storage section 54 Communications Department 55 Input section 56 Display section 5P control program B Bus N Network

Claims

1. Output multiple classifications related to residential interiors. We accept the selection of a category. A virtual space is generated that mimics a living space, composed of interior components according to the selected classification. Obtain a virtual space image from a virtual viewpoint within the generated virtual space. The acquired virtual space image is output to a screen that allows for a walkthrough within the virtual space. Select one of the interior components in the virtual space, If the virtual viewpoint is set within a predetermined distance from the selected interior component, information about the selected interior component will be displayed. An image output method characterized by having a computer perform the processing.

2. If the virtual viewpoint is set within a predetermined distance from the selected interior component, the texture image of the selected interior component is switched to a high-resolution texture image. The image output method according to claim 1, characterized in that the processing is performed by the computer.

3. A first classification output unit that outputs multiple classifications related to residential interiors, A reception desk that accepts classification selections, A first generation unit generates a virtual space that mimics a living space composed of interior components according to the selected classification, A setting unit that sets the virtual viewpoint within the generated virtual space according to the instructions from the first user, An acquisition unit that acquires a virtual space image viewed from the aforementioned virtual viewpoint, A first image output unit outputs the acquired virtual space image to a screen that allows for a walkthrough within the virtual space, A selection unit for selecting any of the interior components within the virtual space, When the virtual viewpoint is set within a predetermined distance from the selected interior component, a display unit that displays information about the selected interior component is provided. A first system having, An image output system characterized by comprising the following features.

4. Before the first system, A classification acquisition unit that obtains a classification selected by the first user from multiple classifications related to residential interiors, A second generation unit generates a virtual space that mimics a living space composed of interior components corresponding to the acquired classification, A selection reception unit that accepts selections from a second user for multiple pre-configured virtual viewpoints within a virtual space, A second acquisition unit acquires a virtual space image viewed from the aforementioned virtual viewpoint, A second image output unit outputs the acquired virtual space image. The second system having The image output system according to claim 3, characterized by comprising the following:

5. A classification output unit that outputs multiple classifications related to residential interiors, A reception desk that accepts classification selections, A generation unit that generates a virtual space that mimics a living space composed of interior components according to the selected classification, An acquisition unit that acquires a virtual space image as seen from a virtual viewpoint within the generated virtual space, An image output unit that outputs the acquired virtual space image to a screen that allows a walkthrough within the virtual space, A selection unit for selecting any of the interior components within the virtual space, When the virtual viewpoint is set within a predetermined distance from the selected interior component, a display unit that displays information about the selected interior component is provided. An image output device characterized by comprising the following features.

6. Output multiple classifications related to residential interiors. We accept the selection of a category. A virtual space is generated that mimics a living space, composed of interior components according to the selected classification. Obtain a virtual space image from a virtual viewpoint within the generated virtual space. The acquired virtual space image is output to a screen that allows for a walkthrough within the virtual space. Select one of the interior components in the virtual space, If the virtual viewpoint is set within a predetermined distance from the selected interior component, information about the selected interior component will be displayed. An image output program characterized by having a computer perform the processing.

7. A first classification output unit that outputs multiple classifications related to residential interiors, The first reception area accepts the selection of classification, A first generation unit generates a virtual space that mimics a living space composed of interior components according to the selected classification, A settings unit that sets the virtual viewpoint within the generated virtual space according to the user's instructions, An acquisition unit that acquires a virtual space image viewed from the aforementioned virtual viewpoint, A first image output unit outputs the acquired virtual space image to a screen that allows for a walkthrough within the virtual space, A selection unit for selecting any of the interior components within the virtual space, When the virtual viewpoint is set within a predetermined distance from the selected interior component, a display unit is provided that displays information about the selected interior component. An ID acquisition unit that acquires identification information assigned to the coordinator, A storage unit that stores the acquired identification information and the virtual space image in association with each other, A result output unit that outputs the acquired identification information and the virtual space image. A showroom system having A second classification output unit that outputs multiple classifications related to residential interiors, A second reception area where classification selections are accepted, A second generation unit generates a virtual space that mimics a living space composed of interior components according to the selected classification, A second acquisition unit that acquires a virtual space image viewed from a predetermined virtual viewpoint, A second image output unit outputs the acquired virtual space image to a screen that allows for a walkthrough within the virtual space. A sales representative system having, A third receiving unit that receives the aforementioned identification information, A third image output unit retrieves the virtual space image corresponding to the received identification information from the storage unit and outputs it to a screen that allows walking through the virtual space. Coordinator system with An image output system characterized by comprising the following features.

Images