Interior-layout assistance method, interior-layout assistance system, and program

Abstract

The present invention provides an interior-layout support method which is convenient for considering the layout of products in a room. The method includes: a first step of acquiring or generating 3D CAD data of a room; a second step of specifying, by a user, a classification and budget of commercial goods that are at least one of furniture and home appliances to be placed in the room; a third step of determining, on the basis of the CAD data acquired or generated in the first step and the classification and budget of commercial goods specified in the second step, a plurality of placement candidates each indicating a different combination of a space in the room, where commercial goods can be placed, and commercial goods to be placed in the space; and a fourth step of displaying a 3D CAD image, in which the commercial goods are placed in the space where commercial goods can be placed, for each of the plurality of placement candidates determined in the third step.

Description

Indoor layout support method, system and program

[0001] The present invention relates to an indoor layout support method, system, and program used to consider the layout of products in a room.

[0002] For example, in real estate-related businesses such as construction, renovation, and moving, safety and accuracy are highly required, and drawing blueprints and site floor plans is one of the tasks that is required. These have traditionally been created by architects and CAD experts. In this case, the site is first surveyed, and then a scaled plan is created based on the survey results. Then, technical symbols, numbers, materials, colors, etc. are added to the blueprint to complete the work, which takes about one to two weeks.

[0003] Japanese Patent Application Laid-Open No. 2021-012528

[0004] The resulting blueprints and instructions are then visualized in the minds of those who view them, and the work is carried out according to the dimensions, including the construction, wiring, installation, and coloring. Because a wide variety of businesses are involved in building or renovating a building, the work must proceed carefully, with each business checking with the other to ensure that the three-dimensional image in each person's mind remains consistent. Furthermore, each business must carry dozens of paper blueprints and instructions, resulting in a huge amount of paper being required. As such, traditional work sites have the problem of wasting manpower, time, and resources. There is also a demand for the efficient selection and purchase of commercial materials such as home appliances and furniture based on interior CAD data.

[0005] The present invention has been made in consideration of the above circumstances, and its purpose is to provide an indoor layout support method, system, and program that are convenient for considering where to place commercial materials and selecting commercial materials to purchase when placing them in a room.

[0006] The present invention includes a first step of acquiring or generating 3D CAD data of a room; a second step of receiving from a user a specification of the classification of at least one of furniture and home appliances to be placed in the room and a budget for the classification; a third step of determining a plurality of placement candidates showing different combinations of spaces in the room where the commodities can be placed and the commodities to be placed in the spaces, based on the CAD data acquired or generated in the first step and the classification and budget of the commodities specified in the second step; a fourth step of displaying a 3D CAD image of the commodities placed in the available space for each of the plurality of placement candidates determined in the third step; and a fifth step of displaying a selection screen for selecting other commodities of the same classification as the displayed commodities when the user specifies a commodity on the CAD image displayed in the fourth step, and generating the CAD data in which the other commodities are placed when the other commodities are selected on the selection screen. and a sixth step of storing in memory as history data the CAD data of the interior of a user who has previously purchased merchandise, the placement candidates selected by the user, and the merchandise purchased by the user in association with each other; the third step of determining, based on the history data stored in memory in the sixth step, the placement candidates and merchandise selected by another user who has a combination highly similar to the CAD data of the interior of the room, the specified combination of the category and its budget, as the placement candidates and merchandise with high priority; and the third step of determining the placement candidates and merchandise using a learning model trained by machine learning using as training data a combination of the CAD data of the interior of a user who has previously purchased merchandise, the placement candidates selected by the user, the merchandise purchased by the user, and the attributes of the user.

[0007] The present invention includes a first step of acquiring or generating 3D CAD data of a room; a second step of receiving from a user a specification of the classification of at least one of furniture and home appliances to be placed in the room and a budget for that classification; a third step of determining, based on the CAD data acquired or generated in the first step and the classification and budget for the items specified in the second step, a plurality of placement options showing different combinations of spaces in the room where the items can be placed and the items to be placed in those spaces; a fourth step of displaying, for each of the plurality of placement options determined in the third step, a 3D CAD image of the items placed in the available spaces; and when a user specifies a product on the CAD image displayed in the fourth step, a selection screen is displayed to allow the user to select another product of the same classification as the displayed product, and when another product is selected on the selection screen, The computer executes a fifth step of generating the CAD data in which the other commercial items are placed, and a sixth step of associating the CAD data of the interior of a user who has previously purchased commercial items, the commercial items purchased by the user, and the prices of the purchased items in memory as historical data, and storing these in memory. The third step determines the placement candidates and commercial items using a learning model that has been machine-learned using training data that is a combination of the CAD data of the interior of a user who has previously purchased commercial items and the commercial items purchased by the user, and displays, from among the determined placement candidates and commercial items, placement candidates and commercial items selected by other users that have a combination that is highly similar to the CAD data of the interior of the room, the specified classification, and its budget based on the historical data stored in memory in the sixth step, as placement candidates and commercial items with high priority.

[0008] The present invention also provides a system including a first step of acquiring or generating 3D CAD data of a room; a second step of receiving from a user a designation of the classification of at least one of furniture and home appliances to be placed in the room and a budget for the classification; a third step of determining, based on the CAD data acquired or generated in the first step and the classification and budget for the items designated in the second step, a plurality of arrangement candidates showing different combinations of spaces in the room where the items can be placed and the items to be placed in the spaces; a fourth step of displaying, for each of the plurality of arrangement candidates determined in the third step, a 3D CAD image in which the items are placed in the available spaces; and, when a user designates a product on the CAD image displayed in the fourth step, displaying a selection screen for selecting another product of the same classification as the displayed product, and when another product is selected on the selection screen, displaying the CAD image in which the other product is placed. and a sixth step of associating the CAD data of the interior of a room of a user who has previously purchased a product, the placement candidate selected by the user, and the product purchased by the user, and storing the associated data in memory as historical data. The third step determines, based on the historical data stored in memory in the sixth step, the placement candidate and product selected by another user who has a combination highly similar to the CAD data of the interior of the room, the specified category, and its budget, as the placement candidate and product with high priority. The third step determines the placement candidate and product using a learning model trained by machine learning using as training data a combination of the CAD data of the interior of a room of a user who has previously purchased a product, the placement candidate selected by the user, the product purchased by the user, and the user's attributes.The present invention also provides a system including a first step of acquiring or generating 3D CAD data of a room; a second step of receiving from a user a designation of the classification of at least one of furniture and home appliances to be placed in the room and a budget for the classification; a third step of determining, based on the CAD data acquired or generated in the first step and the classification and budget for the items designated in the second step, a plurality of placement candidates showing different combinations of spaces in the room where the items can be placed and the items to be placed in the spaces; a fourth step of displaying, for each of the plurality of placement candidates determined in the third step, a 3D CAD image in which the items are placed in the available spaces; and, when a user designates a product on the CAD image displayed in the fourth step, displaying a selection screen for selecting another product of the same classification as the displayed product, and when another product is selected on the selection screen, displaying the other product. and a sixth step of associating the CAD data of the interior of a room of a user who has previously purchased a commodity with the commodity purchased by the user and the price of the purchased commodity in memory as historical data, wherein the third step determines the placement candidates and the commodity using a learning model that has been machine-learned using training data that is a combination of the CAD data of the interior of a room of a user who has previously purchased a commodity and the commodity purchased by the user, and displays, from the determined placement candidates and commodity, placement candidates and commodity selected by another user that have a combination that is highly similar to the CAD data of the interior of the room, the specified classification, and a combination of its budget, based on the historical data stored in memory in the sixth step, as placement candidates and commodity with high priority.

[0009] According to the present invention, it is possible to provide an indoor layout support method, system, and program that are convenient for considering where to place commercial materials and selecting commercial materials to purchase when placing them in a room.

[0010] FIG. 1 is a functional block diagram of a communication terminal device according to an embodiment of the present invention. FIG. 2 is a functional block diagram of a server device according to an embodiment of the present invention. FIG. 3 is a flowchart illustrating the overall flow of an indoor layout support method according to a first embodiment of the present invention. FIG. 4 is a flowchart illustrating the flow of a commercial material purchasing operation, such as flooring, in the indoor layout support method according to the first embodiment of the present invention. FIG. 5 is a diagram illustrating a second modified example in which a trained model is used to determine placement candidates and purchase candidate commercial materials. FIG. 6 is a diagram illustrating a trained model used in a third modified example. FIG. 7 is a flowchart illustrating details of a method for generating a 3D image of the room shown in FIG. 3. FIG. 8 is a flowchart illustrating the overall flow of an indoor layout support method according to a second embodiment of the present invention. FIG. 9 is a flowchart illustrating the overall flow of an indoor layout support method according to a third embodiment of the present invention.

[0011] First Embodiment The indoor layout support method of this embodiment can be implemented using a communication terminal device 11 such as a smartphone. Although 3D modeling apps using lidar already exist, this method is differentiated by its ability to measure in millimeters. The dimensions of the photographed room are displayed on the management screen.

[0012] When renovating a room, the construction industry requires staff to create blueprints and take measurements manually using tape measures, which increases costs. The indoor layout support method of this embodiment can improve the efficiency of these tasks.

[0013] In the indoor layout support method of this embodiment, a communication terminal device scans a room with a laser and a camera to generate 3D CAD data of the room, and then identifies the size of the room based on the CAD data. For example, in the indoor layout support method of this embodiment, a 3D CAD image of the room is displayed based on the CAD data. The identified size of the room is displayed in the 3D CAD image. For example, as shown in Figures 1 to 3, the ceiling height, length and width of the floor, etc. of the room are displayed in the 3D image.

[0014] 1 is a functional block diagram of a communication terminal device 11 according to an embodiment of the present invention. As shown in FIG. 1, the communication terminal device 11 includes, for example, a laser 50, a camera 51, a display 52, an operation unit 53, a communication unit 55, a Bluetooth (registered trademark) communication unit 57, a memory 59, and a processing unit 61.

[0015] The laser 50 emits a laser beam into the room while the user does not move around the room and receives the reflected light. The camera 51 captures images of the room while the user does not move around the room and generates image data. The measurement results acquired by the laser 50 and the image results of the camera 51 are managed so that they can be synchronized.

[0016] The display 51 displays an image based on a signal from the processing unit 61. The operation unit 53 is an operation means such as a touch panel, keyboard, or mouse. The communication unit 55 communicates with the video distribution site server device 11 and the management server device 21. The Bluetooth (registered trademark) communication unit 57 communicates with external devices via Bluetooth (registered trademark). The memory 59 stores a program PRG1 executed by the processing unit 61. The processing unit 61 executes the program PRG1 stored in the memory 59 to perform the processing of the communication terminal device 11 defined in this embodiment.

[0017] 2 is a functional block diagram of a server device 21 according to an embodiment of the present invention. As shown in FIG. 2, the server device 21 includes, for example, a communication unit 75, a memory 79, and a processing unit 81. The memory 79 stores a program PRG2 to be executed by the processing unit 81. The processing unit 81 executes the program PRG2 stored in the memory 79 to perform the functions of the server device 21 defined in this embodiment.

[0018] The flow of the indoor layout support method of this embodiment will be described below. <Overall Operation> Fig. 3 is a flowchart for explaining the overall flow of the indoor layout support method of the first embodiment of the present invention. Each step will be described. A part or all of the processing by the processing unit 61 of the communication terminal device 11 described in this specification may be executed by the processing unit 81 of the server device 21.

[0019] Step ST11: Under control of the processing unit 61, the communication terminal device 11 irradiates the room with a laser from the laser 50 to scan the room, and generates 3D image data of the room based on the scan results and the image capture results of the camera 51. The processing unit 61 then converts the 3D image data into 3D CAD data. The processing unit 61 then displays a 3D CAD image of the room corresponding to the 3D CAD data on the display 52. ​​For example, the CAD image shown in FIG. 3 is displayed. Here, the 3D image data is data obtained by combining mesh data and the camera image capture results. The CAD data is data in a CAD format that has been neatly converted into computer graphics and does not include mesh or photographic data. Note that the communication terminal device 11 may acquire CAD data from an external source rather than generating it.

[0020] At this time, the processing unit 61 determines which areas in the room have already acquired point clouds and which areas have not, based on the indoor position of the communication terminal device 11 and the image captured by the camera 51. The processing unit 61 then displays the areas in which point clouds have already been acquired and the areas in which point clouds have not been acquired in different colors on the display 52. ​​This makes it possible to acquire point clouds for all necessary areas in the room.

[0021] Step ST12: The user operates the operation unit 53 to specify and select the classification (type) of commercial materials, such as furniture and home appliances, to be placed in the target room. For example, the processing unit 61 displays a product classification selection field on the display 52, from which the user may select, or the user may directly input the product classification. The product classification may be a product category such as a refrigerator, shelf, bookcase, sofa, etc.

[0022] Step ST13: Based on the CAD data generated or acquired in step ST11 and the type and budget of the product specified or selected in step ST12, the processing unit 61 determines multiple placement candidates that indicate different combinations of spaces in the room where the product can be placed and the product to be placed in those spaces. Each placement candidate has a different product and a different position for placing the product. In this case, candidates for placing different products may be different placement candidates even if the placement of the product type is the same.

[0023] The processing unit 61 identifies the size of the space in the room where the product can be placed based on the CAD data in step ST11. Specifically, it identifies the width, depth, height, etc. Furthermore, the processing unit 61 acquires from an external server, based on the budget and the identified size, whether or not there is a product that can be placed in the space in the room, and if so, data on the product. For example, the processing unit 61 accesses an external server to search for product candidates within the budget based on the product classification specified in step ST12, and acquires the size of the product candidate. Next, the processing unit 61 identifies spaces in the room where the product candidate of the acquired size can be placed. If there are multiple identified spaces, it determines a placement candidate for each of them. If multiple product classifications are specified, it determines a placement space for each product. In this case, it determines placement candidates so that products of different classifications are not placed in the same space within a single placement candidate.

[0024] Step ST14: The processing unit 61 performs a process of displaying a 3D CAD image of the product arranged in the available space for each of the multiple placement candidates determined in step ST13. The user reviews and decides which pattern (combination of products and their placement) is best while viewing the CAD images of the multiple placement candidates. When the user operates the operation unit 53 to specify a product on the CAD image, the processing unit 61 displays on the display 52 a selection screen for selecting another product in the same category as the displayed product. When another product is selected on the selection screen, the processing unit 61 generates CAD data in which the other product is arranged and displays a CAD image based on the generated data.

[0025] Step ST15: When the user operates the operation unit 53 to request the purchase of the product selected on the CAD image displayed in step ST14, the processing unit 61 proceeds to step ST16.

[0026] Step ST16: When the purchase request is input in step ST15, the processing unit 61 performs the purchase process for the product selected by the user. The communication terminal device 11 accesses the server device 21 or another product sales site via the communication unit 55 to perform the purchase process.

[0027] <Purchasing Operation of Commercial Materials, Such as Flooring> Fig. 4 is a flowchart for explaining the flow of purchasing operation of commercial materials, such as flooring, in the indoor layout support method according to the first embodiment of the present invention. Each step will be described.

[0028] Step ST21: The user operates the operation unit 53 to specify commercial materials such as walls and flooring in the room on the CAD image of the room displayed on the display 52 based on the CAD data generated in step ST11 shown in Figure 3.

[0029] Step ST22: The processing unit 61 identifies the classification (type) of the product specified in step ST21 and acquires vendor information indicating a vendor that provides the product and that is pre-associated with the product classification. The processing unit 61 then provides CAD data corresponding to the specified CAD image to the vendor's communication device based on the acquired vendor information. The CAD data may be provided by sending the CAD data to the vendor's email address or by uploading it to a system accessible by the vendor.

[0030] Step ST23: Based on the CAD data acquired in step ST22, the contractor transmits product data (candidate product data) of multiple candidate products to the communication terminal device 11. The processing unit 61 stores the multiple candidate product data received from the contractor in the memory 59. The communication terminal device 11 may also acquire from the contractor the product data, as well as the price of the product and construction cost estimate information, and display them on the display 52.

[0031] Step ST24: Based on the CAD data of the room and the candidate product data stored in memory 59 in step ST23, processing unit 61 generates new CAD data incorporating images corresponding to the candidate product data into the product locations, and generates corresponding CAD images. The generation of the CAD images is performed for each of the multiple candidate product data.

[0032] Step ST25: When the user operates the operation unit 53 to make a purchase request for the product selected on the CAD image displayed in step ST23, the processing unit 61 proceeds to step ST25.

[0033] Step ST26: When the purchase request is input in step ST24, the processing unit 61 performs a purchase process for the product selected by the user.

[0034] <First variant of first embodiment> After the processing of the first embodiment described above, the processing unit 61 associates the CAD data of the interior of a user who has previously purchased a commercial product with the placement candidate selected by the user and the commercial product purchased by the user, and stores the data in the memory 59 as history data.

[0035] In step ST14 shown in FIG. 3, when the processing unit 61 displays CAD images of multiple candidates on the display 52, it determines, based on the history data stored in the memory 59, placement candidates and commercial products selected by other users that have a combination that is highly similar to the indoor CAD data generated or acquired in step ST11 and the combination of the product classification and its budget specified in step ST12, as placement candidates and commercial products with high priority.

[0036] A high degree of similarity is, for example, the greater the number of commonalities in the area or shape of the room based on the CAD data, the greater the number of commonalities in the specified product classification, or the closer the budget. Then, in step ST13, placement candidates and products with a high degree of similarity are preferentially displayed on the display 52 as CAD images. Displaying with a high degree of similarity may involve indicating the priority using numbers or the like, or displaying on the screen in order of priority, or by placing them at the front of the display order. This allows placement candidates and products that the user is more likely to select to be displayed preferentially, allowing the user to make an efficient decision.

[0037] Second Modification of First Embodiment In step ST13 shown in FIG. 3, the processing unit 61 may determine placement candidates and purchase candidate commodities using the trained model.

[0038] FIG. 5 is a diagram illustrating a second modified example in which placement candidates and purchase candidate products are determined using a trained model. That is, the processing unit 61 performs the process of step ST13 shown in FIG. 3 using the trained model. The trained model is obtained by machine learning using a combination of CAD data of the interior of a user who previously purchased a product, placement candidates selected by the user, products purchased by the user, and the user's attributes as training data. The processing unit 61 may create the trained model, or may re-train it based on newly acquired information. Alternatively, the trained model may be generated by another organization.

[0039] The processing unit 61 inputs the CAD data, the user's attributes, and the trained model and outputs information on placement candidates and purchase candidate products. The output may be only the highest priority information, or multiple prioritized outputs may be generated. In step ST14, the processing unit 61 generates candidate CAD images based on the output of the trained model and the CAD data. In this way, by using the trained model, it is possible to determine candidates suitable for the user's attributes, etc., as the amount of training data increases. This second variant may also combine the functions of the first variant described above. By adopting the trained model in this way, it is possible to display more appropriate placement candidates and purchase candidates to the user as the track record increases.

[0040] <Third variant of first embodiment> In the third variant, the processing unit 61 performs the processing of the first embodiment described above, and then associates the CAD data of the interior of a user who has previously purchased commercial materials with the commercial materials purchased by the user and the prices of the purchased products, and stores the data in the memory 59 as history data.

[0041] In addition, in step ST13 shown in FIG. 3 , the processing unit 61 determines placement candidates and products using a learning model that has been machine-learned using the combination of the CAD data of the interior of a user who has previously purchased a product and the product purchased by the user as training data. By adopting this learning model, more appropriate placement candidates and purchase candidates can be displayed to the user as the track record increases. FIG. 6 is a diagram illustrating the learning model used in the third modified example. As shown in FIG. 6 , the learning model inputs the CAD data of the user's interior and product data specified by the user, and outputs product placement candidates and purchase candidate products.

[0042] Next, the processing unit 61 performs processing to determine and display, as high-priority placement candidates and products, placement candidates and products selected by other users that have a combination that is highly similar to the combination of the indoor CAD data, the classification specified by the user, and the budget thereof, based on the history data stored in memory 59 in step ST16 shown in Fig. 3. A high degree of similarity means, for example, that there are many common points in the indoor area and shape based on the CAD data, there are many common points in the specified product classification and budget (purchase price), and the closer the budget, the higher the priority.

[0043] The processing unit 61 preferentially displays placement candidates and commercial materials with high similarity as CAD images on the display 52. ​​Displaying with high priority may mean indicating the priority using numbers or the like, or displaying on the screen in descending order of priority, or putting them at the front of the display order, etc. This allows placement candidates and commercial materials that the user is more likely to select to be preferentially displayed, allowing the user to make an efficient decision.

[0044] The third modification is identical to the first embodiment, except for the details described in the third modification. According to the third modification, in addition to using a learning model to determine placement candidates and products that best suit the user's situation, placement candidates and products selected by the user that have a high similarity to the CAD data of the room, the specified classification, and the combination of their budget are determined as "high priority" placement candidates and products. In other words, the determination of candidates and high priority candidates is performed in two stages. This makes it easier for users to select placement candidates and products that best suit them.

[0045] The following describes a method for generating 3D CAD data of an indoor space using the communication terminal device 11 in step ST11 shown in Fig. 3. Fig. 7 is a flowchart for explaining the details of the method for generating a 3D image of an indoor space shown in Fig. 3. Each step in Fig. 4 will be described below.

[0046] Step ST41: The processing unit 61 of the communication terminal device 11 identifies (estimates) the indoor position of the communication terminal device 11. For example, a self-position estimation technique using LiDAR is used as the self-position estimation.

[0047] Step ST42: The user moves the communication terminal device 11 to irradiate the entire room or a part of the room with a laser beam from the laser 50, and detects the reflected laser beam. The processing unit 61 then generates point cloud data of the room and the objects in the room based on the detection results. At the same time, the camera 51 captures an image of the laser irradiation position of the laser 50.

[0048] Step ST43: The processing unit 61 converts the point cloud data generated in step ST42 from the coordinate system of the position of the communication terminal device 11 identified in step ST41 into an indoor coordinate system.

[0049] Step ST44: The processing unit 61 extracts valid data from the point cloud data converted in step ST43 (deletes unnecessary data). This extraction process is performed by deleting point clouds that have the same or similar attributes as point clouds that have been previously identified as unnecessary.

[0050] Step ST45: The processing section 61 converts the point cloud data that has been subjected to the extraction process in step ST44 into mesh (polygon) data.

[0051] Step ST46: The processing unit 61 converts the mesh generated in step ST45 into surface data using a predetermined plane detection technique.

[0052] Step ST47: Based on the surface data generated in step ST46, the processing unit 61 generates 3D image data (3D image) by pasting the image captured by the camera 51 acquired in step ST42 onto the mesh.

[0053] Step ST48: The processing unit 61 converts the 3D image data generated in step ST47 into the 3D CAD data and displays it on the display 52. ​​The processing unit 61 identifies (measures) the size of the room (height, length, width, and layout) based on the 3D CAD data, and displays the dimensions in the 3D CAD image. Furthermore, since CAD data can be generated by recognizing walls through the above-mentioned process, a 3D image of the layout without furniture, etc. in the room can be generated.

[0054] Note that a learning model may be employed in some steps of the above-described processing.

[0055] According to the indoor layout support method of this embodiment, by using a communication terminal device 11 such as a smartphone, it is possible to instantly and easily create a 3D image anywhere with high accuracy, which leads to the utilization and general dissemination of 3D technology in various fields. For example, it can be used in real estate-related businesses such as renovation companies, construction companies, and moving companies.

[0056] Furthermore, the indoor layout support method of this embodiment can alleviate labor shortages in the construction industry and improve the experience during renovations. In other words, if 3D scanners can replace the manual surveying and blueprinting that have been performed up until now, labor can be allocated to other more advanced tasks, enabling the provision of higher value-added products and leading to cost reductions.

[0057] It also creates an environment where workers can concentrate on the tasks they are supposed to be doing, and it makes it possible to simplify and clarify the craftsmanship that previously could only be learned through experience. It also contributes to the recruitment and development of young talent who will lead the future.

[0058] 3D scanning technology makes surveying much easier and less time-consuming. Furthermore, by being able to colorize the 3D images, insert photos, and view them from all angles, it frees people from having only mental images, allowing everyone involved to work with the same image of the finished product. Being able to manage everything on a device also reduces the amount of unnecessary paper being produced. These efficiencies are ultimately passed on to consumers, allowing buildings and their surroundings to be improved at a more cost-effective pace than before.

[0059] Furthermore, according to this embodiment, an app has been developed that can create 3D models that can be measured with high accuracy and quality using only a smartphone (communication terminal device 11) equipped with LiDAR, allowing anyone to easily and inexpensively perform 3D scanning.

[0060] According to this embodiment, measurable 3D models and photographic data can be acquired and saved simply by taking indoor video using the communication terminal device 11. Furthermore, according to this embodiment, data can be output to a personal computer and shared in various formats, such as point clouds and meshes. Furthermore, the communication terminal device 11, through the processing shown in FIG. 4, enables more accurate measurements than a tape measure.

[0061] Furthermore, according to this embodiment, multiple different patterns of placement candidates can be provided to the user depending on the type of merchandise the user wishes to purchase and their budget, allowing the user to consider placement candidates that the user would not have thought of themselves.Since candidate merchandise can be determined along with multiple placement candidates and CAD images with images of the merchandise placed in each placement candidate can be provided to the user, the user can have a more concrete image of the merchandise placed in each placement candidate based on the CAD image, and it is possible to avoid a gap between the image and reality.

[0062] Furthermore, according to this embodiment, the user simply specifies a commercial material such as flooring on a CAD image, and based on the commercial material data from the commercial material provider, the user can easily grasp from the CAD image what the interior will look like when that material is used.

[0063] Second Embodiment Fig. 8 is a flowchart illustrating the overall flow of an indoor layout support method according to a second embodiment of the present invention. Each step will be described below. A part or all of the processing performed by the processing unit 61 of the communication terminal device 11 described in this specification may be executed by the processing unit 81 of the server device 21.

[0064] Step ST51: Under control of the processing unit 61, the communication terminal device 11 irradiates the room with a laser from the laser 50 to scan the room, and generates 3D image data of the room based on the scan results and the image capture results of the camera 51. The processing unit 61 then converts the 3D image data into 3D CAD data. The processing unit 61 displays a 3D CAD image of the room corresponding to the 3D CAD data on the display 52. ​​For example, the CAD image shown in FIG. 3 is displayed. Here, the 3D image data is data obtained by combining mesh data and the camera image capture results. The CAD data is data in a CAD format that has been neatly converted into computer graphics and does not include mesh or photographic data. Note that the communication terminal device 11 may acquire CAD data from an external source rather than generating it.

[0065] At this time, the processing unit 61 determines which areas in the room have already acquired point clouds and which areas have not, based on the indoor position of the communication terminal device 11 and the image captured by the camera 51. The processing unit 61 then displays the areas in which point clouds have already been acquired and the areas in which point clouds have not been acquired in different colors on the display 52. ​​This makes it possible to acquire point clouds for all necessary areas in the room.

[0066] Step ST52: On the CAD image displayed in step ST51, the user operates the operation unit 53 to specify a space in the room where the product is to be placed. The space is specified by moving a cursor or the like to surround the area.

[0067] Step ST53: The user operates the operation unit 53 to input the product classification of the product to be placed in the space designated in step ST52. The input may be performed by displaying a product classification selection field on the display 52 and selecting the product from there, or the user may input the product classification directly. The product classification may be a product category such as a refrigerator, shelf, bookcase, sofa, etc.

[0068] Step ST54: The processing unit 61 of the communication terminal device 11 specifies the size of the space specified in step ST52 based on the CAD data in step ST51. Specifically, it specifies the width, depth, height, etc.

[0069] Step ST55: The processing unit 61 searches for products that can be placed in the space of the size specified in step ST54 from among the products belonging to the product category input in step ST53. In this search, the user may specify at least one of the budget, product brand, and function, and search for products that match the specified specifications. The processing unit 61 displays a product selection screen showing the search results on the display 52. ​​The user selects a product using the product selection screen. The communication terminal device 11 accesses the server device 21 or another product sales site and displays the search results.

[0070] Step ST56: The processing unit 61 displays on the display 52 a 3D CAD image in which the product selected in step ST55 is arranged in the space designated in step ST52.

[0071] Step ST57: When the user operates the operation unit 53 to request the purchase of the product selected in step ST55, the processing unit 61 proceeds to step ST58.

[0072] When the purchase request is input in step ST57, the processing unit 61 performs the purchase process for the product selected by the user. The communication terminal device 11 accesses the server device 21 or another product sales site via the communication unit 55 to perform the purchase process.

[0073] <Third embodiment> In the above-described embodiment, an example was given of purchasing products to be placed in a scanned room, but in this embodiment, a case will be described in which the renewal of commercial materials (supplies) such as walls and flooring in a room is considered.

[0074] The flow of the indoor layout support method of this embodiment will be described below. Fig. 9 is a flowchart for explaining the overall flow of the indoor layout support method of the third embodiment of the present invention. Each step will be described.

[0075] Step ST61: Under control of the processing unit 61, the communication terminal device 11 irradiates the room with a laser from the laser 50 to scan the room, and generates 3D image data of the room based on the scan result and the image capture result of the camera 51. The processing unit 61 then converts the 3D image data into 3D CAD data. Step ST61 is the same as the processing of step ST11 described in the first embodiment.

[0076] Step ST62: On the CAD image displayed in step ST61, the user operates the operation unit 53 to specify commercial materials (building materials) such as flooring and walls for the interior of the room. The specification is performed by using a cursor or the like to specify a point on the image of the commercial material or by moving the cursor to surround the area.

[0077] Step ST63: The user operates the operation unit 53 to input the product classification of the product specified in step ST62. This input can be performed by displaying a product classification selection field on the display 52 and allowing the user to select from there, or by directly inputting the product classification. The product classification can be, for example, a product category such as wall or flooring.

[0078] Step ST64: The processing unit 61 of the communication terminal device 11 transmits (shares) the product specified by the user in step ST61 together with the 3D image to a contractor (architectural, design, construction, etc.) previously associated with the product classification specified by the user in step ST63. At this time, the communication terminal device 11 transmits information on the size of the product (wall size, flooring size, etc.) to the contractor terminal device. Information such as the user's request regarding the product may also be transmitted from the communication terminal device 11 to the contractor's terminal device.

[0079] Step ST65: Based on the product, the user's request, and the CAD image shared in step ST64, the contractor proposes a product suitable for the user's room, and transmits estimate information such as the product price and construction cost to the communication terminal device 11. At this time, the contractor may provide a CAD image incorporating the image of the product to the communication terminal device 11. In addition, proposal information regarding the product may be transmitted from the contractor's terminal device to the communication terminal device 11.

[0080] As described above, according to the indoor layout support method of the second embodiment, a user can request a contractor to provide a proposal or quote for commercial materials, such as building materials, for use in the room by simply specifying the materials from a CAD image of the room using the communication terminal device 11. Furthermore, contractors can obtain 3D images of the user's room, enabling them to provide more appropriate proposals and quotes, including for renovations. Furthermore, allowing contractors, such as design offices, to use the functions of the communication terminal device 11 also enables them to provide more appropriate proposals.

[0081] The present invention is not limited to the above-described embodiments, and those skilled in the art may make various modifications, combinations, subcombinations, and substitutions of the components of the above-described embodiments within the technical scope of the present invention or equivalents thereof.

[0082] The present invention is applicable to a system for selecting products to be placed in a room.

[0083] REFERENCE SIGNS LIST 11: communication terminal device 21: server device 50: laser 51: camera 52: display 53: operation unit 55: communication unit 57: Bluetooth (registered trademark) communication unit 61: processing unit 75: communication unit 79: memory

Claims

1. A method for implementing the present invention comprising: a first step of acquiring or generating 3D CAD data of a room; a second step of receiving from a user a classification of at least one of furniture and home appliances to be placed in the room and a budget for the classification; a third step of determining a plurality of placement options showing different combinations of available spaces for the items in the room and the items to be placed in the spaces, based on the CAD data acquired or generated in the first step and the classification and budget for the items specified in the second step; a fourth step of displaying a 3D CAD image of the items placed in the available spaces for each of the plurality of placement options determined in the third step; and a fifth step of displaying a selection screen for selecting another item of the same classification as the displayed item when the user selects a product on the CAD image displayed in the fourth step, and generating the CAD data in which the other product is placed when the other product is selected on the selection screen. and a sixth step of storing in memory as history data the CAD data of the interior of a user who has previously purchased a product, the product purchased by the user, and the price of the purchased product, in association with each other. The third step determines the placement candidates and products using a learning model that has been machine-learned using a combination of the CAD data of the interior of a user who has previously purchased a product and the product purchased by the user as training data, and displays, from among the determined placement candidates and products, placement candidates and products selected by other users that have a combination that is highly similar to the combination of the CAD data of the interior, the specified classification, and its budget based on the history data stored in memory in the sixth step, as placement candidates and products with high priority.

2. The indoor layout support method of claim 1, further comprising the steps of: a seventh step of receiving from a user a designation of at least one of the interior materials, flooring and walls, based on the CAD data; an eighth step of transmitting the CAD data to a vendor pre-associated with the product designated in the seventh step; and a ninth step of receiving from the vendor product data relating to candidates for the product designated in the seventh step; and in the fourth step, displaying the CAD image based on the product data received in the ninth step.

3. The indoor layout support method according to claim 1, wherein the second step receives a designation of the user's family composition from the user, and the third step determines the multiple layout candidates based on the family composition designated in the second step.

4. The indoor layout support method according to claim 1, further comprising, after the fourth step, a step of carrying out a process in which the user purchases the candidate items in response to an instruction from the user.

5. The indoor layout support method according to claim 1, wherein the first step is an eleventh step of identifying a position within the room of a main device performing laser irradiation; a twelfth step of generating a point cloud of the room and objects within the room by the laser irradiation; a thirteenth step of converting the generated point cloud from the coordinate system of the identified main device to an indoor coordinate system; a fourteenth step of extracting valid data from the point cloud; a fifteenth step of converting the point cloud into mesh (polygon) data; a sixteenth step of converting the mesh into surface data; a seventeenth step of generating 3D image data by pasting a captured image onto the mesh based on the surface data; and an eighteenth step of converting the 3D image data generated in the seventeenth step into 3D CAD data.

6. The indoor layout support method according to claim 5, further comprising the steps of: identifying a shape and size of the room based on the surface data; and displaying the size within the display image.

7. The indoor layout support method according to claim 5, further comprising: determining areas of the room for which the point cloud has already been acquired and areas for which the point cloud has not been acquired based on the position of the main device and the captured image; and displaying the areas for which the point cloud has already been acquired and the areas for which the point cloud has not been acquired in different colors on the screen.

8. A method for providing a system for providing a display of a display of a room, comprising: a first step of acquiring or generating 3D CAD data of a room; a second step of receiving from a user a classification of at least one of furniture and home appliances to be placed in the room and a budget for the classification; a third step of determining a plurality of placement options showing different combinations of spaces in the room in which the items can be placed and the items to be placed in the spaces, based on the CAD data acquired or generated in the first step and the classification and budget for the items specified in the second step; a fourth step of displaying a 3D CAD image in which the items are placed in the available spaces for each of the plurality of placement options determined in the third step; and a fifth step of displaying, when a user specifies a product on the CAD image displayed in the fourth step, a selection screen for selecting another product of the same classification as the displayed product, and, when another product is selected on the selection screen, generating the CAD data in which the other product is placed. and a sixth step of storing in memory as history data the CAD data of the interior of a user who has previously purchased a product, the placement candidate selected by the user, and the product purchased by the user in association with each other. The third step determines, based on the history data stored in memory in the sixth step, the placement candidate and the product selected by another user who has a combination that is highly similar to the combination of the CAD data of the interior of the room, the specified classification, and its budget, as the placement candidate and product with high priority. The third step determines, based on the history data stored in memory in the sixth step, the placement candidate and the product with high priority, using a learning model that has been machine-learned using a combination of the CAD data of the interior of a user who has previously purchased a product, the placement candidate selected by the user, the product purchased by the user, and the attributes of the user as training data.

9. A method for providing a system for providing a display of a display of a room, comprising: a first step of acquiring or generating 3D CAD data of a room; a second step of receiving from a user a classification of at least one of furniture and home appliances to be placed in the room and a budget for the classification; a third step of determining a plurality of placement options showing different combinations of spaces in the room in which the items can be placed and the items to be placed in the spaces, based on the CAD data acquired or generated in the first step and the classification and budget for the items specified in the second step; a fourth step of displaying a 3D CAD image in which the items are placed in the available spaces for each of the plurality of placement options determined in the third step; and a fifth step of displaying, when a user specifies a product on the CAD image displayed in the fourth step, a selection screen for selecting another product of the same classification as the displayed product, and, when another product is selected on the selection screen, generating the CAD data in which the other product is placed. a sixth step of associating the CAD data of the interior of a user who has previously purchased a product with the product purchased by the user and the price of the purchased product in memory as historical data, wherein the third step of the program determines the placement candidates and products using a learning model that has been machine-learned using a combination of the CAD data of the interior of a user who has previously purchased a product with the product purchased by the user as training data, and displays, from among the determined placement candidates and products, the placement candidates and products selected by another user that have a combination that is highly similar to the combination of the CAD data of the interior, the specified classification and its budget based on the historical data stored in memory in the sixth step, as the placement candidates and products with high priority.

10. A method for providing a system comprising: a first step of acquiring or generating 3D CAD data of a room; a second step of receiving from a user a classification of at least one of furniture and home appliances to be placed in the room and a budget for the classification; a third step of determining a plurality of placement options showing different combinations of spaces in the room in which the products can be placed and the products to be placed in the spaces, based on the CAD data acquired or generated in the first step and the classification and budget for the products specified in the second step; a fourth step of displaying a 3D CAD image in which the products are placed in the available spaces for each of the plurality of placement options determined in the third step; and a fifth step of displaying, when a user specifies a product on the CAD image displayed in the fourth step, a selection screen for selecting another product of the same classification as the displayed product, and, when another product is selected on the selection screen, generating the CAD data in which the other product is placed. a sixth step of associating the CAD data of the interior of a user who has previously purchased a product, the placement candidate selected by the user, and the product purchased by the user and storing them in memory as historical data; wherein the third step determines, based on the historical data stored in memory in the sixth step, the placement candidate and product selected by another user who has a combination that is highly similar to the combination of the CAD data of the interior of the room, the specified classification, and its budget, as the placement candidate and product with high priority, and the third step determines, using a learning model that has been machine-learned using a combination of the CAD data of the interior of a user who has previously purchased a product, the placement candidate selected by the user, the product purchased by the user, and the attributes of the user as training data, as the placement candidate and product.

11. A first means for acquiring or generating 3D CAD data of a room; a second means for receiving from a user a specification of a classification of at least one of furniture and home appliances to be placed in the room and a budget for the classification; a third means for determining a plurality of placement options showing different combinations of spaces in the room in which the items can be placed and the items to be placed in the spaces, based on the CAD data acquired or generated by the first means and the classification and budget of the items specified by the second means; a fourth means for displaying a 3D CAD image in which the items are placed in the available spaces for each of the plurality of placement options determined by the third means; a fifth means for displaying a selection screen for selecting another item of the same classification as the displayed item when a user specifies a product on the CAD image displayed by the fourth means, and generating the CAD data in which the other product is placed when the other product is selected on the selection screen; and a sixth means for associating the CAD data of the interior of a user who has purchased merchandise in the past, the merchandise purchased by the user, and the price of the purchased item in memory as history data, wherein the third means determines the placement candidates and merchandise using a learning model trained by machine learning using a combination of the CAD data of the interior of a user who has purchased merchandise in the past and the merchandise purchased by the user as training data, and displays, from among the determined placement candidates and merchandise, the placement candidates and merchandise selected by another user having a combination that is highly similar to the combination of the CAD data of the interior, the specified classification, and its budget based on the history data stored in memory by the sixth means, as the placement candidates and merchandise with high priority.

12. A first means for acquiring or generating 3D CAD data of a room; a second means for receiving from a user a specification of a classification of at least one of furniture and home appliances to be placed in the room and a budget for the classification; a third means for determining a plurality of placement options showing different combinations of spaces in the room in which the items can be placed and the items to be placed in the spaces, based on the CAD data acquired or generated by the first means and the classification and budget of the items specified by the second means; a fourth means for displaying a 3D CAD image in which the items are placed in the available spaces for each of the plurality of placement options determined by the third means; a fifth means for displaying a selection screen for selecting another item of the same classification as the displayed item when a user specifies a product on the CAD image displayed by the fourth means, and generating the CAD data in which the other product is placed when the other product is selected on the selection screen; and a sixth means for storing in memory as history data the CAD data of the interior of a user who has purchased commercial materials in the past, the placement candidates selected by the user, and the commercial materials purchased by the user in association with each other, wherein the third means determines, based on the history data stored in memory by the sixth means, the placement candidates and commercial materials selected by other users who have a combination that is highly similar to the combination of the CAD data of the interior of the room, the specified classification, and its budget, as the placement candidates and commercial materials with high priority, and the third means determines, using a learning model that has been machine-learned using a combination of the CAD data of the interior of a user who has purchased commercial materials in the past, the placement candidates selected by the user, the commercial materials purchased by the user, and the attributes of the user as training data, as the placement candidates and commercial materials.

Images