Drawing creation device and drawing creation method

Abstract

To provide a drawing creation device capable of creating a layout drawing including an object arranged at a target location with little effort.SOLUTION: In a drawing creation device that uses a processor to execute processing of generating a layout drawing showing arrangement of an object on a target location on the basis of point cloud data acquired by performing three-dimensional measurement processing on the target location, the processor recognizes, on the basis of the point cloud data, the object present at the target location, generates a plane projection image by projecting each point of the point cloud data representing the recognized object onto a predetermined plane, and generates and outputs the layout drawing on the basis of the plane projection image. In particular, the processor extracts a contour of the object from the plane contour image and generates a layout drawing that represents the object using a graphic form that approximates the contour.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to a drawing creation device and drawing creation method that generate a layout drawing that shows the arrangement of objects in a target location, based on point cloud data acquired by performing three-dimensional measurement processing on the target location. [Background technology]

[0002] Buildings are used for various purposes such as offices, stores, homes, and factories. When considering a renovation plan for the interior of such a building, a floor plan showing the interior layout of the building is required. In this case, it is very time-consuming for an operator to measure the interior of the building and create a floor plan, so a technology that can easily create a floor plan showing the interior layout of a building is desired.

[0003] A known technology for easily creating floor plans showing the interior layout of such buildings is to perform 3D measurements of the target location based on images taken of the interior of a building as the target location, generate point cloud data for the target location, recognize the walls, floors, and ceilings of the target location based on the point cloud data, and create a floor plan based on the recognition results (see Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-145072 A Summary of the Invention [Problem to be solved by the invention]

[0005] Various objects, such as furniture such as shelves, desks, chairs, and curtains, and electrical appliances such as refrigerators, televisions, and air conditioners, are arranged inside a building. For this reason, a layout diagram in which the objects arranged inside the building are drawn on a plan view showing the layout of the interior of the building may be required. In this case, it is desirable to create the layout diagram with minimal effort. However, conventional techniques do not take such demands into consideration, and there is a problem in that the layout diagram cannot be created with minimal effort.

[0006] Therefore, a main object of the present invention is to provide a drawing creation device and a drawing creation method that can create a layout drawing including an object placed at a target location with minimal effort. [Means for solving the problem]

[0007] The drawing creation device of the present invention is a drawing creation device that uses a processor to execute a process of generating a layout drawing representing the arrangement of objects in a target location based on point cloud data acquired by performing 3D measurement processing of the target location, and is configured to recognize objects present in the target location based on the point cloud data, generate a planar projection image by projecting each point of the point cloud data representing the recognized objects onto a specified plane, and generate and output the layout drawing based on the planar projection image.

[0008] In addition, the drawing creation method of the present invention is a drawing creation method in which a processor executes a process to generate a layout drawing representing the arrangement of objects in a target location based on point cloud data acquired by performing 3D measurement processing of the target location, the method being configured to recognize objects present in the target location based on the point cloud data, generate a planar projection image by projecting each point of the point cloud data representing the recognized objects onto a specified plane, and generate and output the layout drawing based on the planar projection image. Effect of the Invention

[0009] According to the present invention, a layout drawing including an object placed in a target location can be created with little effort. [Brief description of the drawings]

[0010] [Figure 1] Overall configuration of a drawing creation system according to the present embodiment. [Diagram 2] A block diagram showing the schematic configuration of a user terminal and a server. [Diagram 3] FIG. 13 is an explanatory diagram showing a shooting screen displayed on a user terminal. [Figure 4] FIG. 13 is an explanatory diagram showing a point cloud data confirmation screen displayed on a user terminal. [Diagram 5] FIG. 13 is an explanatory diagram showing a setting screen displayed on a user terminal. [Figure 6] FIG. 1 is an explanatory diagram showing an overview of object recognition processing performed by a server; [Figure 7] FIG. 1 is an explanatory diagram showing an overview of object recognition processing performed by a server; [Figure 8] An explanatory diagram showing the outline of the processing performed on the server in batch drawing mode [Figure 9] FIG. 13 is an explanatory diagram showing a layout diagram confirmation screen displayed on a user terminal in the batch drawing mode. [Figure 10] Flow diagram showing the procedure of processing performed on the server in batch drawing mode [Figure 11] An explanatory diagram showing the outline of the processing performed by the server in the type-specific drawing mode [Figure 12] FIG. 13 is an explanatory diagram showing a layout diagram confirmation screen displayed on a user terminal in a type-specific drawing mode. [Figure 13] FIG. 13 is an explanatory diagram showing a layout diagram confirmation screen displayed on a user terminal in a type-specific drawing mode. [Figure 14] Flow diagram showing the procedure of processing performed on the server in the type-specific drawing mode [Figure 15] An explanatory diagram showing an overview of the processing performed by the server in the individual drawing mode. [Figure 16] FIG. 13 is an explanatory diagram showing a layout diagram confirmation screen displayed on a user terminal in an individual drawing mode. [Figure 17]FIG. 13 is an explanatory diagram showing a layout diagram confirmation screen displayed on a user terminal in an individual drawing mode. [Figure 18] Flow diagram showing the procedure of processing performed on the server in the individual drawing mode [Figure 19] FIG. 11 is an explanatory diagram showing a point cloud data confirmation screen displayed on a user terminal according to another example. [Figure 20] FIG. 13 is an explanatory diagram showing an overview of a process performed by a server according to another example. [Figure 21] FIG. 11 is an explanatory diagram showing a layout diagram confirmation screen displayed on a user terminal according to another example. [Figure 22] FIG. 11 is a block diagram showing a schematic configuration of a user terminal and a server according to a second embodiment. [Figure 23] FIG. 11 is an explanatory diagram showing a layout diagram confirmation screen displayed on a user terminal according to the second embodiment; [Figure 24] FIG. 11 is an explanatory diagram showing a layout editing screen displayed on a user terminal according to the second embodiment; [Diagram 25] FIG. 11 is an explanatory diagram showing an editing state of a layout diagram according to the second embodiment; [Figure 26] FIG. 11 is an explanatory diagram showing a layout diagram confirmation screen displayed on a user terminal according to the second embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The first invention made to solve the above problems is a drawing creation device that uses a processor to execute a process for generating a layout drawing showing the arrangement of objects in a target location based on point cloud data acquired by performing 3D measurement processing of the target location, wherein the processor recognizes objects present in the target location based on the point cloud data, generates a planar projection image by projecting each point of the point cloud data representing the recognized objects onto a specified plane, and generates and outputs the layout drawing based on the planar projection image.

[0012] This makes it possible to create a layout drawing including an object placed at a target location with little effort.

[0013] In a second aspect of the present invention, the processor sets the processing conditions in response to a user's operation input on a setting screen relating to processing conditions.

[0014] This allows the user to specify processing conditions.

[0015] In a third aspect of the present invention, the processor is configured to set an actual length represented by one pixel of the planar projection image as the processing condition.

[0016] This allows the user to specify the degree of detail of the planar projection image.

[0017] In a fourth aspect of the present invention, the processor is configured to set, as the processing condition, a type of object to be illustrated in the layout drawing.

[0018] This allows the user to specify the type of object to be drawn on the layout drawing.

[0019] In addition, in a fifth invention, the processor is configured to set, as the processing conditions, a drawing mode relating to a procedure for generating figures of each object to be drawn in the layout drawing from the point cloud data, and a display mode relating to whether or not to display characters indicating the type of object in the layout drawing.

[0020] This allows the user to specify the drawing mode and the display mode.

[0021] In a sixth aspect of the present invention, the processor extracts a contour line of an object from the planar projection image, and generates the layout drawing in which the object is represented by a figure approximating the contour line.

[0022] This makes it possible to generate an appropriate layout drawing in which a figure representing the outline of an object is drawn.

[0023] In a seventh aspect of the present invention, the processor generates the layout drawing in which an object is expressed by a polygon having the contour line as a side as the figure approximating the contour line.

[0024] This allows the object to be represented in a layout diagram with an appropriate figure. In this case, the figure data should preferably include information (coordinates) of the vertices of the polygon.

[0025] In an eighth aspect of the present invention, the processor generates the layout drawing in which an object is represented by a straight line passing through a center of an image of the contour line as the figure approximating the contour line.

[0026] According to this, for example, a thin object such as a curtain will have a long and thin outline in a flat projection image, so by expressing it as a line segment figure rather than a polygon, editing work becomes easier. In this case, it is preferable for the figure data to include information (coordinates) on the endpoints of the line segments.

[0027] In addition, in a ninth invention, the processor is configured to generate the planar projection image of the entire object to be illustrated by collectively projecting each point of the point cloud data representing the object to be illustrated, and to generate the layout drawing including a figure representing the overall outline of the object to be illustrated based on the planar projection image.

[0028] This allows the user to properly grasp the overall arrangement of the objects.

[0029] In addition, in a tenth invention, the processor is configured to generate the planar projection image for each type of object to be illustrated by projecting each point of the point cloud data representing the object to be illustrated separately by type, and to generate the layout drawing including figures representing the outline of each type of object to be illustrated based on the planar projection image.

[0030] With this, even when different types of objects overlap each other, such as when a chair is stored under a table, the different types of objects are represented by separate figures, allowing the user to properly grasp the arrangement of the objects.

[0031] In addition, the 11th invention is configured such that the processor generates the planar projection image of each individual part of the object to be illustrated by dividing each point of the point cloud data representing the object to be illustrated into individual parts and projecting them, and generates the layout drawing including a figure representing the outline of each individual part of the object to be illustrated based on the planar projection image.

[0032] According to this, the objects to be illustrated are separately illustrated as individual objects, thereby preventing confusion over the external shapes of the objects to be illustrated and enabling the user to properly grasp the placement status of the objects.

[0033] In a twelfth aspect of the present invention, the processor is configured to superimpose characters indicating a type of an object near the object drawn on the layout diagram.

[0034] With this, characters indicating the type of object are superimposed on the layout drawing, eliminating the need for the user to input characters using a CAD application or the like.

[0035] In addition, a 13th invention is a drawing creation method in which a processor executes a process to generate a layout drawing representing the arrangement of objects in a target location based on point cloud data acquired by performing 3D measurement processing of the target location, the method being configured to recognize objects present in the target location based on the point cloud data, generate a planar projection image by projecting each point of the point cloud data representing the recognized objects onto a predetermined plane, and generate and output the layout drawing based on the planar projection image.

[0036] According to this, like the first invention, a layout drawing including an object placed at a target location can be created with little effort.

[0037] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0038] (First embodiment) FIG. 1 is a diagram showing the overall configuration of a drawing creation system according to this embodiment.

[0039] This system includes a user terminal 1 (client) and a server 2 (drawing creation device). The user terminal 1 and the server 2 are connected via a network such as a LAN or the Internet.

[0040] The user terminal 1 includes a device main body 11 and a camera 12. The user terminal 1 can be configured as a tablet terminal or a notebook PC.

[0041] The camera 12 is a visible camera, i.e., a monocular camera that detects visible light to capture an image of a subject, and outputs a captured image, for example, a color image in the RGB format. For the three-dimensional measurement performed by the server 2, the user terminal 1 may be provided with sensors such as a depth camera that measures the distance to the subject and an IMU (Inertial Measurement Unit) that detects three-dimensional angular velocity and acceleration, in addition to the camera 12 (visible camera).

[0042] A user (worker) walks through the target location while holding the device main body 11 of the user terminal 1. At this time, the target location is photographed by the camera 12 of the user terminal 1, and photographed images of each point in the target location are acquired sequentially. Note that, although an example has been shown here in which photographed images are acquired by a user holding the device main body 11 walking through the target location, photographed images of the target location may also be acquired using a self-propelled robot or the like equipped with the user terminal 1.

[0043] Next, there will be described a schematic configuration of the user terminal 1 and the server 2. FIG.

[0044] In addition to the camera 12, the user terminal 1 includes a display 13 (display unit), an input device 14, a storage unit 15, a processor 16 (CPU), and a communication unit 17.

[0045] The display 13 displays various operation screens related to shooting operations, checking layout drawings, editing operations, etc., and setting screens for setting various processing conditions. The input device 14 is used by the user to perform input operations. When the user terminal 1 is configured as a tablet terminal, a touch panel display in which a touch panel as the input device 14 and a display panel as the display 13 are integrated is provided.

[0046] The storage unit 15 stores programs executed by the processor 16. The storage unit 15 also stores images captured by the camera 12. The captured images are transmitted to the server 2 at appropriate timing.

[0047] The processor 16 executes the programs stored in the storage unit 15 to perform various processes related to shooting by the camera 12, displaying on the display 13, obtaining operation information by the input device 14, and the like.

[0048] The communication unit 17 communicates with the server 2. Specifically, the communication unit 17 transmits to the server 2 images captured by the camera 12 and operation information acquired by a user operating the input device 14. The communication unit 17 also receives display information of various screens transmitted from the server 2.

[0049] The server 2 includes a communication unit 21, a storage unit 22, and a processor 23.

[0050] The communication unit 21 communicates with the user terminal 1. Specifically, the communication unit 21 receives captured images and operation information transmitted from the user terminal 1. In addition, the communication unit 21 transmits, to the user terminal 1, display information of various screens to be displayed on the user terminal 1.

[0051] The storage unit 22 stores programs and the like executed by the processor 23. The storage unit 22 also stores point cloud data generated by the processor 23, data of a layout diagram (layout model), and the like.

[0052] The processor 23 performs various processes by executing the programs stored in the storage unit 22. In the present embodiment, the processor 23 performs a point cloud data generation process, a layout drawing generation process, a screen generation process, a layout file generation process, a processing condition setting process, and the like.

[0053] In the point cloud data generation process (three-dimensional measurement process), the processor 23 generates point cloud data as three-dimensional spatial information related to the target location based on the captured images received from the user terminal 1. In the point cloud data generation process, for example, the SLAM method is used. Note that the point cloud data is defined in a coordinate system having the same size as the real space, for example, and the actual dimensions of each part of the target location can be measured from the point cloud data.

[0054] In the layout drawing generation process, the processor 23 generates a layout drawing (layout model) of a target location based on the point cloud data generated in the point cloud data generation process. The layout drawing generation process includes an object recognition process, a point cloud data extraction process, a planar projection process, a contour line extraction process, and a graphic representation process.

[0055] In the object recognition process, the processor 23 recognizes objects present in the target location based on the point cloud data generated in the point cloud data generation process, and adds the type of object represented by each point of the point cloud data as attribute information. At this time, the floor, wall, and ceiling are recognized as objects present in the target location. In addition, objects arranged on the floor, wall, and ceiling, such as shelves, desks, chairs, refrigerators, televisions, curtains, air conditioners, and cash registers, are recognized. In the object recognition process, an object recognition engine (machine learning model) constructed by machine learning such as deep learning may be used.

[0056] In the point cloud data extraction process, the processor 23 extracts point cloud data representing objects belonging to a type designated by the user as a drawing target from among the objects recognized in the object recognition process. In this case, in the batch drawing mode (first drawing mode), point cloud data of all types of objects designated by the user as a drawing target is extracted. On the other hand, in the type-specific drawing mode (second drawing mode), point cloud data is extracted for each type designated by the user as a drawing target. Also, in the individual-specific drawing mode (third drawing mode), point cloud data is extracted for each individual object belonging to a type designated by the user as a drawing target.

[0057] In the planar projection process, the processor 23 projects each point of the point cloud data extracted in the point cloud data extraction process onto a predetermined plane to generate a planar projection image. At this time, a planar projection image with a number of pixels based on the pixel size specified by the user is generated.

[0058] In the contour line extraction process, the processor 23 extracts the contour line representing the external shape of the object to be illustrated from the planar projection image generated in the planar projection process, and generates contour line data including only the image of the contour line.

[0059] In the graphic processing, the processor 23 generates graphic data including a graphic approximating the contour of the object to be illustrated, that is, a graphic representing the outer shape of the object, based on the contour data generated in the contour extraction processing. The graphic data is composed of a plurality of points and line segments connecting the points. The coordinates of each point constituting the graphic are stored in, for example, a clockwise order in the graphic data.

[0060] In the screen generation process, the processor 23 generates display information for a screen to be displayed on the user terminal 1. The display information is transmitted to the user terminal 1, and the screen is displayed on the user terminal 1.

[0061] In the layout file generation process, the processor 23 generates and outputs a layout file in which data of the layout diagram (layout model) generated in the layout diagram generation process is stored. The layout file may be created in a versatile format, such as DXF (Drawing Exchange Format, registered trademark). This allows detailed correction and finishing of the layout diagram to be performed using a drawing application (CAD application). In this case, when a file is generated in a versatile format, various drawing applications can be used.

[0062] In the processing condition setting process, the processor 23 sets processing conditions (parameters) related to various processes performed by the server 2 based on operation information received in response to a user operation on the user terminal 1. Specifically, processing conditions related to a layout drawing generation process, a screen generation process, and the like are set in response to a user operation on a setting screen (see FIG. 5) displayed on the user terminal 1.

[0063] Next, a description will be given of the photographing screen 101 displayed on the user terminal 1. FIG.

[0064] On the shooting screen 101, a current shot image 102, that is, an image shot in real time by the camera 12, is displayed.

[0065] Furthermore, the shooting screen 101 is provided with a "start shooting" button 111 and a "recording confirmation" button 112. When the user operates the "start shooting" button 111, shooting by the camera 12 starts, and the captured images are stored in the storage unit 15 and displayed on the shooting screen 101. When the user operates the "recording confirmation" button 112, a transition is made to a mode in which the captured images stored in the storage unit 15 are played back. This allows the user to confirm whether or not the shooting has been performed appropriately.

[0066] Furthermore, a "check point cloud data" button 113 is provided on the photographing screen 101. When the user operates the "check point cloud data" button 113, the screen transitions to a point cloud data check screen 201 (see FIG. 4).

[0067] Next, a description will be given of the point cloud data confirmation screen 201 displayed on the user terminal 1. FIG 4 is an explanatory diagram showing the point cloud data confirmation screen 201.

[0068] A point cloud image 202 is displayed on the point cloud data confirmation screen 201. The server 2 performs a three-dimensional measurement process based on the captured image received from the user terminal 1, and generates point cloud data as the measurement result. The point cloud image 202 is created by imaging (rendering) each point of the point cloud data as viewed from a specific viewpoint. This allows the user to confirm that the point cloud data, which is the three-dimensional measurement result, has been appropriately generated.

[0069] Furthermore, the point cloud data confirmation screen 201 is provided with a "layout drawing creation setting" button 114, a "layout drawing creation" button 115, and a "file output" button 116. When the user operates the "layout drawing creation setting" button 114, a setting screen 301 (see FIG. 5) is displayed. When the user operates the "layout drawing creation" button 115, a layout drawing generation process is executed in the server 2, and a layout drawing confirmation screen 401 (see FIG. 9, etc.) is displayed, displaying the results of the process. When the user operates the "file output" button 117, a layout file is generated and output.

[0070] Next, a description will be given of the setting screen 301 displayed on the user terminal 1. FIG.

[0071] The setting screen 301 is provided with a pixel size setting section 302 , a target object setting section 303 , and a processing mode setting section 304 .

[0072] The pixel size setting section 302 sets the pixel size that defines the degree of resolution of the planar projection image, that is, the actual length represented by one pixel of the planar projection image. The pixel size setting section 302 is provided with an input field 305 for the pixel size. The user can input and specify the pixel size in the input field 305.

[0073] The target object setting unit 303 sets the type of object to be illustrated, that is, the type of object to be drawn on the layout drawing. The target object setting unit 303 is provided with a check box 306 for each type of object. In this example, check boxes 306 for each type of object are provided: shelf, desk, chair, curtain, refrigerator, television, air conditioner, and cash register. The user can select the type of object to be illustrated by operating the check boxes 306. The target object setting unit 303 is also provided with an "add / delete" button 307. When the user operates the "add / delete" button 307, an editing screen (not shown) for the type of object to be illustrated is displayed, where the user can add or delete the type of object to be illustrated. Note that the objects displayed in the target object setting unit 303 are not limited to the eight objects in FIG. 5, and can be added or changed in correspondence with the check boxes 306 from a target object database (not shown) stored as recognizable objects.

[0074] The processing mode setting unit 304 sets the processing mode in the server 2. The processing modes include a drawing mode and a display mode. The drawing mode relates to a procedure for generating the figures of each object to be drawn on the layout drawing from point cloud data. The display mode relates to whether or not to display characters indicating the type of object on the layout drawing. In this example, the drawing modes include a batch drawing mode, a type-specific drawing mode, and an individual-specific drawing mode. In addition, the display modes include an object name non-display mode and an object name display mode.

[0075] The processing mode setting section 304 is provided with a radio button 308 for each combination of drawing mode and display mode. The radio buttons 308 are provided for a total of five combinations: a batch drawing mode, a combination of a type-specific drawing mode and an object name non-display mode, a combination of a type-specific drawing mode and an object name display mode, a combination of an individual-specific drawing mode and an object name non-display mode, and a combination of an individual-specific drawing mode and an object name display mode. The user can select one of the combinations of drawing mode and display mode by operating the radio button 308.

[0076] Next, an overview of the object recognition process performed by the server 2 will be described. Figures 6 and 7 are explanatory diagrams showing an overview of the object recognition process. Note that the object recognition process uses machine learning such as deep learning to recognize various objects and output the object names, etc.

[0077] The server 2 generates point cloud data as three-dimensional spatial information regarding the target location based on the captured images of the target location received from the user terminal 1. The server 2 also recognizes objects present in the target location based on the generated point cloud data, and adds the type of object represented by each point of the point cloud data as attribute information to each point of the point cloud data as an object recognition result.

[0078] In the example shown in FIG. 6(A), a room that is a target location has a floor and walls. In addition, a shelf, a desk, and a chair are placed on the floor. In addition, a door is provided on the wall. FIG. 6(B) shows an object recognition result for the point cloud data shown in FIG. 6(A). In FIG. 6(B), the object recognition result added to each point of the point cloud data is expressed by a shade of color for convenience. Here, the floor and walls that exist in the target location are recognized. In addition, a shelf, a desk, and a chair placed on the floor are recognized. In addition, a door provided on the wall is recognized.

[0079] In the example shown in FIG. 7(A), a floor and walls are provided in a room that is a target location, similar to the example shown in FIG. 6(A). Also, a door is provided on the wall. Meanwhile, a desk and three chairs are placed on the floor. FIG. 7(B) shows an object recognition result for the point cloud data shown in FIG. 7(A). In FIG. 7(B), the object recognition result added to each point of the point cloud data is expressed by a shade of color for convenience. Here, similar to the example shown in FIG. 6(B), the floor and walls present in the target location are recognized. Also, a door provided on the wall is recognized. Also, a desk and three chairs placed on the floor are recognized. The three chairs are of the same type but different individuals, so they are recognized separately.

[0080] The floor and walls of the room are also recognized separately, as are the four walls surrounding the room.

[0081] Next, the batch drawing mode (first drawing mode) will be described. Fig. 8 is an explanatory diagram showing an overview of the process performed by the server 2 in the batch drawing mode.

[0082] In the batch drawing mode, the point cloud data of the target location is processed collectively for all objects to be drawn, and graphic data of the entire objects to be drawn is created. In the example shown in Figure 8, a shelf, a desk, and a chair are placed on the floor of the room that is the target location. In addition, desks and chairs are specified as the types of objects to be drawn.

[0083] First, point cloud data including all objects to be illustrated is extracted from the point cloud data. In this example, point cloud data representing a desk and a chair is extracted collectively.

[0084] Next, the point cloud data including all objects to be illustrated is projected onto a plane parallel to the floor, and a planar projection image (collectively, a planar projection image) including all objects to be illustrated is generated. In this example, the point cloud data including both the desk and the chair is projected, and a planar projection image including both the desk and the chair is generated.

[0085] At this time, the number of pixels in the vertical and horizontal directions of the planar projection image is determined based on the size of the target location and the pixel dimensions. For example, if the pixel dimensions are set to 1 cm, then 1 cm square of the target location becomes 1 pixel. Therefore, if the measurement range of the 3D measurement process, i.e., the size of the plane onto which the point cloud data is projected, is 3 m vertically and 5 m horizontally, the number of pixels in the vertical and horizontal directions of the planar projection image will be 300 pixels vertically and 500 pixels horizontally.

[0086] The planar projection image can be a binary image that expresses the presence or absence of an object in black and white. In this case, for example, pixels onto which points of the point cloud data representing the object to be illustrated are projected are black, and pixels onto which points of the point cloud data representing the object are not projected are white. Alternatively, black and white may be reversed, that is, pixels onto which points of the point cloud data representing the object are projected are white, and pixels onto which points of the point cloud data representing the object are not projected are black.

[0087] In this embodiment, a layout drawing as a plan view is created by projecting the point cloud data onto a plane parallel to the floor, but a layout drawing as a side view may also be created by projecting the point cloud data onto a plane parallel to the wall surface.

[0088] Next, the contours of the entire objects to be illustrated are extracted from the planar projection image including all of the objects to be illustrated, and contour data of the entire objects to be illustrated (collectively, contour data) is generated. In this example, the contours representing the external shapes of the desk and chair in their integrated state are extracted from the planar projection image including both the desk and chair.

[0089] Next, based on the contour line data of the entire object to be drawn, a figure that approximates the contour line of the entire object to be drawn is obtained, and figure data (lump figure data) of the entire object to be drawn is generated. In this example, a polygonal figure including both a desk and a chair is obtained. At this time, the vertices of the polygon are set so that the sides of the polygon trace the contour line.

[0090] Next, a description will be given of the layout drawing confirmation screen 401 displayed on the user terminal 1 in the batch drawing mode.

[0091] When the user operates the "Create layout drawing" button 115 on the point cloud data confirmation screen 201 (see Fig. 4), the screen transitions to a layout drawing confirmation screen 401 shown in Fig. 9. At this time, the layout drawing generation process is executed in the server 2, and a layout drawing 402 as a result of the process is displayed on the layout drawing confirmation screen 401. The user can visually check the displayed layout drawing 402 and confirm that the layout drawing 402 has been appropriately generated.

[0092] Next, a procedure of the process performed by the server 2 in the batch drawing mode will be described.

[0093] In the server 2, first, the processor 23 recognizes an object existing in a target location based on the point cloud data (object recognition process) (ST101).

[0094] Next, the processor 23 extracts point cloud data representing objects belonging to a type designated by the user as an object to be illustrated from among the objects recognized in the object recognition process (point cloud data extraction process) (ST102).

[0095] Next, the processor 23 projects each point of the point cloud data extracted in the point cloud data extraction process onto a predetermined plane to generate a planar projection image (planar projection process) (ST103).

[0096] Next, processor 23 extracts a contour line representing the outer shape of the object to be illustrated from the planar projection image generated by the planar projection process, and generates contour line data including only the image of the contour line (contour line extraction process) (ST104).

[0097] Next, processor 23 generates graphic data including a graphic approximating the contour of the object to be illustrated, based on the contour data generated in the contour extraction process (graphical processing) (ST105).

[0098] Next, the type-specific drawing mode (second drawing mode) will be described. Fig. 11 is an explanatory diagram showing an overview of the process performed by the server 2 in the type-specific drawing mode.

[0099] In the type-based drawing mode, the point cloud data of the target location is processed by classifying it into the types of objects to be drawn, and graphic data for each type of object is created. In the example shown in Fig. 11, a shelf, a desk, and a chair are placed on the floor of the room that is the target location. In addition, a desk and a chair are specified as the types of objects to be drawn.

[0100] First, point cloud data for each type of object to be illustrated (point cloud data by type) is extracted from the point cloud data. In this example, point cloud data representing each of a desk and a chair is extracted separately.

[0101] Next, the point cloud data for each type of object to be illustrated is projected separately onto a plane parallel to the floor, and a planar projection image for each type of object (type-specific planar projection image) is generated. In this example, the point cloud data for each of the desk and chair is projected separately, and a planar projection image for each of the desk and chair is generated.

[0102] Next, from the planar projection image of each type of object to be illustrated, the contour line of each type of object is extracted, and contour line data (collectively, contour line data) of each type of object is generated. In this example, the contour lines representing the external shapes of each of the desk and chair are extracted separately from the planar projection images of each of the desk and chair.

[0103] Next, based on the contour line data for each type of object to be illustrated, a figure approximating the contour line for each type of object is obtained, and figure data for each type of object (figure data by type) is generated. In this example, polygonal (rectangular) figures representing the outlines of each of the desk and chair are obtained separately.

[0104] Next, the graphic data for each type of object to be illustrated is integrated to create a layout diagram. In this example, the graphic data for a desk and a chair are integrated to create a layout diagram.

[0105] Next, a description will be given of the layout drawing confirmation screen 401 displayed on the user terminal 1 in the type-specific drawing mode.

[0106] When the user operates the "Create layout drawing" button 115 on the point cloud data confirmation screen 201 (see Fig. 4), the screen transitions to a layout drawing confirmation screen 401 shown in Fig. 12 and Fig. 13. At this time, the layout drawing generation process is executed in the server 2, and a layout drawing 402 as a result of the process is displayed on the layout drawing confirmation screen 401.

[0107] In this example, the desk and chair are partially overlapping. Specifically, the chair except for the backrest is stored under the tabletop of the desk. In this case, in the batch drawing mode (see FIG. 9), the desk and chair are integrated and expressed as one figure, but in the type-specific drawing mode, the desk and chair are expressed as separate figures, allowing the user to properly grasp the arrangement of the desks and chairs.

[0108] 13 shows a case where a combination of the individual drawing mode and the object name display mode is selected. In this case, characters indicating the type of object are displayed near the figure representing the object in the layout diagram 402. On the other hand, FIG. 12 shows a case where a combination of the individual drawing mode and the object name non-display mode is selected. In this case, characters indicating the type of object are not displayed on the layout diagram 402.

[0109] Next, the procedure of the process performed by the server 2 in the type-specific drawing mode will be described.

[0110] In the server 2, the processor 23 first recognizes an object existing in a target location based on the point cloud data (object recognition process) (ST101). Next, the processor 23 initializes N, which indicates the order of the type designated by the user as the illustrated object (ST111).

[0111] Next, processor 23 extracts point cloud data representing an object that belongs to the Nth type among the types designated by the user as objects to be illustrated, among the objects recognized in the object recognition process (point cloud data extraction process) (ST112).

[0112] Next, the processor 23 projects each point of the point cloud data extracted in the point cloud data extraction process onto a predetermined plane to generate a planar projection image (planar projection process) (ST113).

[0113] Next, processor 23 extracts a contour line representing the outer shape of the object to be illustrated from the planar projection image generated by the planar projection process, and generates contour line data including only the image of the contour line (contour line extraction process) (ST114).

[0114] Next, processor 23 generates graphic data including a graphic approximating the contour of the object to be illustrated, based on the contour data generated in the contour extraction process (graphical process) (ST115).

[0115] Next, processor 23 determines whether or not the processing for all of the types of objects designated as the objects to be depicted has been completed (ST116). If the processing for all types of objects designated as the objects to be depicted has not been completed (No in ST116), processor 23 adds 1 to N (ST117), returns to ST112, and proceeds to the processing for the next type. On the other hand, if the processing for all types of objects designated as the objects to be depicted has been completed (Yes in ST116), this flow ends.

[0116] Next, the individual drawing mode (third drawing mode) will be described. Fig. 15 is an explanatory diagram showing an overview of the process performed by the server 2 in the individual drawing mode.

[0117] In the individual drawing mode, the point cloud data of the target location is processed separately for each individual object to be drawn, and graphic data for each individual object is created. In the example shown in Fig. 15, a shelf, a desk, and three chairs are placed on the floor of the room that is the target location. In addition, a desk and a chair are specified as the types of objects to be drawn.

[0118] First, point cloud data for each individual object to be illustrated (individual point cloud data) is extracted from the point cloud data. In this example, point cloud data representing the desk and each of the three chairs is extracted separately.

[0119] Next, based on the point cloud data for each object to be illustrated, graphic data for each object (individual graphic data) is generated. In this example, graphic data for each of the desk and three chairs is generated based on the point cloud data for each of the desk and three chairs.

[0120] Next, the graphic data for each object to be drawn is integrated to create a layout diagram. In this example, the graphic data for a desk and three chairs is integrated to create a layout diagram.

[0121] Although not shown in FIG. 15, the plane projection process, the contour line extraction process, and the graphic representation process are carried out for each individual object to be illustrated.

[0122] That is, the point cloud data for each object to be illustrated is projected separately onto a plane parallel to the floor, and a planar projection image for each object (individual planar projection image) is generated. In this example, the point cloud data for each of the desk and three chairs is projected separately, and a planar projection image for each of the desk and three chairs is generated.

[0123] Next, the contour line of each individual object is extracted from the planar projection image of each individual object to be illustrated, and individual contour line data (individual contour line data) is generated. In this example, the contour lines representing the external shapes of each of the desk and three chairs are extracted separately from the planar projection images of each of the desk and three chairs.

[0124] Next, based on the contour line data for each individual object to be illustrated, a figure approximating the contour line for each individual object is obtained, and individual figure data (individual figure data) is generated. In this example, polygonal (rectangular) figures representing the outlines of the desk and the three chairs are obtained separately.

[0125] Next, a description will be given of the layout drawing confirmation screen 401 displayed on the user terminal 1 in the individual drawing mode.

[0126] When the user operates the "Create layout drawing" button 115 on the point cloud data confirmation screen 201 (see Fig. 4), the screen transitions to a layout drawing confirmation screen 401 shown in Fig. 16 and Fig. 17. At this time, a layout drawing generation process is executed on the server 2, and a layout drawing 402 as a result of the process is displayed on the layout drawing confirmation screen 401.

[0127] In the individual drawing mode, if the objects are different but of the same type, each object is represented by an independent figure. In this example, two chairs are arranged side by side in close proximity. When there are multiple objects of the same type, the two chairs may be represented as one figure in the type drawing mode, but in the individual drawing mode, the two chairs are represented as separate figures, allowing the user to properly grasp the arrangement of the two chairs.

[0128] 17 shows a case where a combination of the individual drawing mode and the object name display mode is selected. In this case, characters indicating the type of object are displayed near the figure representing the object in the layout diagram 402. On the other hand, FIG. 16 shows a case where a combination of the individual drawing mode and the object name non-display mode is selected. In this case, characters indicating the type of object are not displayed on the layout diagram 402.

[0129] Next, a procedure of the process performed by the server 2 in the individual drawing mode will be described.

[0130] In the server 2, the processor 23 first recognizes objects existing in a target location based on the point cloud data (object recognition process) (ST101). Next, the processor 23 initializes N, which indicates the order of the objects belonging to the type designated by the user as the illustrated target (ST121).

[0131] Next, the processor 23 extracts point cloud data representing an object that is the Nth individual from among the objects recognized in the object recognition process and belonging to the type specified by the user as the object to be illustrated (point cloud data extraction process) (ST122).

[0132] Next, the processor 23 projects each point of the point cloud data extracted in the point cloud data extraction process onto a predetermined plane to generate a planar projection image (planar projection process) (ST123).

[0133] Next, processor 23 extracts a contour line representing the outer shape of the object to be illustrated from the planar projection image generated by the planar projection process, and generates contour line data including only the image of the contour line (contour line extraction process) (ST124).

[0134] Next, processor 23 generates graphic data including a figure approximating the contour of the object to be illustrated, based on the contour data generated in the contour extraction process (graphical process) (ST125).

[0135] Next, processor 23 determines whether or not the process for all objects of the type designated as the object to be depicted has been completed (ST126). If the process for all objects of the type designated as the object to be depicted has not been completed (No in ST126), processor 23 adds 1 to N (ST127), returns to ST122, and proceeds to the process for the next individual. On the other hand, if the process for all objects of the type designated as the object to be depicted has been completed (Yes in ST126), this flow ends.

[0136] Next, another example of an object that becomes elongated when projected will be described. Fig. 19 is an explanatory diagram showing a point cloud data confirmation screen 201 displayed on the user terminal 1. Fig. 20 is an explanatory diagram showing an overview of the processing performed by the server 2. Fig. 21 is an explanatory diagram showing a layout plan confirmation screen 401 displayed on the user terminal 1.

[0137] As shown in Fig. 19, in this example, as in the previous example (see Fig. 4), a shelf, a desk, and a chair are arranged on the floor of a room that is a target location, and a door is provided on the wall. Meanwhile, in this example, a window is provided on the wall of the room that is a target location, and a curtain is provided to cover the window. Here, it is assumed that a desk, a chair, a shelf, and a curtain are specified as the types of objects to be illustrated.

[0138] In this case, as shown in Fig. 20(A), a long and thin image representing the curtain appears in the planar projection image. As shown in Fig. 20(B), in the contour data obtained by extracting the contour from the planar projection image, a long and thin rectangular contour image appears within the range of the curtain. When the contour data is subjected to a graphic processing, the long and thin rectangular contour image is approximated by a polygonal figure, and the curtain is expressed as a long and thin rectangular figure. However, editing work that is performed by selecting vertices of a long and thin rectangular figure is troublesome.

[0139] Therefore, in this embodiment, as shown in Fig. 20(C), an image of a contour line forming a long and thin rectangle that appears within the range of an object in the contour line data is approximated to a straight line (line segment). Specifically, when the image of the contour line is approximated to a long and thin rectangular figure, the long and thin rectangular figure is replaced with a straight line that is its center line. Note that the center line of a rectangle is a straight line connecting the centers of the short sides of the rectangle. Also, for example, when the flattening ratio (ratio of width to length) of the rectangle is equal to or less than a predetermined threshold value, the image of the contour line forming a long and thin rectangle is approximated to a straight line. Also, the outputted figure data includes the coordinates of the two end points of the line.

[0140] 21, on the layout chart confirmation screen 401, the curtain is displayed as a straight line figure in the layout chart 402. In this case, editing work is performed by selecting the end points of the line in the editing state, so editing work is easy.

[0141] Thus, in this embodiment, when the image of the contour in the contour data is in the form of an elongated rectangle, the image of the contour is approximated to a straight line (line segment) passing through its center, but when the image of the contour is in the form of an elongated curved band, it may be approximated to a continuous straight line (broken line) in which multiple points are connected to follow the curved shape.

[0142] Second embodiment Next, a second embodiment will be described. Note that the points not mentioned here are the same as those in the above embodiment. Fig. 22 is a block diagram showing a schematic configuration of a user terminal 1 and a server 2 according to the second embodiment.

[0143] In this embodiment, similarly to the first embodiment (see FIG. 2), the processor 23 of the server 2 performs point cloud data generation processing, layout drawing generation processing, screen generation processing, layout file generation processing, processing condition setting processing, etc. Furthermore, in this embodiment, the processor 23 performs layout editing processing. In the layout editing processing, the processor 23 corrects the layout drawing (layout model) of the target location in response to an editing operation by the user on the user terminal 1.

[0144] Next, a description will be given of the layout drawing confirmation screen 401 displayed on the user terminal 1. Fig. 23 is an explanatory diagram showing the layout drawing confirmation screen 401.

[0145] A layout drawing 402 generated by the layout drawing generation process is displayed on the layout drawing confirmation screen 401. The user can visually check the displayed layout drawing 402 to confirm whether the layout drawing 402 has been generated appropriately.

[0146] Furthermore, the layout drawing confirmation screen 401 is provided with an "Edit layout drawing" button 117. When the user confirms that the layout drawing 402 has not been generated appropriately, the user operates the "Edit layout drawing" button 117. This transitions to a layout editing screen 501 (see FIG. 24), where the user can perform editing work to eliminate the problem with the layout drawing 402.

[0147] Fig. 24 is an explanatory diagram showing a layout editing screen 501 displayed on the user terminal 1. Fig. 25 is an explanatory diagram showing a layout drawing 502 in an edited state. Fig. 26 is an explanatory diagram showing a layout drawing confirmation screen 401 displayed on the user terminal 1.

[0148] A layout diagram 502 in an edited state is displayed on a layout editing screen 501. In the layout diagram 502 in an edited state, vertices of a polygonal figure representing an object placed in a target location are displayed. In the layout diagram 502 in an edited state, a user can perform operations such as adding, deleting, moving, copying, connecting vertices, and disconnecting vertices.

[0149] In this example, since the desk and chair are represented by one figure, the desk and chair are corrected to be represented by separate figures. Specifically, from the initial state shown in Fig. 25(A), first, an operation is performed to cut the line connecting the two vertices as shown in Fig. 25(B). Next, an operation is performed to move the vertex. Next, an operation is performed to connect the two vertices with a line as shown in Fig. 25(C), resulting in the edit end state shown in Fig. 25(D).

[0150] In the layout diagram 502 in the edited state, the figures representing the objects may be displayed in a different manner for each individual object, for example, with the vertices displayed in different colors.

[0151] When editing of the layout diagram 502 is completed in this manner, the layout diagram confirmation screen 401 shown in Fig. 26 is displayed. The layout diagram 402 in the edited state is displayed on the layout diagram confirmation screen 401. In the layout diagram 402, the desk and the chair have been corrected so as to be represented by separate rectangles.

[0152] As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. In addition, it is also possible to combine the components described in the above embodiments to create new embodiments. [Industrial Applicability]

[0153] The drawing creation device and drawing creation method of the present invention have the effect of enabling a layout drawing including objects placed at a target location to be created with minimal effort, and are useful as a drawing creation device and drawing creation method that generates a layout drawing showing the arrangement of objects at a target location based on point cloud data acquired by performing 3D measurement processing of the target location. [Explanation of symbols]

[0154] 1: User terminal 2: Server (drawing device) 11: Device body 12: Camera 13: Display 14: Input devices 15: Storage section 16: Processor 17: Communications Department 21: Communications Department 22: Storage section 23: Processor 101: Shooting screen 102: Photographed image 201: Point cloud data confirmation screen 202: Point cloud image 301: Setting screen 302: Pixel size setting unit 303: Target object setting unit 304: Processing mode setting section 401: Layout diagram confirmation screen 402: Layout diagram 501: Layout editing screen 502: Layout diagram

Claims

1. A drawing creation device that uses a processor to execute a process of generating a layout drawing that represents an arrangement situation of objects in a target location based on point cloud data acquired by performing a three-dimensional measurement process on the target location, The processor: setting a processing mode for generating at least the object to be illustrated and the layout drawing in response to a user's operation input on a setting screen relating to processing conditions; Recognizing an object present at a target location based on the point cloud data; generating a planar projection image by projecting each point of the point cloud data representing the recognized object onto a predetermined plane; A drawing creation device that generates and outputs the layout drawing based on the planar projection image.

2. The processor:

2. The drawing creation device according to claim 1, wherein an actual length represented by one pixel of the planar projection image is set as the processing condition.

3. The processor:

2. The drawing creation apparatus according to claim 1, wherein the processing conditions include setting a type of object to be drawn in the layout drawing.

4. The processor: The drawing creation device according to claim 1, characterized in that the processing conditions include a drawing mode relating to the procedure for generating figures of each object to be drawn on the layout drawing from the point cloud data, and a display mode relating to whether or not to display characters indicating the type of object on the layout drawing.

5. The processor: Extracting a contour line of an object from the planar projection image; 2. The drawing creation device according to claim 1, wherein the layout drawing is generated by expressing an object using a figure that approximates the contour line.

6. The processor:

6. The drawing creation device according to claim 5, wherein the layout drawing is generated in which an object is represented by a polygon having the contour line as a side as the figure approximating the contour line.

7. The processor:

6. The drawing creation device according to claim 5, wherein the layout drawing is generated in which an object is represented by a straight line passing through the center of an image of the contour line as the figure approximating the contour line.

8. The processor: generating the planar projection image of the entire object to be illustrated by collectively projecting each point of the point cloud data representing the object to be illustrated; 2. The drawing creation device according to claim 1, wherein the layout drawing is generated based on the planar projection image, the layout drawing including a graphic representing the overall outline of an object to be drawn.

9. The processor: generating the planar projection image for each type of object to be illustrated by classifying and projecting each point of the point cloud data representing the object to be illustrated; 2. The drawing creation device according to claim 1, wherein the layout drawing is generated based on the planar projection image, the layout drawing including a graphic representing the outer shape of each type of object to be drawn.

10. The processor: generating the planar projection image for each individual object to be illustrated by dividing each point of the point cloud data representing the object to be illustrated and projecting the individual points, 2. The drawing creation device according to claim 1, wherein the layout drawing is generated based on the planar projection image, the layout drawing including a graphic representing the outer shape of each individual object to be drawn.

11. The processor:

2. The drawing creation device according to claim 1, wherein a character representing the type of an object is superimposed near the object drawn on the layout drawing.

12. A drawing creation method for generating a layout drawing showing an arrangement situation of objects in a target location based on point cloud data acquired by performing a three-dimensional measurement process on the target location, the method comprising: setting a processing mode for generating at least the object to be illustrated and the layout drawing in response to a user's operation input on a setting screen relating to processing conditions; Recognizing an object present at a target location based on the point cloud data; generating a planar projection image by projecting each point of the point cloud data representing the recognized object onto a predetermined plane; A drawing creation method, characterized in that the layout drawing is generated and output based on the planar projection image.

Images