Three-dimensional shape generation device, three-dimensional shape generation system, three-dimensional shape generation method and program

Abstract

To appropriately generate three-dimensional shape information according to a user's level of skill and knowledge.SOLUTION: A management server 5, which is an example of three-dimensional shape generation apparatus includes a processing unit 53, which is an example of three-dimensional information generation means, configured to generate three-dimensional shape information on the basis of point cloud information indicating a three-dimensional point cloud. The processing unit 53 executes first processing or second processing on the basis of an instruction operation received through a confirmation button 1300, which is an example of instruction receiving screen for receiving an instruction operation to designate the first processing or the second processing, the first processing being configured to execute all or a part of generation processing for generating three-dimensional shape information on the basis of operation input, the second processing being configured to execute the generation processing on the basis of storage processing stored in advance without depending on the operation input.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a three-dimensional shape generation device, a three-dimensional shape generation system, a three-dimensional shape generation method, and a program. [Background technology]

[0002] Patent document 1 describes a three-dimensional model generation device that generates a three-dimensional model having at least the outer shape of an object based on three-dimensional point cloud data of the object, and that includes an abstract three-dimensional model acquisition means that acquires a dimension-variable abstract three-dimensional model having an outer shape corresponding to each outer shape of the object, and a three-dimensional model determination means that determines the match between the abstract three-dimensional model and the three-dimensional point cloud data while changing the dimensions of the abstract three-dimensional model, and determines the abstract three-dimensional model that has the highest match as the three-dimensional model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2020-197979 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to appropriately generate three-dimensional shape information according to the user's level of skill and knowledge. [Means for solving the problem]

[0005] The three-dimensional shape generating device according to the present invention includes a three-dimensional information generating means for generating three-dimensional shape information based on point cloud information indicating a three-dimensional point cloud, and the three-dimensional information generating means It is composed of multiple generation processes, including a modeling process that matches a specific area in a 3D point cloud with a model shape based on operation input and replaces the specific area with the model shape,The first process or the second process is executed based on an instruction operation received on an instruction reception screen that receives an instruction operation to instruct a first process that executes all or part of a generation process for generating three-dimensional shape information based on an operation input, or a second process that executes a generation process based on a pre-stored storage process without based on an operation input. [Effects of the Invention]

[0006] According to the present invention, three-dimensional shape information can be appropriately generated according to the user's level of skill and knowledge. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is an overall configuration diagram of a three-dimensional shape generation system according to an embodiment of the present invention; [Figure 2] FIG. 2 is a hardware configuration diagram of a terminal device and a management server according to the embodiment. [Figure 3] FIG. 1 is a functional block diagram of a three-dimensional shape generation system according to an embodiment of the present invention. [Figure 4] FIG. 4 is a conceptual diagram illustrating an example of a setting information management table according to the embodiment. [Figure 5] FIG. 10 is a sequence diagram illustrating an example of a three-dimensional shape generation process according to the present embodiment. [Figure 6] FIG. 4 is an explanatory diagram of a setting screen according to the embodiment. [Figure 7] 10 is a flowchart illustrating an example of a three-dimensional shape generation process according to the present embodiment. [Figure 8] FIG. 4 is an explanatory diagram of a registration process according to the present embodiment. [Figure 9] 5A to 5C are explanatory diagrams of a noise removal process according to the present embodiment. [Figure 10] FIG. 10 is an explanatory diagram of a segmentation process according to the present embodiment. [Figure 11] FIG. 2 is an explanatory diagram of a modeling process according to the present embodiment. [Figure 12] 10 is a flowchart illustrating an example of a three-dimensional shape generation process according to a modified example of the present embodiment. [Figure 13] FIG. 10 is an explanatory diagram of a display screen according to a second modified example of the present embodiment. [Figure 14] FIG. 10 is an explanatory diagram of a modeling process according to a second modified example. [Figure 15] 10 is a flowchart showing an example of a three-dimensional shape generation process according to a second modified example. DETAILED DESCRIPTION OF THE INVENTION

[0008] In industries such as civil engineering and construction, BIM / CIM is being promoted in order to address issues such as the declining birthrate and aging population, and to improve labor productivity.

[0009] BIM is an abbreviation for Building Information Modeling, and is a solution that uses a building database that adds attribute data such as cost, finish, and management information to a three-dimensional digital model of a building created on a computer (hereinafter referred to as a 3D model), allowing information to be used in all processes from building design and construction to maintenance and management.

[0010] CIM is an abbreviation for Construction Information Modeling, and is a solution for the civil engineering field (general infrastructure such as roads, electricity, gas, and water) that was proposed following the example of BIM, which was being promoted in the construction field. Like BIM, CIM is being worked on as a way to improve the efficiency and sophistication of the entire construction production system by sharing information between stakeholders, mainly through 3D models.

[0011] An important aspect of promoting BIM / CIM is how to easily create 3D models of buildings and public facilities.

[0012] When building a new building, it is relatively easy to convert to BIM / CIM because the finished product can be modeled from scratch using 3D CAD software. On the other hand, in the case of existing buildings, the original blueprints may no longer exist, or may differ from the blueprints due to renovations, making the conversion to BIM / CIM more difficult. Converting existing buildings to BIM in this way is called As-Build BIM, and is an important issue for promoting future BIM / CIM conversion.

[0013] One method for realizing As-Build BIM is to use a laser scanner (hereafter referred to as LS) to measure the space and create a CAD model from the measured point cloud. Previously, the space was measured using photographs and tape measures, and sketched to restore the space, which resulted in a huge amount of work costs, but the introduction of LS has dramatically improved the efficiency of this work.

[0014] While As-Build using LS makes modeling easier, it also creates a new task called point cloud processing, which did not exist in conventional work.General point cloud processing involves multi-point measurement using LS, aligning each point cloud, creating an integrated point cloud, removing unnecessary point clouds such as noise, and finally converting the point cloud into a CAD model.

[0015] These processes are performed using commercially available point cloud processing software, but point cloud processing software is multifunctional and each function has many parameters to set, making it difficult for inexperienced people to use.

[0016] Furthermore, although the system has the functionality to automate each process, the automation can fail the further the situation deviates from ideal, such as when points cannot be acquired due to occlusion, etc. In such cases, it requires labor and time to manually fill in missing areas by comparing the point cloud with the photograph.

[0017] In view of the above problems, the present embodiment aims to generate three-dimensional shape information according to the user's level of skill and knowledge.

[0018] 1 is a diagram showing the overall configuration of a three-dimensional shape generation system according to an embodiment of the present invention. The three-dimensional shape generation system 1 of this embodiment is constructed by a terminal device 3, which is an example of a communication terminal, and a management server 5, which is an example of a three-dimensional shape generation device.

[0019] The management server 5 is an example of a three-dimensional shape generation device that uses model shape information indicating a three-dimensional model shape to generate three-dimensional shape information indicating a three-dimensional shape corresponding to a three-dimensional point cloud.

[0020] Here, a three-dimensional point cloud is a collection of coordinate points in a virtual three-dimensional space that can be handled by a computer or the like. A three-dimensional point cloud is also called a point cloud. A three-dimensional point cloud is a collection of coordinate points corresponding to measurement points on the surface of an object when the space containing the object is measured using a laser scanner or the like. Color information may also be added to each coordinate point, and the RGB values ​​of each coordinate point may be added as color information.

[0021] Although the example shows the 3D point cloud being measured using a laser scanner LS, other optical or mechanical measurement methods can also be used. Optical measurement methods include using a stereo camera or Visual SLAM.

[0022] Furthermore, three-dimensional shape information is information that indicates the three-dimensional shape of an object that can be handled by a computer or the like. Information that indicates the three-dimensional shape is information that can geometrically identify the three-dimensional shape; for example, in the case of a sphere, the coordinates of the center and the radius correspond to information that indicates the three-dimensional shape. When the three-dimensional shape of an object is represented by a polyhedron (polygon), the coordinate points of each vertex of the polyhedron are an example of information that indicates the three-dimensional shape. In addition, any information that can uniquely define the shape of the object can be used as information that represents the three-dimensional shape.

[0023] The three-dimensional shape information may include information relating to the color and material of the object in addition to information indicating the three-dimensional shape of the object.

[0024] The three-dimensional model shape is a model such as a template used to generate three-dimensional shape information from a three-dimensional point cloud. The model shape information is information that indicates the three-dimensional model shape, and one piece of model shape information corresponds to one three-dimensional model shape.

[0025] The terminal device 3 and the management server 5 can communicate with each other via a communication network 100. The communication network 100 is constructed using the Internet, a mobile communication network, a LAN (Local Area Network), etc. The communication network 100 may include not only wired communication networks but also wireless communication networks such as 3G (3rd Generation), WiMAX (Worldwide Interoperability for Microwave Access), and LTE (Long Term Evolution). The terminal device 3 can also communicate with each other using short-range communication technologies such as NFC (Near Field Communication) (registered trademark).

[0026] <Hardware configuration> 2 is a diagram showing the hardware configuration of the terminal device and management server according to this embodiment. Each piece of hardware configuration of the terminal device 3 is indicated by a reference number in the 300s. Each piece of hardware configuration of the management server 5 is indicated by a reference number in the 500s in parentheses.

[0027] The terminal device 3 includes a CPU (Central Processing Unit) 301, a ROM (Read Only Memory) 302, a RAM (Random Access Memory) 303, an HD (Hard Disk) 304, an HDD (Hard Disk Drive) 305, a recording medium 306, a media I / F 307, a display 308, a network I / F 309, a keyboard 311, a mouse 312, a CD-RW (Compact Disc-ReWritable) drive 314, and a bus line 310.

[0028] Of these, the CPU 301 controls the overall operation of the terminal device 3. The ROM 302 stores programs used to drive the CPU 301. The RAM 303 is used as a work area for the CPU 301. The HD 304 stores various data such as programs. The HDD 305 controls the reading and writing of various data from and to the HD 304 under the control of the CPU 301. The media I / F 307 controls the reading and writing (storage) of data from and to a recording medium 306 such as a flash memory. The display 308 displays various information such as a cursor, menus, windows, characters, and images. The network I / F 309 is an interface for data communication using the communication network 100. The keyboard 311 is a type of input means equipped with multiple keys for inputting characters, numbers, various instructions, etc. The mouse 312 is a type of input means for selecting and executing various instructions, selecting processing targets, moving the cursor, etc. The CD-RW drive 314 controls the reading and writing of various data from and to a CD-RW 513, which is an example of a removable recording medium.

[0029] The management server 5 also includes a CPU 501, a ROM 502, a RAM 503, a HD 504, a HDD 505, a recording medium 506, a media I / F 507, a display 508, a network I / F 509, a keyboard 511, a mouse 512, a CD-RW drive 514, and a bus line 510. These components have the same configuration as the above-described components (CPU 301, ROM 302, RAM 303, HD 304, HDD 305, recording medium 306, media I / F 307, display 308, network I / F 309, keyboard 311, mouse 312, CD-RW drive 314, and bus line 310), and therefore description thereof will be omitted.

[0030] It should be noted that a CD-R drive or the like may be used instead of the CD-RW drive 314 (514). The terminal device 3 and management server 5 may each be constructed by a single computer, or may be constructed by multiple computers in which each unit (function, means, or memory unit) is divided and arbitrarily assigned.

[0031] FIG. 3 is a functional block diagram of the three-dimensional shape generation system according to this embodiment.

[0032] As shown in Fig. 3, the terminal device 3 has a transmitting / receiving unit 31, a receiving unit 32, a display control unit 34, and a storage / reading unit 39. Each of these units is a function or a means for performing the function, which is realized when any of the components shown in Fig. 2 operates in response to an instruction from the CPU 301 in accordance with a program loaded from the HD 304 onto the RAM 303. The terminal device 3 also has a storage unit 3000 constructed by the RAM 303 and HD 304 shown in Fig. 2.

[0033] (Functional configuration of terminal device) Next, each component of the terminal device 3 will be described.

[0034] The transmitter / receiver unit 31 is an example of a transmitting means, and is realized by instructions from the CPU 301 shown in Figure 2 and the network I / F 309, and transmits and receives various data (or information) with other terminals, devices, or systems via the communication network 100.

[0035] The reception unit 32 is an example of a reception means, and is realized mainly by commands from the CPU 301 shown in FIG. 2, as well as the keyboard 311 and mouse 312, and receives various inputs from the user.

[0036] The display control unit 34 is an example of a display control means, and is realized by commands from the CPU 301 shown in FIG. 2, and causes the display 308, which is an example of a display unit, to display various images and screens.

[0037] The storage / reading unit 39 is an example of a storage control means, and is executed by commands from the CPU 301 shown in FIG. 2, as well as the HDD 305, the media I / F 307, and the CD-RW drive 314, to store various data in the storage unit 3000, the recording medium 306, and the CD-RW 313, and to read various data from the storage unit 3000, the recording medium 306, and the CD-RW 313.

[0038] <Management server functional configuration> The management server 5 has a transmitting / receiving unit 51, a processing unit 53, a determining unit 55, a setting unit 57, and a storage / reading unit 59. Each of these units is a function or a means for performing a function that is realized when any of the components shown in Fig. 2 operates in response to an instruction from the CPU 501 in accordance with a program loaded from the HD 504 onto the RAM 503. The management server 5 also has a memory unit 5000 constructed by the HD 504 shown in Fig. 2. The memory unit 5000 is an example of a memory means.

[0039] (Management server functional configuration) Next, we will explain each component of the management server 5. The management server 5 may be configured to distribute each function among multiple computers. Furthermore, although the management server 5 will be explained as a server computer existing in a cloud environment, it may also be a server existing in an on-premise environment.

[0040] The transmitter / receiver unit 51 is an example of a transmitting means, and is realized by instructions from the CPU 501 shown in Figure 2 and the network I / F 509, and transmits and receives various data (or information) with other terminals, devices, or systems via the communication network 100.

[0041] The processing unit 53 is realized by instructions from the CPU 501 shown in Fig. 2, and performs various processes described below. The processing unit 53 is an example of a three-dimensional information generating means that generates three-dimensional shape information.

[0042] The determination unit 55 is realized by instructions from the CPU 501 shown in FIG. 2, and makes various determinations, which will be described later.

[0043] The setting unit 57 is realized by commands from the CPU 501 shown in FIG. 2, and performs various settings and decisions, which will be described later.

[0044] 2, and by the HDD 505, media I / F 507, and CD-RW drive 514, to perform processing such as storing various data in the storage unit 5000, recording medium 506, and CD-RW 513, and reading various data from the storage unit 5000, recording medium 506, and CD-RW 513. The storage unit 5000, recording medium 506, and CD-RW 513 are examples of storage means.

[0045] The storage unit 5000 stores a setting information management DB 5001, which is configured by a setting information management table, a storage processing management DB 5002, a point cloud management DB 5003, and a three-dimensional shape management DB 5004.

[0046] The setting information management DB 5001 stores and manages various types of information, the storage processing DB 5002 stores and manages various processing programs for generating three-dimensional shapes, the point cloud management DB 5003 stores and manages three-dimensional point cloud information for generating three-dimensional shapes, and the three-dimensional shape management DB 5004 stores and manages three-dimensional shape information.

[0047] Here, the various processing programs stored and managed in the storage processing DB 5002 are an example of storage processing that executes a part of the generation processing for generating three-dimensional shape information.

[0048] FIG. 4 is a conceptual diagram showing an example of a setting information management table according to this embodiment.

[0049] The setting information management table is a table for managing three-dimensional point cloud data for generating a three-dimensional shape, and the execution order and processing mode of some of the generation processes for generating the three-dimensional shape. A setting information management DB 5001 configured with a setting information management table such as that shown in Fig. 4 is constructed in the storage unit 5000. In this setting information management table, the file names of the three-dimensional point cloud data and the execution order and processing mode of some of the generation processes for generating the three-dimensional shape are associated and managed for each user ID.

[0050] Some generation processes for generating a three-dimensional shape include registration, denoising, segmentation, and modeling processes.

[0051] The registration process converts multiple 3D point clouds into a single integrated 3D point cloud, while the noise removal process removes unnecessary points from the 3D point cloud.

[0052] The segmentation process is a process of labeling specific points in a 3D point cloud to make them distinguishable from other point clouds, and multiple specific point clouds may be labeled differently to make them distinguishable from each other. The segmentation process may also be performed in conjunction with a clustering process that groups closely spaced points from the labeled point cloud.

[0053] The modeling process is a process of matching specific points in the three-dimensional point cloud with the model shape and replacing the specific points with the model shape.

[0054] The processing modes include manual processing in which all or part of the generation processing for generating three-dimensional shape information is executed based on operation input without being based on storage processing; automatic processing in which all or part of the generation processing for generating three-dimensional shape information is executed based on pre-stored storage processing without being based on operation input; and mixed processing in which all or part of the generation processing for generating three-dimensional shape information is executed based on operation input and storage processing.

[0055] The manual process and the mixed process are examples of a first process that executes all or part of the generation process for generating three-dimensional shape information based on operation input, and the automatic process is an example of a second process that executes all or part of the generation process for generating three-dimensional shape information based on a pre-stored storage process without based on operation input.

[0056] If the management server 5 is configured to distribute each function among multiple computers, a first computer may execute all or part of the generation process for generating three-dimensional shape information based on operation input without being based on storage processing, and a second computer may execute all or part of the generation process for generating three-dimensional shape information based on pre-stored storage processing without being based on operation input.

[0057] FIG. 5 is a sequence diagram showing an example of a three-dimensional shape generation process according to this embodiment.

[0058] The reception unit 32 of the communication terminal 3 receives an input operation related to the user information of the user on the input / output screen displayed on the display 308 (step S1). The transmission / reception unit 31 transmits a request for a setting screen including the user information received in step S1 to the management server 5 of the communication terminal 3, and the transmission / reception unit 51 of the management server 5 receives the request transmitted from the communication terminal 3 (step S2).

[0059] Next, the memory / read unit 59 of the management server 5 searches the setting information management DB 5001 using the user information included in the request received in step S2 as a search key, and reads out the file name, execution order and processing mode of the generation process of the three-dimensional point cloud data associated with the user information included in the request. The setting unit 57 of the management server 5 then generates a setting screen based on the file name, execution order and processing mode read out by the memory / read unit 59 (step S3).

[0060] This setting screen includes a point cloud setting screen that accepts a point cloud setting operation that sets a three-dimensional point cloud used to generate three-dimensional shape information, a processing setting screen that accepts a processing setting operation that sets the execution order and processing mode of some of the generation processes for generating three-dimensional shape information, and an instruction acceptance screen that accepts an instruction operation that instructs a first process or a second process.

[0061] The transmitting / receiving unit 51 transmits setting screen information relating to the setting screen generated in step S3 to the communication terminal 3, and the transmitting / receiving unit 31 of the communication terminal 3 receives the setting screen information transmitted from the management server 5 (step S4). Step S4 is an example of a transmitting step of transmitting instruction acceptance screen information relating to the instruction acceptance screen, and the transmitting / receiving unit 51 is an example of a transmitting means.

[0062] Next, the display control unit 34 of the communication terminal 3 causes the display 308 to display the setting screen received in step S4 (step S5). The accepting unit 32 of the communication terminal 3 accepts a predetermined input operation by the user on the displayed setting screen. This input operation includes a point cloud setting operation, a processing setting operation, and an instruction operation to instruct the first processing or the second processing. Step S5 is an example of a accepting step of accepting an instruction operation to instruct the first processing or the second processing.

[0063] The transmitter / receiver 31 transmits input information related to the input operation received by the receiver 32 to the management server 5, and the transmitter / receiver 51 of the management server 5 receives the input information transmitted from the communication terminal 3 (step S6). This input information includes point cloud setting information indicating the three-dimensional point cloud set by the point cloud setting operation and processing setting information indicating the execution order and processing mode of the generation process set by the processing setting operation.

[0064] The storage / reading unit 59 of the management server 5 updates the execution order and processing mode of the generation process associated with the user information and stored in the setting information management DB 5001 based on the processing setting information included in the input information received in step S6 (step S7).

[0065] Next, the storage / readout unit 59 of the management server 5 searches the point cloud management DB 5003 using the point cloud setting information included in the input information received in step S6 as a search key to read out the three-dimensional point cloud data associated with the point cloud setting information. The storage / readout unit 59 also searches the storage / readout management DB 5002 using the processing mode of the generation process in the processing setting information included in the input information received in step S6 as a search key to read out the processing program associated with the processing setting information. The processing unit 53 of the management server 5 generates three-dimensional shape information based on the three-dimensional point cloud data and processing program read out from the storage / readout unit 59, and the execution order and processing mode of the generation process in the processing setting information included in the input information received in step S6 (step S8).

[0066] Step S8 is an example of a three-dimensional information generation step that generates three-dimensional shape information for executing the first process or the second process based on point cloud information indicating a three-dimensional point cloud and an instruction operation accepted on an instruction acceptance screen that accepts an instruction operation for instructing the first process or the second process.

[0067] If the processing mode includes manual processing and mixed processing, the setting unit 57 of the management server 5 generates an operation screen that accepts operation input for generating three-dimensional shape information, and the transmitting / receiving unit 51 transmits operation screen information related to the operation screen to the communication terminal 3 (step S9).

[0068] The transmitter / receiver 31 of the communication terminal 3 receives the operation screen information transmitted from the management server 5, the display control unit 34 of the communication terminal 3 displays the received operation screen on the display 308, and the reception unit 32 of the communication terminal 3 receives the user's specified input operation on the displayed operation screen (step S10).

[0069] This input operation includes an operation input for executing all or part of the generation process for generating three-dimensional shape information.

[0070] The transmitter / receiver 31 transmits input information relating to the input operation received by the reception unit 32 to the management server 5, and the transmitter / receiver 51 of the management server 5 receives the input information transmitted from the communication terminal 3 (step S11).

[0071] This input information includes operation input information by operation input for executing all or part of the generation process for generating three-dimensional shape information, and the processing unit 53 of the management server 5 generates the three-dimensional shape information based on the operation input information included in the input information received in step S11. The operation input information is an example of history information when the three-dimensional information generation means executed the first process.

[0072] The communication terminal 3 and the management server 5 repeatedly execute steps S8 to S11 as necessary.

[0073] The processing unit 53 converts the generated three-dimensional shape information into a CAD format or the like, and the storage / readout unit 59 stores the converted three-dimensional shape information in the three-dimensional shape management DB 5004, the recording medium 506, or the CD-RW 513 (step S12). The CAD format is three-dimensional shape information that can be handled by 3D CAD. When the three-dimensional shape information is to be handled by a commercially available 3D CAD, the three-dimensional shape information is converted into the 3D CAD format.

[0074] The storage / readout unit 59 updates the processing program stored in the storage processing management DB 5002 based on the operation input for executing all or part of the generation processing for generating the three-dimensional shape information in step S10 (step S13).

[0075] The storage / readout unit 59 is an example of an updating means that updates the storage process based on the history information when the three-dimensional information generating means executed the first process.

[0076] The transmitter / receiver 51 transmits the determined three-dimensional shape information to the communication terminal 3 (step S14). The transmitter / receiver 31 of the communication terminal 3 receives the three-dimensional shape information transmitted from the management server 5, and the display controller 34 of the communication terminal 3 displays the received three-dimensional shape on the display 308 (step S15).

[0077] 6 is an explanatory diagram of a display screen according to this embodiment, showing a display screen 1000 displayed on the display 308 of the terminal device 3 in step S5 of the sequence diagram shown in FIG.

[0078] The display control unit 34 of the communication terminal 3 causes the display screen 1000 to display a user information display screen 1100, a setting screen 1200, and a confirm button 1300, which is an example of an instruction acceptance screen.

[0079] The setting screen 1200 includes a point cloud setting screen 1210 and a processing setting screen 1220 .

[0080] The point cloud setting screen 1210 is a screen for accepting a point cloud setting operation for setting point cloud information indicating a three-dimensional point cloud used to generate three-dimensional shape information, and the display control unit 34 displays point cloud setting boxes 1212 and 1214 in association with the file names of the plurality of point cloud data read by the storage / reading unit 59. A plurality of point cloud setting boxes 1212 and 1214 can be set.

[0081] The process setting screen 1220 is a screen for accepting a process setting operation for setting the execution order and processing mode of some of the generation processes for generating three-dimensional shape information.

[0082] The processing setting screen 1220 includes an automatic processing batch setting box 1231, an automatic registration processing setting box 1232, an automatic noise removal processing setting box 1233, an automatic segmentation processing setting box 1234, and an automatic modeling processing setting box 1235 for setting automatic processing that executes generation processing for generating three-dimensional shape information based on storage processing without being based on operational input.

[0083] The processing setting screen 1220 also includes a manual processing batch setting box 1241, a manual registration processing setting box 1242, a manual noise removal processing setting box 1243, a manual segmentation processing setting box 1244, and a manual modeling processing setting box 1245 for setting manual processing that executes generation processing for generating three-dimensional shape information based on operation input without based on storage processing.

[0084] The processing setting screen 1220 further includes a mixed processing batch setting box 1251, a mixed registration processing setting box 1252, a mixed noise removal processing setting box 1253, a mixed segmentation processing setting box 1254, and a mixed modeling processing setting box 1255 for setting mixed processing that executes generation processing to generate three-dimensional shape information based on operation input and storage processing.

[0085] The processing setting screen 1220 also includes a registration processing order setting box 1262, a noise removal processing order setting box 1263, a segmentation processing order setting box 1264, and a modeling processing order setting box 1265 for setting the order in which multiple generation processes are to be performed.

[0086] The automatic processing batch setting box 1231 is a screen for accepting a setting operation for setting the automatic registration processing setting box 1232 to the automatic modeling processing setting box 1235 all at once.

[0087] The automatic registration process setting box 1232 is a screen for accepting a setting operation for setting the execution of the registration process by automatic processing.

[0088] The automatic noise removal processing setting box 1233 is a screen for accepting a setting operation for setting the automatic execution of noise removal processing.

[0089] The automatic segmentation processing setting box 1234 is a screen for accepting a setting operation for setting that the segmentation processing is to be executed automatically.

[0090] The automatic modeling process setting box 1235 is a screen for accepting a setting operation for setting that the modeling process is to be executed automatically.

[0091] The manual processing batch setting box 1241 is a screen for accepting a setting operation for setting the manual registration processing setting box 1242 to the manual modeling processing setting box 1245 all at once.

[0092] The manual registration process setting box 1242 is a screen for accepting a setting operation for setting that the registration process is to be performed manually.

[0093] The manual noise removal process setting box 1243 is a screen for accepting a setting operation for setting that the noise removal process is to be executed manually.

[0094] The manual segmentation process setting box 1244 is a screen for accepting a setting operation for setting that the segmentation process is to be executed manually.

[0095] The manual modeling process setting box 1245 is a screen for accepting a setting operation for setting that the modeling process is to be executed manually.

[0096] The mixed processing batch setting box 1251 is a screen for accepting a setting operation for setting the mixed registration processing setting box 1252 to the mixed modeling processing setting box 1255 all at once.

[0097] The mixed registration processing setting box 1252 is a screen for accepting a setting operation for setting that the registration processing is to be performed by mixed processing.

[0098] The mixed noise removal processing setting box 1253 is a screen for accepting a setting operation for setting that noise removal processing is to be performed by mixed processing.

[0099] The mixed segmentation processing setting box 1254 is a screen for accepting a setting operation for setting that the segmentation processing is to be performed by mixed processing.

[0100] The mixed modeling process setting box 1255 is a screen for accepting a setting operation for setting that the modeling process is to be executed by mixed processing.

[0101] Here, in the automatic processing batch setting box 1231, the manual processing batch setting box 1241, and the mixed processing batch setting box 1251, all generation processes include at least a registration process, a noise removal process, a segmentation process, and a modeling process.

[0102] The registration process order setting box 1262 is a screen for accepting a setting operation for setting the order in which the registration processes are to be executed.

[0103] The noise removal process order setting box 1263 is a screen for accepting a setting operation for setting the order in which the noise removal processes are to be executed.

[0104] The segmentation process order setting box 1264 is a screen for accepting a setting operation for setting the order in which the segmentation processes are to be executed.

[0105] The modeling process order setting box 1265 is a screen for accepting a setting operation for setting the order in which the modeling processes are to be executed.

[0106] When various setting boxes are pointed to with a pointing device such as a mouse 312, the display control unit 34 displays a check mark in the various setting box, and the reception unit 32 receives various setting operations, and when the confirm button 1300 is operated, the various setting operations are confirmed.

[0107] Then, as described in step S6 of FIG. 5, the transmitting / receiving unit 31 transmits to the management server 5 input information including various setting information resulting from the various setting operations received by the receiving unit 32.

[0108] The processing setting screen 1220 may be configured to accept a setting operation for setting each of the registration processing, the noise removal processing, the segmentation processing, and the modeling processing to be executed multiple times.

[0109] Furthermore, for example, if no setting operation is performed in any of the automatic processing batch setting box 1231, the manual processing batch setting box 1241, the mixed processing batch setting box 1242, and the mixed registration processing setting box 1252, the registration processing is set not to be executed. The same applies to the noise removal processing, the segmentation processing, and the manual modeling processing.

[0110] FIG. 7 is a flowchart showing an example of a three-dimensional shape generation process according to this embodiment, and shows an example of a process corresponding to step S8 in FIG.

[0111] The processing unit 53 of the management server 5 acquires the three-dimensional point cloud data read out from the storage / readout unit 59 (step S21), and the setting unit 57 sets the order N for executing various generation processes to 1 (step S22).

[0112] The setting unit 57 determines the Nth generation process based on the execution order of the generation processes in the process setting information received in step S6 (step S23).

[0113] The determination unit 55 determines whether the processing mode of the Nth generation process determined in step S23 is manual processing, based on the processing mode of the generation process in the processing setting information received in step S6 (step S24).

[0114] If the processing mode is manual processing, the setting unit 57 of the management server 5 generates an operation screen that accepts operation input for generating three-dimensional shape information, and the transmission / reception unit 51 transmits operation screen information related to the operation screen to the communication terminal 3 (step S25).

[0115] The processing unit 53 of the management server 5 executes the N-th generation process based on the operation input information received in step S11 (step S26).

[0116] The determination unit 55 determines whether the Nth generation process has been completed (step S27), and if not, returns to step S26 to continue the process.

[0117] When the Nth generation process is completed, the judgment unit 55 judges whether the Nth generation process is the last generation process (step S28), and if it is not the last process, the setting unit 57 adds 1 to the order N for executing various generation processes (step S29) and returns to step S23 to continue processing.

[0118] If the processing mode is not manual processing in step S24, the determination unit 55 determines whether the processing mode of the N-th generation processing determined in step S23 is automatic processing (step S30).

[0119] If the processing mode is not automatic processing, the setting unit 57 of the management server 5 generates an operation screen that accepts operation input for generating three-dimensional shape information, and the transmitting / receiving unit 51 transmits operation screen information related to the operation screen to the communication terminal 3 (step S31). Here, instead of step S30, the determination unit 55 may determine whether the processing mode of the Nth generation process determined in step S23 is mixed processing, and execute step S31 if the processing mode is mixed processing.

[0120] The processing unit 53 of the management server 5 executes the N-th generation process based on the operation input information received in step S11 and the processing program read from the storage process management DB 5002 in association with the process setting information (step S32).

[0121] The determination unit 55 determines whether the Nth generation process has been completed (step S33), and if not completed, returns to step S32 to continue the process, and if completed, proceeds to step S28.

[0122] If the processing mode is automatic processing in step S30, the processing unit 53 of the management server 5 executes the Nth generation process based on the processing program read out from the storage processing management DB 5002 in association with the processing setting information (step S34).

[0123] The determination unit 55 determines whether the Nth generation process has been completed (step S35), and if not completed, returns to step S34 to continue the process, and if completed, proceeds to step S28.

[0124] FIG. 8 is an explanatory diagram of the registration process according to this embodiment.

[0125] 8 to 11 show a display screen 1000 that is displayed on the display 308 of the terminal device 3 when the processing mode is mixed processing in step S10 of the sequence diagram shown in FIG.

[0126] 8 to 11, the display control unit 34 of the communication terminal 3 displays an operation screen 1400, an automatic process execution button 1310, and a confirmation button 1300 on the display screen 1000. The automatic process execution button 1310 is a setting screen that accepts a setting operation for setting a part of the process to be executed by a processing program, and is not displayed when the processing mode is manual processing.

[0127] In Figure 8, the display control unit 34 of the communication terminal 3 displays a first three-dimensional point cloud 1410 and a second three-dimensional point cloud 1420 as two-dimensional or three-dimensional images on the operation screen 1400, and the user can perform registration processing in various ways as shown below by operating a pointing device such as a mouse 312.

[0128] (Manual processing only) When the user moves the first three-dimensional point cloud 1410 and the second three-dimensional point cloud 1420 relatively by translation, rotation, etc. so that they overlap, aligns the feature point 1410a included in the first three-dimensional point cloud 1410 with the feature point 1420a included in the second three-dimensional point cloud 1420, and operates the confirm button 1300, the reception unit 32 receives the user's specified input operation on the displayed operation screen.

[0129] Then, the processing unit 53 of the management server 5 converts the first three-dimensional point cloud 1410 and the second three-dimensional point cloud 1420 into one integrated three-dimensional point cloud based on the operation input information contained in the input information received in step S11, and completes the registration process.

[0130] An experienced user selects locations that will be feature points and aligns them with floors and walls based on density, objects, surfaces, etc.

[0131] Alternatively, when acquiring the first three-dimensional point cloud 1410 and the second three-dimensional point cloud 1420, markers are placed at positions that can be feature points, and the markers in the first three-dimensional point cloud 1410 and the second three-dimensional point cloud 1420 are aligned so that they overlap. Furthermore, alignment can be performed by viewing from various directions using a 3D mouse or the like.

[0132] (Automatic processing followed by manual processing) When the user operates the automatic process execution button 1310 before performing an operation input on the operation screen 1400 , the accepting unit 32 accepts setting information for the automatic process execution button 1310 .

[0133] Then, the processing unit 53 of the management server 5 aligns the first three-dimensional point cloud 1410 and the second three-dimensional point cloud 1420 using a processing program related to registration processing read out from the storage processing management DB 5002 based on the setting information contained in the input information received in step S11.

[0134] Next, as shown in step S9, the setting unit 57 of the management server 5 generates an operation screen including the first three-dimensional point cloud 1410 and the second three-dimensional point cloud 1420 aligned by the processing program, and the transmission / reception unit 51 transmits operation screen information related to the operation screen to the communication terminal 3.

[0135] Next, when the user fine-tunes the relative positions of the first three-dimensional point cloud 1410 and the second three-dimensional point cloud 1420 that have been aligned by the processing program and operates the confirm button 1300, the processing unit 53 of the management server 5 converts the first three-dimensional point cloud 1410 and the second three-dimensional point cloud 1420, whose relative positions have been fine-tuned by the user, into a single integrated three-dimensional point cloud, and completes the registration process.

[0136] (Manual processing followed by automatic processing) When the user moves the first three-dimensional point cloud 1410 and the second three-dimensional point cloud 1420 relatively by translation, rotation, etc. so that they overlap, aligns the feature point 1410a included in the first three-dimensional point cloud 1410 with the feature point 1420a included in the second three-dimensional point cloud 1420, and operates the automatic processing execution button 1310, the reception unit 32 receives the user's specified input operation on the displayed operation screen and setting information for the automatic processing execution button 1310.

[0137] Then, the processing unit 53 of the management server 5 fine-tunes the relative positions of the first three-dimensional point cloud 1410 and the second three-dimensional point cloud 1420 aligned by the user, using a processing program related to the registration processing read out from the storage processing management DB 5002 based on the operation input information and setting information contained in the input information received in step S11.

[0138] Next, as shown in step S9, the setting unit 57 of the management server 5 generates an operation screen including the first three-dimensional point cloud 1410 and the second three-dimensional point cloud 1420, whose relative positions have been fine-tuned by the processing program, and the transmission / reception unit 51 transmits operation screen information related to the operation screen to the communication terminal 3.

[0139] Next, when the user checks the first three-dimensional point cloud 1410 and the second three-dimensional point cloud 1420 whose relative positions have been fine-tuned by the processing program and operates the confirm button 1300, the processing unit 53 of the management server 5 converts the first three-dimensional point cloud 1410 and the second three-dimensional point cloud 1420 whose relative positions have been fine-tuned by the processing program into a single integrated three-dimensional point cloud, and completes the registration process.

[0140] FIG. 9 is an explanatory diagram of the noise removal process according to this embodiment.

[0141] The display control unit 34 of the communication terminal 3 displays the three-dimensional point cloud 1430 as a two-dimensional or three-dimensional image on the operation screen 1400, and the user can perform noise removal processing in various ways as shown below by operating a pointing device such as a mouse 312.

[0142] (Manual processing only) When the user selects an unnecessary point group 1440 from the three-dimensional point group 1430 and operates the confirm button 1300, the accepting unit 32 accepts a predetermined input operation by the user on the displayed operation screen.

[0143] Then, the processing unit 53 of the management server 5 removes the unnecessary point group 1440 from the three-dimensional point group 1430 based on the operation input information included in the input information received in step S11, and ends the noise removal process.

[0144] (Automatic processing followed by manual processing) When the user operates the automatic process execution button 1310 before performing an operation input on the operation screen 1400 , the accepting unit 32 accepts setting information for the automatic process execution button 1310 .

[0145] Then, the processing unit 53 of the management server 5 selects unnecessary point clouds 1440 from the three-dimensional point cloud 1430 using a processing program related to noise removal processing read out from the storage processing management DB 5002 based on the setting information contained in the input information received in step S11.

[0146] Next, as shown in step S9, the setting unit 57 of the management server 5 generates an operation screen including the three-dimensional point cloud 1430 and the unnecessary point cloud 1440 selected by the processing program, and the transmission / reception unit 51 transmits operation screen information related to the operation screen to the communication terminal 3.

[0147] Next, when the user fine-tunes the region of the unnecessary point cloud 1440 selected by the processing program and operates the confirm button 1300, the processing unit 53 of the management server 5 removes the unnecessary point cloud 1440, the region of which has been fine-tuned by the user, from the three-dimensional point cloud 1430, and ends the noise removal process. Specifically, the user widens or narrows the region of the unnecessary point cloud 1440 selected by the processing unit 53, removes specific points from the region of the unnecessary point cloud 1440 selected by the processing unit 53, or adds specific points to the region of the unnecessary point cloud 1440 selected by the processing unit 53.

[0148] (Manual processing followed by automatic processing) When the user selects an unnecessary point group 1440 from the three-dimensional point group 1430 and operates the automatic processing execution button 1310, the reception unit 32 receives the user's specified input operation on the displayed operation screen and setting information for the automatic processing execution button 1310.

[0149] Then, based on the operation input information and setting information included in the input information received in step S11, the processing unit 53 of the management server 5 fine-tunes the area of ​​the unnecessary point cloud 1440 selected by the user, using a processing program related to noise removal processing read from the storage processing management DB 5002. Specifically, the processing unit 53 widens or narrows the area of ​​the unnecessary point cloud 1440 selected by the user, removes specific points from the area of ​​the unnecessary point cloud 1440 selected by the user, or adds specific points to the area of ​​the unnecessary point cloud 1440 selected by the user.

[0150] Next, as shown in step S9, the setting unit 57 of the management server 5 generates an operation screen including the three-dimensional point cloud 1430 and the unnecessary point cloud 1440 whose area has been fine-tuned by the processing program, and the transmitting / receiving unit 51 transmits operation screen information related to the operation screen to the communication terminal 3.

[0151] Next, when the user checks the unnecessary point cloud 1440 whose area has been fine-tuned by the processing program and operates the confirm button 1300, the processing unit 53 of the management server 5 removes the unnecessary point cloud 1440 whose area has been fine-tuned by the processing program from the three-dimensional point cloud 1430, and terminates the noise removal process.

[0152] FIG. 10 is an explanatory diagram of the segmentation process according to this embodiment.

[0153] The display control unit 34 of the communication terminal 3 displays the three-dimensional point cloud 1430 as a two-dimensional or three-dimensional image on the operation screen 1400, and the user can perform segmentation processing in a variety of ways, as shown below, by operating a pointing device such as a mouse 312.

[0154] (Manual processing only) When the user forms a labeled point group 1450 (black circle) by labeling a specific point group in the three-dimensional point group 1430 and operates the confirm button 1300, the reception unit 32 receives a predetermined input operation by the user on the displayed operation screen.

[0155] Then, the processing unit 53 of the management server 5 determines the labeled point group 1450 based on the operation input information included in the input information received in step S11, and ends the segmentation process.

[0156] (Automatic processing followed by manual processing) When the user operates the automatic process execution button 1310 before performing an operation input on the operation screen 1400 , the accepting unit 32 accepts setting information for the automatic process execution button 1310 .

[0157] Then, the processing unit 53 of the management server 5 forms a labeled point cloud 1450 (black circles) by labeling specific point clouds in the three-dimensional point cloud 1430 using a processing program related to segmentation processing read from the storage processing management DB 5002 based on the setting information contained in the input information received in step S11.

[0158] Next, as shown in step S9, the setting unit 57 of the management server 5 generates an operation screen including the three-dimensional point cloud 1430 and the labeled point cloud 1450 formed by the processing program, and the transmission / reception unit 51 transmits operation screen information related to the operation screen to the communication terminal 3.

[0159] Next, when the user fine-tunes the labeled point cloud 1450 formed by the processing program and operates the confirm button 1300, the processing unit 53 of the management server 5 confirms the labeled point cloud 1450 fine-tuned by the user and ends the segmentation process.

[0160] (Manual processing followed by automatic processing) When a user forms a labeled point group 1450 (black circle) by labeling a specific point group in the three-dimensional point group 1430 and operates the automatic processing execution button 1310, the reception unit 32 receives the user's specified input operation on the displayed operation screen and setting information for the automatic processing execution button 1310.

[0161] Then, the processing unit 53 of the management server 5 fine-tunes the labeled point cloud 1450 formed by the user using a processing program related to segmentation processing read out from the storage processing management DB 5002 based on the operation input information and setting information contained in the input information received in step S11.

[0162] Next, as shown in step S9, the setting unit 57 of the management server 5 generates an operation screen including the three-dimensional point cloud 1430 and the labeled point cloud 1450 fine-tuned by the processing program, and the transmission / reception unit 51 transmits operation screen information related to the operation screen to the communication terminal 3.

[0163] Next, when the user checks the labeled point cloud 1450 that has been fine-tuned by the processing program and operates the confirm button 1300, the processing unit 53 of the management server 5 confirms the labeled point cloud 1450 by the processing program and terminates the segmentation process.

[0164] FIG. 11 is an explanatory diagram of the modeling process according to this embodiment.

[0165] The modeling process is a process of matching a specific area in a three-dimensional point cloud with a model shape and replacing the specific area with the model shape.

[0166] The display control unit 34 of the communication terminal 3 displays, on the operation screen 1400, model information 1460 including a three-dimensional point cloud 1430 and multiple model shapes 1461, 1462, and 1463, each of which has a different shape, as a two-dimensional or three-dimensional image, and the user can perform modeling processing in a variety of ways, as shown below, by operating a pointing device such as a mouse 312.

[0167] (Manual processing only) When the user compares the labeled point cloud 1450 in the three-dimensional point cloud 1430 with multiple model shapes 1461, 1462, and 1463, selects a model shape to replace a specific area, and operates the confirm button 1300, the reception unit 32 receives a predetermined input operation by the user on the displayed operation screen. If the user does not find an optimal model shape, the user may adjust the dimensions and shape of the selected model shape. Here, as an example, the optimal model shape is the model shape with the smallest total distance calculated by calculating the distance between the model shape and each point in the point cloud.

[0168] Then, the processing unit 53 of the management server 5 replaces the labeled point group 1450 with the model shape based on the operation input information included in the input information received in step S11, and ends the modeling process.

[0169] As a result, even if there is a missing portion of the point cloud in a specific region of the three-dimensional point cloud 1430, for example, the user can execute the modeling process while taking into account the missing portion of the point cloud.

[0170] Specifically, if there is a missing point cloud in a specific area and it is divided into two areas, the processing program for the modeling process will replace it with two model shapes, but the user can consider that there is a point cloud in the missing part as well and replace it with one model shape that includes the two areas.

[0171] (Automatic processing followed by manual processing) When the user operates the automatic process execution button 1310 before performing an operation input on the operation screen 1400 , the accepting unit 32 accepts setting information for the automatic process execution button 1310 .

[0172] Then, based on the setting information contained in the input information received in step S11, the processing unit 53 of the management server 5 uses a processing program related to the modeling process read from the storage processing management DB 5002 to match a specific area in the three-dimensional point cloud 1430 with multiple model shapes 1461, 1462, 1463 and select a specific model shape.

[0173] Next, as shown in step S9, the setting unit 57 of the management server 5 generates an operation screen including the three-dimensional point cloud 1430 and the model shape selected by the processing program, and the transmission / reception unit 51 transmits operation screen information related to the operation screen to the communication terminal 3.

[0174] Next, when the user fine-tunes the dimensions or shape of the selected model shape selected by the processing program, or changes the model shape selected by the processing program to another model shape, and operates the confirm button 1300, the processing unit 53 of the management server 5 replaces the labeled point cloud 1450 with the model shape finally adjusted by the user, and ends the modeling process.

[0175] (Manual processing followed by automatic processing) When a user compares the labeled point group 1450 in the three-dimensional point group 1430 with multiple model shapes 1461, 1462, and 1463, selects a specific model shape, and operates the automatic processing execution button 1310, the reception unit 32 receives the user's specified input operation on the displayed operation screen and setting information for the automatic processing execution button 1310.

[0176] Then, the processing unit 53 of the management server 5 fine-tunes the dimensions and shape of the model shape selected by the user using a processing program related to the modeling process read out from the storage processing management DB 5002 based on the operation input information and setting information contained in the input information received in step S11.

[0177] Next, as shown in step S9, the setting unit 57 of the management server 5 generates an operation screen including the three-dimensional point cloud 1430 and the model shape adjusted by the processing program, and the transmission / reception unit 51 transmits operation screen information related to the operation screen to the communication terminal 3.

[0178] Next, when the user checks the model shape adjusted by the processing program and operates the confirm button 1300, the processing unit 53 of the management server 5 replaces the labeled point cloud 1450 with the model shape changed by the processing program, and ends the modeling process.

[0179] FIG. 12 is a flowchart showing an example of a three-dimensional shape generation process according to a modified example of this embodiment.

[0180] FIG. 12 shows an example of processing corresponding to step S8 in FIG. 5 when the automatic processing batch setting box 1231, the manual processing batch setting box 1241, or the mixed processing batch setting box 1251 is set on the display screen 1000 shown in FIG. 6.

[0181] The processing unit 53 of the management server 5 acquires the three-dimensional point cloud data read from the memory / read unit 59 (step S41), and the setting unit 57 determines the order of the generation processes based on the execution order of the generation processes in the processing setting information received in step S6 (step S42).

[0182] The determination unit 55 determines whether the processing mode of the generation process in the processing setting information received in step S6 is manual processing (step S43).

[0183] If the processing mode is manual processing, the setting unit 57 of the management server 5 generates an operation screen that accepts operation input for generating three-dimensional shape information based on the order of the generation processing determined in step S42, and the transmission / reception unit 51 transmits operation screen information related to the operation screen to the communication terminal 3 (step S44).

[0184] The processing unit 53 of the management server 5 executes the generation process in the order determined in step S42 based on the operation input information received in step S11 (step S45).

[0185] The determination unit 55 determines whether the generation process has ended (step S46), and if the generation process has not ended, the process returns to step S44 and continues.

[0186] If the processing mode is not manual processing in step S43, the determination unit 55 determines whether the processing mode of the generation processing is automatic processing (step S47).

[0187] If the processing mode is not automatic processing, the setting unit 57 of the management server 5 generates an operation screen for accepting operation input for generating three-dimensional shape information based on the order of generation processing determined in step S42, and the transmitting / receiving unit 51 transmits operation screen information related to the operation screen to the communication terminal 3 (step S48). Here, instead of step S47, the determination unit 55 may determine whether the processing mode of the generation processing is mixed processing, and execute step S48 if the processing mode is mixed processing.

[0188] The processing unit 53 of the management server 5 executes the generation process in the order determined in step S42 based on the operation input information received in step S11 and the processing program read from the storage processing management DB 5002 in association with the processing setting information (step S49).

[0189] The determination unit 55 determines whether the generation process has ended (step S50), and if the generation process has not ended, the process returns to step S48 and continues.

[0190] If the processing mode is automatic processing in step S47, the processing unit 53 of the management server 5 executes the generation processing in the order determined in step S42 based on the processing program read from the storage processing management DB 5002 in association with the processing setting information (step S51).

[0191] The determination unit 55 determines whether the generation process has ended (step S52), and if the generation process has not ended, the process returns to step S51 and continues.

[0192] FIG. 13 is an explanatory diagram of a display screen according to a second modified example of this embodiment.

[0193] The display screen 1000 shown in FIG. 13 differs from the display screen 1000 shown in FIG. 6 in that the processing setting screen 1220 does not include an automatic processing batch setting box 1231, a manual processing batch setting box 1241, a mixed processing batch setting box 1251, a registration processing order setting box 1262, a noise removal processing order setting box 1263, a segmentation processing order setting box 1264, and a modeling processing order setting box 1265.

[0194] The processing setting screen 1220 according to the second modified example is a screen that accepts a processing setting operation for setting a processing mode for only one generation process among the registration process, the noise removal process, the segmentation process, and the modeling process.

[0195] FIG. 14 is an explanatory diagram of the modeling process according to the second modified example.

[0196] 14 differs from display screen 1000 shown in Fig. 11 in that it includes a next process button 1330. Next process button 1330 is a button for returning to display screen 1000 shown in Fig. 13.

[0197] When the user operates the next process button 1330, the accepting unit 32 of the communication terminal 3 accepts setting information for the next process button 1330.

[0198] The transmitting / receiving unit 31 transmits input information relating to the input operation received by the receiving unit 32 to the management server 5, and the transmitting / receiving unit 51 of the management server 5 receives the input information transmitted from the communication terminal 3.

[0199] The setting unit 57 of the management server 5 generates a setting screen based on the setting information for the next processing button 1330 included in the received input information, and the transmission / reception unit 51 transmits setting screen information related to the generated setting screen to the communication terminal 3.

[0200] The transmitting / receiving unit 31 of the communication terminal 3 receives the setting screen information transmitted from the management server 5, and the display control unit 34 causes the display 308 to display the setting screen shown in FIG.

[0201] FIG. 15 is a flowchart showing an example of a three-dimensional shape generation process according to the second modified example.

[0202] The processing unit 53 of the management server 5 acquires the three-dimensional point cloud data read from the memory / read unit 59 (step S61), and the setting unit 57 determines the generation process to be executed based on the processing setting information received in step S6 (step S62).

[0203] The determination unit 55 determines whether the processing mode of the generation process in the processing setting information received in step S6 is manual processing (step S63).

[0204] If the processing mode is manual processing, the setting unit 57 of the management server 5 generates an operation screen that accepts operation input for executing the generation processing determined in step S62, and the transmission / reception unit 51 transmits operation screen information related to the operation screen to the communication terminal 3 (step S64).

[0205] The processing unit 53 of the management server 5 executes the generation process determined in step S62 based on the operation input information received in step S11 (step S65).

[0206] The determination unit 55 determines whether setting information for the next process button 1330 has been received (step S66), and if not, determines whether the generation process has ended (step S67). If the generation process has not ended, the process returns to step S65 and continues.

[0207] In step S66, if setting information for the next processing button 1330 has been received, the setting unit 57 generates a setting screen as shown in FIG. 13, and the transmitting / receiving unit 51 transmits setting screen information related to the generated setting screen to the communication terminal 3 (step S68).

[0208] If the processing mode is not manual processing in step S63, the determination unit 55 determines whether the processing mode of the generation processing is automatic processing (step S69).

[0209] If the processing mode is not automatic processing, the setting unit 57 of the management server 5 generates an operation screen for accepting operation input for executing the generation processing determined in step S62, and the transmitting / receiving unit 51 transmits operation screen information related to the operation screen to the communication terminal 3 (step S70). Here, instead of step S69, the determination unit 55 may determine whether the processing mode of the generation processing is mixed processing, and execute step S70 if the processing mode is mixed processing.

[0210] The processing unit 53 of the management server 5 executes the generation process determined in step S62 based on the operation input information received in step S11 and the processing program read from the storage processing management DB 5002 in association with the processing setting information (step S49).

[0211] The determination unit 55 determines whether setting information for the next process button 1330 has been received (step S72), and if not, determines whether the generation process has ended (step S73). If the generation process has not ended, the process returns to step S70 and continues.

[0212] In step S72, if setting information for the next process button 1330 has been received, the process proceeds to step S68.

[0213] If the processing mode is automatic processing in step S69, the processing unit 53 of the management server 5 executes the generation processing determined in step S62 based on the processing program read from the storage processing management DB 5002 in association with the processing setting information (step S74).

[0214] The determination unit 55 determines whether setting information for the next process button 1330 has been received (step S75), and if not, determines whether the generation process has ended (step S76). If the generation process has not ended, the process returns to step S74 and continues.

[0215] In step S75, if setting information for the next processing button 1330 has been received, the process proceeds to step S68.

[0216] ●Summary● As described above, the management server 5, which is an example of a three-dimensional shape generation device according to one embodiment of the present invention, includes the processing unit 53, which is an example of a three-dimensional information generation means that generates three-dimensional shape information based on point cloud information indicating a three-dimensional point cloud, and the processing unit 53 executes the first process or the second process based on an instruction operation received by the confirm button 1300, which is an example of an instruction receiving screen that receives an instruction operation to instruct either a first process that executes all or part of the generation process for generating three-dimensional shape information based on an operation input, or a second process that executes the generation process based on a pre-stored storage process without based on an operation input.

[0217] This allows an expert to generate desired three-dimensional shape information by executing the first process, and an unskilled person to easily generate three-dimensional shape information by executing the second process. In other words, three-dimensional shape information can be generated according to the user's level of skill and knowledge.

[0218] The management server 5 further includes a transmitting / receiving unit 51, which is an example of a transmitting means, that transmits instruction acceptance screen information indicating the confirm button 1300 to the terminal device 3, which is an example of a communication terminal capable of communicating with the management server 5.

[0219] This allows the management server 5 to generate three-dimensional shape information based on an instruction operation on the terminal device 3 side to instruct the first process or the second process.

[0220] The transmitting / receiving unit 51 further transmits to the terminal device 3 operation screen information indicating an operation screen 1400 for accepting operation input.

[0221] This allows the management server 5 to generate three-dimensional shape information based on an operation input on the terminal device 3 side.

[0222] The processing unit 53 executes the first process or the second process based on the setting operation and instruction operation accepted on the setting screen 1200 that accepts the setting operation for setting the first process or the second process.

[0223] This allows an expert to generate desired three-dimensional shape information by setting the first process, and an unskilled person to easily generate three-dimensional shape information by setting the second process. In other words, three-dimensional shape information can be generated according to the user's level of skill and knowledge.

[0224] The transmitting / receiving unit 51 further transmits operation screen information indicating the setting screen 1200 to the terminal device 3.

[0225] This allows the management server 5 to generate three-dimensional shape information based on a setting operation for setting the first process or the second process on the terminal device 3 side.

[0226] The management server 5 further includes a setting unit 57, which is an example of a setting means for setting the first process or the second process, based on the user information received by the reception unit 32, which is an example of a user information reception means for receiving user information, and the processing unit 53 executes the first process or the second process based on the setting information and instruction operation of the first process or the second process set by the setting unit 57.

[0227] This allows three-dimensional shape information to be generated by the first process or the second process set based on the user information.

[0228] The first processing includes executing a generating processing based on an operation input without based on a storage processing, or executing a generating processing based on an operation input and a storage processing.

[0229] As a result, experts in all generation processes can generate desired three-dimensional shape information by executing the generation process based on operation input without relying on storage processing, and experts in some generation processes can use operation input and storage processing appropriately for generation processes in which they are skilled and generation processes in which they are not skilled. In other words, three-dimensional shape information can be generated according to the level of proficiency of the experts in terms of skills and knowledge.

[0230] The processing unit 53 executes a plurality of generation processes such as registration processing, noise removal processing, segmentation processing, and modeling processing to generate three-dimensional shape information, and executes a first process or a second process for at least one of the generation processes among the registration processing, noise removal processing, segmentation processing, modeling processing, etc. based on an instruction operation that instructs a first process or a second process for at least one of the plurality of generation processes.

[0231] This allows each of the plurality of generation processes to be executed in accordance with the user's level of skill and knowledge for each of the plurality of generation processes.

[0232] The processing unit 53 executes a plurality of generation processes such as registration processing, noise removal processing, segmentation processing, and modeling processing to generate three-dimensional shape information, and executes the plurality of generation processes in a set order based on the setting operations and instruction operations accepted in the order setting boxes 1262, 1263, 1264, and 1265 of the setting screen 1200, which accepts setting operations to set the order in which the plurality of generation processes are to be executed.

[0233] This allows the expert to execute multiple generation processes in any desired order.

[0234] The processing unit 53 executes a plurality of generation processes in the order stored in advance in the setting information management DB 5001, regardless of the setting operation.

[0235] This allows a non-expert to easily execute multiple generation processes without having to set an order.

[0236] The management server 5 further includes a storage / readout unit 59, which is an example of an update unit that updates the storage process based on the history information when the processing unit 53 executed the first process.

[0237] As a result, the storage process is updated based on the operation input history of the expert, and the accuracy of the three-dimensional shape generated the next time the non-expert executes the second process is improved.

[0238] The management server 5 further includes a storage / readout unit 59, which is an example of a storage control means for storing the three-dimensional shape information in a storage means such as the three-dimensional shape management DB 5004, the recording medium 506, or the CD-RW 513.

[0239] The transmitting / receiving unit 51 transmits the three-dimensional shape information to the terminal device 3. This allows the terminal device 3 to check the three-dimensional shape information.

[0240] A three-dimensional shape generation system 1 according to one embodiment of the present invention is a three-dimensional shape generation system including a management server 5 that generates three-dimensional shape information based on point cloud information indicating a three-dimensional point cloud, and a terminal device 3 that can communicate with the management server 5. The management server 5 includes a transmitter / receiver unit 51 that transmits to the terminal device 3 instruction acceptance screen information indicating a confirmation button 1300 that accepts an instruction operation to instruct the terminal device 3 to perform either a first process that executes all or part of the generation process to generate three-dimensional shape information based on an operation input, or a second process that executes the generation process based on a pre-stored memory process without based on an operation input, and a three-dimensional information generation means that executes the first process or the second process based on the instruction operation accepted by the confirmation button 1300. The terminal device 3 includes a display control unit 34 that displays the confirmation button 1300 on a display 308, and a reception unit 32 that is an example of an operation reception means that accepts an instruction operation on the confirmation button 1300.

[0241] A three-dimensional shape generation method according to one embodiment of the present invention is a three-dimensional shape generation method for generating three-dimensional shape information based on point cloud information indicating a three-dimensional point cloud, and includes a transmission step of transmitting to a terminal device 3 instruction acceptance screen information indicating a confirmation button 1300 for accepting an instruction operation to instruct a first process that executes all or part of a generation process for generating three-dimensional shape information based on an operation input, or a second process that executes a generation process based on a pre-stored memory process without based on an operation input, and a three-dimensional information generation step of executing the first process or the second process based on the instruction operation accepted by the confirmation button 1300.

[0242] A three-dimensional shape generation method according to another embodiment of the present invention is a three-dimensional shape generation method for generating three-dimensional shape information based on point cloud information indicating a three-dimensional point cloud, and includes a receiving step for receiving an instruction operation to instruct a first process that executes all or part of a generation process for generating three-dimensional shape information based on an operation input, or a second process that executes a generation process based on a pre-stored memory process without based on an operation input, and a three-dimensional information generation step that executes the first process or the second process based on the instruction operation.

[0243] A program according to an embodiment of the present invention causes a computer to execute the above three-dimensional shape generation method. [Explanation of symbols]

[0244] 1. 3D shape generation system 100 Communication Network 3. Terminal device (an example of a communication terminal) 31 Transmitting / receiving unit (an example of a transmitting means) 32 Reception unit (an example of a user information reception means or an operation reception means) 34 Display control unit (an example of a display control means) 39 Memory / read unit (an example of memory control means) 308 Display (Example of display unit) 3000 storage section 5. Management server (an example of a 3D shape generation device) 51 Transmitting / receiving unit (an example of a transmitting means) 53 Processing unit (an example of a three-dimensional information generating means) 55 Judgment Department 57 Setting section 59 Memory / read unit (an example of a memory control means, an update means) 5000 storage section 5001 Setting information management DB (an example of setting information management means) 5002 Memory processing management DB (an example of memory processing management means) 5003 Point cloud management DB (an example of point cloud management means) 5004 Three-dimensional shape management DB (an example of a three-dimensional shape management means) 1000 display screen 1100 User information display screen 1200 Settings Screen 1210 Point Cloud Settings Screen 1212, 1214 Point Cloud Settings Box 1220 Processing Settings Screen 1231 Automatic Processing Bulk Setting Box 1232 Automatic registration processing setting box 1233 Automatic noise reduction processing setting box 1234 Automatic segmentation processing setting box 1235 Automatic Modeling Processing Setting Box 1241 Manual Processing Bulk Setting Box 1242 Manual registration processing setting box 1243 Manual noise reduction processing setting box 1244 Manual Segmentation Processing Setting Box 1245 Manual Modeling Processing Setting Box 1251 Mixed Processing Batch Setting Box 1252 Mixed Registration Processing Setting Box 1253 Mixed noise reduction processing setting box 1254 Mixed Segmentation Processing Setting Box 1255 Mixed Modeling Processing Settings Box 1262 Registration Processing Order Setting Box 1263 Noise reduction processing order setting box 1264 Segmentation Processing Order Setting Box 1265 Modeling process order setting box 1300 Confirm button (example of instruction reception screen) 1310 Automatic processing execution button 1320 Next Process Button 1400 Operation screen 1410 First 3D point cloud 1420 Second 3D Point Cloud 1430 3D point cloud 1440 Unwanted points 1450 Labeled Point Clouds 1460 Model Information 1461, 1462, 1463 model shapes 1500 Generation shape setting screen (example of the second reception screen) 1510, 1520 Generated shape setting box 1600 Generated shape display screen 1610, 1620 Generated shape

Claims

1. a three-dimensional information generating means for generating three-dimensional shape information based on point cloud information indicating a three-dimensional point cloud; The three-dimensional information generation means is composed of a plurality of generation processes, including a modeling process that matches a specific area in a three-dimensional point cloud with a model shape based on operation input and replaces the specific area with the model shape, and the three-dimensional shape generation device executes the first process or the second process based on an instruction operation received on an instruction reception screen that receives an instruction operation to instruct a first process that executes all or part of the generation process to generate the three-dimensional shape information based on operation input, or a second process that executes the generation process based on a pre-stored storage process without being based on the operation input.

2. The three-dimensional shape generating device further includes a transmitting means for transmitting instruction acceptance screen information indicating the instruction acceptance screen to a communication terminal capable of communicating with the three-dimensional shape generating device. The three-dimensional shape generating device according to claim 1.

3. the transmitting means further transmits, to the communication terminal, operation screen information indicating an operation screen for accepting the operation input. The three-dimensional shape generating device according to claim 2.

4. A three-dimensional shape generation device as described in any one of claims 1 to 3, wherein the three-dimensional information generation means executes the first process or the second process based on the setting operation accepted on a setting screen that accepts setting operations to set the first process or the second process, and the instruction operation.

5. The three-dimensional shape generating device further includes a transmitting unit for transmitting setting screen information indicating the setting screen to a communication terminal capable of communicating with the three-dimensional shape generating device.

5. The three-dimensional shape generating device according to claim 4.

6. further comprising a setting unit that sets the first process or the second process based on the user information received by a user information receiving unit that receives user information; The three-dimensional information generating means A three-dimensional shape generating device according to any one of claims 1 to 5, which executes the first process or the second process based on the setting information of the first process or the second process set by the setting means and the instruction operation.

7. The first process includes: A three-dimensional shape generating device according to any one of claims 1 to 6, comprising: executing the generation process based on the operation input without based on the storage process; or executing the generation process based on the operation input and the storage process.

8. The three-dimensional information generating means Execute a plurality of generation processes for generating the three-dimensional shape information; A three-dimensional shape generation device according to any one of claims 1 to 7, wherein, for at least one of the plurality of generation processes, the first process or the second process is executed based on the instruction operation that instructs the first process or the second process.

9. The three-dimensional information generating means Execute a plurality of generation processes for generating the three-dimensional shape information; A three-dimensional shape generation device as described in any one of claims 1 to 8, which executes the multiple generation processes in the set order based on the setting operation accepted on a setting screen that accepts setting operations to set the order in which the multiple generation processes are to be executed, and the instruction operation.

10. 10. The three-dimensional shape generating device according to claim 9, wherein the three-dimensional information generating means executes the plurality of generating processes in a pre-stored order without being based on the setting operation.

11. 11. The three-dimensional shape generating device according to claim 1, further comprising an update means for updating the storage process based on history information when the three-dimensional information generating means executed the first process.

12. 12. The three-dimensional shape generating device according to claim 1, further comprising a storage control means for storing the three-dimensional shape information in a storage means.

13. The three-dimensional shape generating device further includes a transmitting means for transmitting the three-dimensional shape information to a communication terminal capable of communicating with the three-dimensional shape generating device. The three-dimensional shape generating device according to any one of claims 1 to 12.

14. A three-dimensional shape generation system including a three-dimensional shape generation device that generates three-dimensional shape information based on point cloud information indicating a three-dimensional point cloud, and a communication terminal that can communicate with the three-dimensional shape generation device, The three-dimensional shape generation device a three-dimensional information generating means configured with a plurality of generating processes, including a modeling process for matching a specific area in the three-dimensional point cloud with a model shape based on an operation input and replacing the specific area with the model shape; a transmitting means for transmitting to the communication terminal instruction acceptance screen information showing an instruction acceptance screen for accepting an instruction operation for instructing a first process for executing all or a part of a generation process for generating the three-dimensional shape information based on an operation input, or a second process for executing the generation process based on a pre-stored storage process without based on the operation input; Equipped with the three-dimensional information generating means executes the first process or the second process based on the instruction operation accepted on the instruction accepting screen; The communication terminal a display control means for displaying the instruction acceptance screen on a display unit; an operation accepting means for accepting the instruction operation on the instruction accepting screen; A three-dimensional shape generation system equipped with the above.

15. A three-dimensional shape generation method for generating three-dimensional shape information based on point cloud information indicating a three-dimensional point cloud, comprising: a three-dimensional information generation step including a plurality of generation processes, including a modeling process for matching a specific area in the three-dimensional point cloud with a model shape based on an operation input and replacing the specific area with the model shape; a transmitting step of transmitting, to the communication terminal, instruction acceptance screen information showing an instruction acceptance screen for accepting an instruction operation for instructing a first process for executing all or a part of a generation process for generating the three-dimensional shape information based on an operation input, or a second process for executing the generation process based on a storage process stored in advance without based on the operation input; Including, The three-dimensional shape generating method includes executing the first process or the second process based on the instruction operation accepted on the instruction acceptance screen, in the three-dimensional information generating step.

16. A three-dimensional shape generation method for generating three-dimensional shape information based on point cloud information indicating a three-dimensional point cloud, comprising: a three-dimensional information generation step including a plurality of generation processes, including a modeling process for matching a specific area in the three-dimensional point cloud with a model shape based on an operation input and replacing the specific area with the model shape; a receiving step of receiving an instruction operation to instruct a first process of executing all or a part of a generation process for generating the three-dimensional shape information based on an operation input, or a second process of executing the generation process based on a storage process stored in advance without based on the operation input; Including, The three-dimensional shape generating method includes executing the first process or the second process based on the instruction operation in the three-dimensional information generating step.

17. A program for causing a computer to execute the three-dimensional shape generation method according to claim 15 or 16.

Images