Management system and management program

Abstract

To provide a management system capable of improving convenience on a three-dimensional point group, and a management program.SOLUTION: A management system comprises: acquisition means for acquiring point group information indicative of a three-dimensional point group corresponding to the outer shape of an object provided in a three-dimensional space; identification means for identifying an object point serving as an object associating input information input by a user and corresponding to the object on the basis of the three-dimensional point group indicated by the point group information acquired by the acquisition means; and management means for performing processing for associating the object point identified by the identification means with input information. The point group information indicates at least a three-dimensional position of the three-dimensional group. The identification means identifies a gravity center corresponding to the gravity center position of the object on the basis of the three-dimensional point group indicated by the point group information acquired by the acquisition means.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a management system and a management program. [Background technology]

[0002] Conventionally, there has been known a technique for expressing the outer shape of an object placed in a three-dimensional space by a three-dimensional point cloud (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-166662 A Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, while it has been possible for users to grasp the external shape of an object based on a three-dimensional point cloud, there has been a demand for technology that improves the convenience of using the three-dimensional point cloud.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide a management system and a management program that can improve the convenience of three-dimensional point clouds. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the objective, the management system described in claim 1 comprises an acquisition means for acquiring point cloud information indicating a three-dimensional point cloud corresponding to the outline of an object set in a three-dimensional space, an identification means for identifying an object point corresponding to the object, which is to be associated with input information input by a user based on the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, and a management means for performing a process of associating the input information with the object point identified by the identification means.

[0007] The management system of claim 2 is the management system of claim 1, wherein the point cloud information indicates at least the three-dimensional position of the three-dimensional point cloud, and the identification means identifies a center of gravity point corresponding to the position of the center of gravity of the object based on the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, and identifies the identified center of gravity point as the object point.

[0008] The management system of claim 3 is the management system of claim 1, wherein the point cloud information indicates at least three-dimensional positions of the three-dimensional point cloud, and the identification means identifies a portion of three-dimensional points corresponding to a specified surface of the object from among a plurality of three-dimensional points included in the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, and identifies the identified portion of three-dimensional points as the target points.

[0009] The management system of claim 4 is the management system of claim 1, wherein the point cloud information indicates at least three-dimensional positions of the three-dimensional point cloud, and the identification means identifies a portion of three-dimensional points identified based on predetermined criteria as representative points from among a plurality of three-dimensional points included in the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, and identifies the identified representative point as the target point.

[0010] The management system of claim 5 is the management system of claim 1, wherein the point cloud information indicates at least the three-dimensional position and color of the three-dimensional point cloud, and the identification means identifies a portion of three-dimensional points whose color is within a specified color range from among a plurality of three-dimensional points included in the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, and identifies the identified portion of three-dimensional points as the target points.

[0011] The management system of claim 6 is the management system of claim 1, wherein the point cloud information indicates at least three-dimensional positions of the three-dimensional point cloud, and the identification means identifies a portion of three-dimensional points from among a plurality of three-dimensional points included in the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, the portion of three-dimensional points having three-dimensional positions included in a specified three-dimensional position range, and identifies the identified portion of three-dimensional points as the target points.

[0012] The management system of claim 7 is a management system of any one of claims 1 to 6, wherein the point cloud information indicates at least three-dimensional positions of the three-dimensional point cloud, and the management means performs a first process of identifying attributes of the object corresponding to the target point identified by the identification means based on a predetermined attribute identification method, and displaying guidance information that guides the input of the input information based on the identified attributes, and a second process of associating the input information entered by the user based on the guidance information displayed in the first process with the target point identified by the identification means.

[0013] The management program described in claim 8 causes a computer to function as: an acquisition means for acquiring point cloud information indicating a three-dimensional point cloud corresponding to the outline of an object set in a three-dimensional space; an identification means for identifying an object point corresponding to the object, which is to be associated with input information entered by a user based on the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means; and a management means for performing a process of associating the input information with the object point identified by the identification means. Effect of the Invention

[0014] According to the management system of claim 1 and the management program of claim 8, by performing a process of associating input information with the target point identified by the identification means, it is possible to associate information input by a user, for example, thereby improving convenience regarding three-dimensional point clouds.

[0015] According to the management system described in claim 2, by identifying a center of gravity point corresponding to the position of the center of gravity of an object as an object point, it becomes possible, for example, to associate input information with an object point that intuitively indicates an object.

[0016] According to the management system described in claim 3, by identifying some three-dimensional points corresponding to a specific surface of the object as target points, it becomes possible, for example, to associate input information with the target points corresponding to the surface of the object.

[0017] According to the management system described in claim 4, by identifying a portion of three-dimensional points among a plurality of three-dimensional points included in a three-dimensional point cloud as a representative point, it becomes possible, for example, to associate input information with a target point corresponding to the representative point.

[0018] According to the management system described in claim 5, by identifying some three-dimensional points whose colors are included in a specified color range as target points, it becomes possible to associate input information with the target points identified based on color, for example.

[0019] According to the management system described in claim 6, by identifying some three-dimensional points whose three-dimensional positions are included in a specified three-dimensional position range as target points, it becomes possible, for example, to associate input information with the target points identified based on the three-dimensional positions.

[0020] According to the management system described in claim 7, by displaying guidance information that guides the input of input information based on attributes, and associating the input information entered based on the displayed guidance information with a target point, it becomes possible, for example, to allow a user to easily input appropriate input information. [Brief description of the drawings]

[0021] [Figure 1] 1 is a block diagram showing a functional concept of an information processing system according to an embodiment of the present invention; [Diagram 2] FIG. 1 is a diagram illustrating an example of a three-dimensional space. [Diagram 3] 13 is a display example of three-dimensional points. [Figure 4] FIG. 11 is a diagram illustrating an example of point cloud related information. [Diagram 5] FIG. 11 is a diagram illustrating an example of target point related information. [Figure 6] FIG. [Figure 7] 13 is a flowchart of an information recording process. [Figure 8] 13 is a display example of a point cloud related display screen. [Figure 9] 13 is a display example of a point cloud related display screen. [Figure 10] FIG. 13 is an explanatory diagram of a specification method. [Figure 11] 13 is a display example of a point cloud related display screen. [Figure 12] FIG. 13 is an explanatory diagram of a specification method. [Figure 13] 13 is a display example of a point cloud related display screen. [Figure 14] FIG. 13 is an explanatory diagram of a specification method. [Figure 15] FIG. 13 is an explanatory diagram of a specification method. [Figure 16] FIG. 11 is a diagram illustrating an example of guidance-related information. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of a management system and a management program according to the present invention will be described in detail with reference to the accompanying drawings.

[0023] [I] Basic Concept of the Embodiment First, a basic concept of the embodiment will be described. The embodiment relates to a management system and a management program. The management system according to the present invention is a system for performing information processing on a three-dimensional point group corresponding to the outline of an object provided in a three-dimensional space, for example, a system for performing processing for associating input information with an object point.

[0024] "Three-dimensional space" is a concept that indicates actual three-dimensional space, and includes, for example, the space inside and outside a building under construction or work, the space inside and outside a building being demolished, or any other space (for example, the space inside a building that has already been constructed, or an outdoor space).

[0025] The term "object" refers to an object that is provided in a three-dimensional space, and is, for example, a concept that refers to all or a part of the object. The term "object" refers to, for example, a floor, a wall, a ceiling, a window, an exterior, a chair, a table, a television, a car, and any other object.

[0026] A "three-dimensional point cloud" is information that indicates the external shape of an object, specifically, a set of points that indicate the external shape of an object. For example, a set of three-dimensional coordinate data obtained by scanning the external shape of an object using a 3D scanner or the like may be used as a "three-dimensional point cloud." Alternatively, as a variation, a set of three-dimensional coordinate data generated using a predetermined conversion program (a program that converts BIM data to three-dimensional coordinate data) based on 3D CAD data such as BIM (Building Information Modeling) data may be used as a "three-dimensional point cloud."

[0027] The term "target point" refers to a point identified based on the three-dimensional point cloud indicated by the point cloud information, and more specifically, refers to a point associated with input information. The term "target point" refers to, for example, any point included in the three-dimensional point cloud, or a point identified by a point included in the three-dimensional point cloud (a point not included in the three-dimensional point cloud), etc.

[0028] The term "input information" refers to information associated with a target point, such as information input by a user. The term "input information" refers to, for example, information related to construction in three-dimensional space, information indicating the characteristics of a target object, and any other information.

[0029] In the embodiment described below, a case will be described in which the "three-dimensional space" is, for example, a space within a building under construction.

[0030] [II] Specific Contents of the Embodiments Next, the specific contents of the embodiment will be described.

[0031] (composition) 1 is a block diagram showing a functional concept of an information processing system according to the present embodiment. The information processing system 100 includes, for example, a terminal device 1 and a server device 2, and these devices are capable of communicating with each other.

[0032] (Configuration - Terminal Device) The terminal device 1 in FIG. 1 is a device carried by a user, for example a tablet terminal, and includes a communication unit 11, a touch pad 12, a display 13, a recording unit 14, and a control unit 15, for example.

[0033] 1, for convenience of explanation, only one terminal device 1 is illustrated, but any number of terminal devices 1 may be provided in the information processing system 100. As a variation, for example, other devices such as a smartphone or a personal computer may be used as the terminal device 1.

[0034] The communication unit 11 is a communication means for communicating with an external device (e.g., the server device 2). The touch pad 12 is an operation means for accepting various operation inputs from a user. The display 13 is an image display means for displaying various images, and for example, the touch pad 12 and the display 13 are integrally formed as a touch panel. The recording unit 14 is a recording means for recording programs and various data. The control unit 15 is a control means for controlling the terminal device 1, and is configured using, for example, a CPU, a RAM, and internal memories such as a ROM.

[0035] (Configuration - Server device) The server device 2 is the management system described above, and includes, for example, a communication unit 21, a recording unit 22, and a control unit 23.

[0036] (Configuration - Server device - Communication unit) The communication unit 21 in FIG. 1 is a communication unit that communicates with an external device (for example, the terminal device 1).

[0037] (Configuration - Information processing system - Server device - Recording unit) 1 is a recording unit that records programs and various data necessary for the operation of the server device 2. The recording unit 22 includes, for example, a point cloud related information database 221 (hereinafter, the database is referred to as "DB") and a target point related information DB 222.

[0038] (Configuration - Server device - Recording section - Point cloud related information DB) Fig. 2 is a diagram illustrating a three-dimensional space, Fig. 3 is a display example of three-dimensional points, and Fig. 4 is a diagram illustrating point cloud related information. Note that the XYZ axes in Fig. 2 are assumed to be mutually orthogonal, the Z axis indicates the vertical direction, and, for example, the X and Y axes indicate the horizontal direction.

[0039] ===Three-dimensional space=== FIG. 2 illustrates a floor 901 and a wall member 902 in a structure under construction, and a case will be described in which the floor 901 and the wall member 902 are the target object.

[0040] Floor 901 is basically installed along the horizontal direction, but there is a slight unevenness in part 901A (a part of floor 901 on the lower side of the drawing in FIG. 2), and the surface of said part 901A is assumed to be slightly concave. Note that the surface of floor 901 in parts other than part 901A is assumed to be flat along the horizontal direction.

[0041] The wall member 902 is a member that constitutes a wall, and is provided, for example, along a vertical direction perpendicular to the floor 901.

[0042] ===Three-dimensional point=== In Fig. 3, a large number of three-dimensional points 800 corresponding to the contours of the floor 901 and wall members 902 in the three-dimensional space in Fig. 2 are illustrated. Note that in Fig. 3, only some of the three-dimensional points are labeled. Also, the number of three-dimensional points is expected to be greater than the number illustrated in Fig. 3 in practice, but for convenience of explanation, the explanation will focus on those shown in Fig. 3. Also, in Fig. 3, for convenience of explanation, lines corresponding to the contours of the floor 901 and wall members 902 in Fig. 2 are illustrated as dotted lines (the same applies to other figures).

[0043] ===Point Cloud Related Information=== 1 is a point cloud related information storage means for storing point cloud related information. "Point cloud related information" is information related to a three-dimensional point cloud (i.e., a set of three-dimensional points), and for example, information of each item shown in FIG. 4 is mutually associated.

[0044] ==3D ID== The three-dimensional point ID in FIG. 4 is three-dimensional point identification information (identification information is also referred to as "ID") for uniquely identifying each three-dimensional point (eg, "ID001" in FIG. 4).

[0045] ==3D point specific information== The three-dimensional point identification information in Fig. 4 is information for identifying each three-dimensional point, and includes, for example, three-dimensional coordinate information and color information. The three-dimensional point identification information in Fig. 4 may be interpreted as corresponding to "point cloud information."

[0046] The three-dimensional coordinate information in Fig. 4 is information indicating the three-dimensional position of each three-dimensional point, and is, for example, information capable of identifying the position of a point in three-dimensional space corresponding to each three-dimensional point. The three-dimensional coordinate information is arbitrary, but for example, three-dimensional coordinates based on an orthogonal coordinate system corresponding to the XYZ axes in Fig. 2 are used. The origin of this three-dimensional coordinate is arbitrary, but for example, an origin determined based on a reference object in the three-dimensional space (for example, a landmark explicitly placed in the three-dimensional space, or an object or part designated by an administrator in the three-dimensional space, etc.) may be set.

[0047] The three-dimensional coordinate information may be expressed in any manner, but for ease of explanation, in FIG. 4, it is shown as "P1 (X:1, Y:1, Z:1)" or the like.

[0048] 4 is information indicating the color determined for each three-dimensional point, for example, information indicating the color (display color) of a point in three-dimensional space corresponding to each three-dimensional point. The color information is arbitrary, but for example, information corresponding to the so-called RGB color model (a model that expresses colors based on the intensity of light of the three primary colors red, green, and blue) is used.

[0049] Although the color information may be expressed in any manner, for convenience of explanation, in FIG. 4, it is illustrated as "C1(R:1,G:1,B:1)" etc. Here, "R:1" indicates that the intensity of red light corresponds to "1", "G:1" indicates that the intensity of green light corresponds to "1", and "B:1" indicates that the intensity of blue light corresponds to "1". That is, for example, the intensity of each color is indicated by a consecutive numerical value such as "1" to "256", and it is possible to specify the color of each three-dimensional point by using a combination of the numerical range of each color.

[0050] ==Example== For example, the information at the top of Fig. 4 indicates that the three-dimensional position (three-dimensional coordinates) of the three-dimensional point indicated by "ID001" (one three-dimensional point among the many three-dimensional points 800 in Fig. 3) is "P1 (X:1, Y:1, Z:1)" and that the color corresponding to "C1 (R:1, G:1, B:1)" is set for the three-dimensional point. That is, for example, it indicates that the position of the point in three-dimensional space corresponding to the three-dimensional point indicated by "ID001" is the position corresponding to the position indicated by "P1 (X:1, Y:1, Z:1)" and that the color of the point in three-dimensional space corresponding to the three-dimensional point is the color corresponding to "C1 (R:1, G:1, B:1)".

[0051] ==Storage method== The method for storing such point cloud related information is arbitrary, but for example, it may be possible to use a known 3D scanner (a device capable of scanning both three-dimensional coordinate information and color information) to scan the floor 901 and wall components 902, etc. in the three-dimensional space of Figure 2 from all directions, and input the obtained three-dimensional coordinate information and color information into the server device 2, and store the information, or it may be stored by any other method.

[0052] Here, it is assumed that, for example, by scanning a floor 901 and wall components 902 in FIG. 2, the point cloud information in FIG. 4 is stored as point cloud related information indicating a large number of three-dimensional points 800 in FIG.

[0053] (Configuration - Server device - Recording unit - Target point related information DB) Fig. 5 is a diagram illustrating an example of target point related information. The target point related information DB222 in Fig. 1 is a target point related information storage means for storing the target point related information. The "target point related information" is information related to the target point described above, and for example, the information of each item shown in Fig. 5 is mutually associated.

[0054] ==Target Point ID== The target point ID in FIG. 5 is an target point ID for uniquely identifying each target point (eg, "ID901" in FIG. 5).

[0055] == Target point specific information == The target point identification information in Fig. 5 is information for identifying each target point, specifically, information indicating the three-dimensional position of each target point, for example, information capable of identifying the position of a point in three-dimensional space corresponding to each target point. For example, a case will be described in which three-dimensional coordinate information similar to the three-dimensional coordinate information in Fig. 4 is used as the target point identification information.

[0056] ==Input information== The input information in Figure 5 is input information associated with each target point (e.g., text information such as "Installed on January 20, 2021" in Figure 5).

[0057] ==Example== For example, the information at the top of Figure 5 indicates that the three-dimensional position (three-dimensional coordinates) of the target point indicated by "ID901" is "P100(...)", and that the input information associated with the target point is "Installed on January 20, 2021."

[0058] ==Storage method== Such target point related information may be stored by any method, but may be stored, for example, by executing an information recording process described below.

[0059] (Configuration - Server device - Control unit) The control unit 23 in FIG. 1 is a control unit that controls the server device 2, and conceptually includes, for example, an acquisition unit, a specification unit, and a management unit.

[0060] ===Acquisition method=== The "acquisition means" is a means for acquiring point cloud information indicating a three-dimensional point cloud corresponding to the outline of an object provided in a three-dimensional space.

[0061] ===Specific means=== Fig. 6 is an explanatory diagram of the identification means. Note that in the "identification method" column of Fig. 6, identification methods (first identification method to fifth identification method) of target points used in the identification means of the present embodiment are illustrated, and in the "identification target" column of Fig. 6, points identified as target points in the corresponding identification methods are illustrated.

[0062] The "identification means" is a means for identifying an object point that corresponds to an object and is a target point to which input information input by a user is to be associated based on the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means. A specific method for identifying the object point by the identification means is arbitrary, but in this embodiment, for example, the first identification method to the fifth identification method shown in FIG. 6 are used.

[0063] ==First identification method== When using the first identification method of Figure 6, the "identification means" identifies a center of gravity point corresponding to the position of the center of gravity of the target object based on the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, and identifies the identified center of gravity point as the target point.

[0064] ==Second identification method== When the second identification method of Figure 6 is used, the "identification means" identifies, for example, a portion of three-dimensional points corresponding to a specified surface of the target object from among a plurality of three-dimensional points included in the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, and identifies the identified portion of three-dimensional points as target points.

[0065] ==Third identification method== When the third identification method of Figure 6 is used, the "identification means" identifies, for example, a portion of three-dimensional points identified based on a predetermined criterion from among a plurality of three-dimensional points included in the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means as representative points, and identifies the identified representative points as target points.

[0066] ==Fourth specific method== When the fourth identification method of Figure 6 is used, the "identification means" identifies, for example, a portion of three-dimensional points whose colors are included in a specified color range from among a plurality of three-dimensional points included in the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, and identifies the identified portion of three-dimensional points as target points.

[0067] ==Fifth specific method== When the fifth identification method of Figure 6 is used, the "identification means" identifies, for example, a portion of three-dimensional points from among a plurality of three-dimensional points included in the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, the portion of three-dimensional points having three-dimensional positions included in a specified three-dimensional position range, and identifies the identified portion of three-dimensional points as target points.

[0068] ===Management means=== The "management means" performs a process of associating input information with the target point identified by the identification means. The processes performed by each part of the control unit 23 will be described later.

[0069] (process) Next, an information recording process executed by the information processing system 100 according to this embodiment will be described. Fig. 7 is a flowchart of the information recording process (in the following description of each process, steps are abbreviated as "S"). The information recording process is a process of identifying an object point and recording input information in association with the identified object point, and is, for example, a process executed by the server device 2, and is a process of storing the object point related information of Fig. 5.

[0070] The timing of execution of this information recording process is arbitrary, but for example, when a user performs a specified operation via the touchpad 12 of his / her terminal device 1, a signal indicating that the specified operation has been performed is transmitted to the server device 2, and the server device 2 starts up when it receives the signal. The following description will begin from the point where execution of the information recording process has begun.

[0071] In the following description, the reference number "800" for the three-dimensional point 800 will be omitted as appropriate.

[0072] ===SA1=== 7, the control unit 23 of the server device 2 acquires point cloud related information. Here, for example, the point cloud related information exemplified in FIG.

[0073] ===SA2=== In SA2 in Fig. 7, the control unit 23 of the server device 2 displays a point cloud related display screen on which information on a three-dimensional point cloud is displayed. Fig. 8 is a display example of the point cloud related display screen. In Fig. 8, the lines corresponding to the contours of the floor 901 and the wall member 902 in Fig. 2 are illustrated as dotted lines, but these dotted lines are described for convenience of explanation, and these dotted lines are not actually displayed on the point cloud related display screen (the same applies to other figures). In fact, the three-dimensional points 800 on the point cloud related display screen in Fig. 8 are displayed in large numbers at high density, so that the user can recognize the contours corresponding to the contours of the floor 901 and the wall member 902 in Fig. 2 by visually checking the point cloud related display screen.

[0074] The specific processing is arbitrary, but for example, the three-dimensional position (three-dimensional coordinates) and color of each three-dimensional point are identified based on the three-dimensional coordinate information and color information of the point cloud related information (Figure 4) acquired by SA1, and screen information for displaying each three-dimensional point displayed in the identified three-dimensional position and the identified color is generated and transmitted, thereby displaying a point cloud related display screen including a large number of three-dimensional points on the display 13 of the terminal device 1.

[0075] Here, for example, by generating and transmitting screen information for displaying a large number of three-dimensional points in Fig. 8, the large number of three-dimensional points in Fig. 8 are displayed. Note that, on the point cloud related display screen in Fig. 8, each of the large number of three-dimensional points is displayed in a color indicated by the color information in Fig. 4, and therefore, is displayed in a display mode corresponding to the floor 901 and wall member 902 in the three-dimensional space in Fig. 2.

[0076] When a user performs a predetermined operation (for example, an operation to change the viewpoint that is the reference for displaying a large number of three-dimensional points, such as swiping up, down, left, or right a predetermined distance from a predetermined position on the screen) via the touch pad 12 on the point cloud related display screen of FIG. 8, information indicating that the predetermined operation has been performed is transmitted from the terminal device 1 to the server device 2, and the server device 2 generates and transmits screen information that displays each three-dimensional point in a display state corresponding to the predetermined operation, thereby making it possible to change the display state of the large number of three-dimensional points displayed on the point cloud related display screen.

[0077] That is, by a predetermined operation by the user, it is possible to change the viewpoint and display three-dimensional points corresponding to the rear side (-X side) of the wall member 902 in FIG. 2 on the point cloud relation display screen.

[0078] ===SA3=== In SA3 of Fig. 7, the control unit 23 of the server device 2 identifies the target point based on the point cloud related information acquired in SA1. The specific process is arbitrary, but for example, when a user inputs, via his / her terminal device 1, identification method information indicating which identification method is to be specified among the first identification method to the fifth identification method of Fig. 6, to the server device 2, the server device 2 identifies the target point by the identification method indicated by the identification information input by the user. Identification of the target point by each identification method will be described below.

[0079] ==First identification method== Fig. 9 is a display example of the point cloud relation display screen, and Fig. 10 is an explanatory diagram of the specification method. In the first specification method, for example, the following first to third steps are performed.

[0080] =1st Step= When identification method information specifying the first identification method is input, the control unit 23 of the server device 2 sends request information to the terminal device 1 requesting the specification of a three-dimensional point to be targeted for the process of identifying the center of gravity, and displays it on the display 13.

[0081] Here, for example, the control unit 23 of the server device 2 transmits guidance message information such as “Which object's center of gravity do you want to identify?” as request information, and displays it on the terminal device 1.

[0082] =Second Step= On the other hand, after visually confirming the guidance message information, the user performs an operation to designate a 3D point corresponding to any object from among the 3D points (FIG. 8) displayed on the point cloud relation display screen displayed at SA2 in FIG. 7 as a 3D point to be subjected to processing for identifying the center of gravity. In this case, information indicating the designated 3D point is transmitted to the server device 2, so that the server device 2 can grasp which 3D point the user designated.

[0083] Here, for example, the user performs an operation to surround some three-dimensional points corresponding to the wall member 902 in Fig. 2 via the touch pad 12, as shown by the polygon of reference numeral 71 in Fig. 9, thereby designating the part of three-dimensional points surrounded by the polygon of reference numeral 71 as three-dimensional points to be targeted for processing to identify them as center of gravity points (center of gravity points corresponding to the position of reception of the wall member 902 in Fig. 2). Note that as a variation, for example, the three-dimensional points may be designated by tapping.

[0084] =Third Step= On the other hand, the control unit 23 of the server device 2 refers to the point cloud related information in Figure 4 and performs a calculation to calculate three-dimensional coordinates that are the average of the three-dimensional coordinates of each of the multiple three-dimensional points specified by the user, as a calculation to calculate the three-dimensional coordinates of the center of gravity point corresponding to the position of the center of gravity of the object specified by the user, identifies the point corresponding to the three-dimensional position indicated by the calculated three-dimensional coordinates (average three-dimensional coordinates) as the center of gravity point, and identifies the identified center of gravity point as the object point.

[0085] The specific content of the calculation to calculate the center of gravity is arbitrary, but for example, if there are n three-dimensional points and the three-dimensional coordinates of the n three-dimensional points are (X1, Y1, Z1), (X2, Y2, Z2), ... (Xn, Yn, Zn), the three-dimensional coordinate of the center of gravity corresponding to the average of the three-dimensional coordinates will be calculated as (average value of X1 to Xn, average value of Y1 to Yn, average value of Z1 to Zn).

[0086] Here, for example, a center of gravity 801 shown in Fig. 10 is specified as the target point. Basically, the center of gravity 801 is a different point that does not match any of the multiple three-dimensional points (multiple three-dimensional points exemplified in Fig. 8) shown in the point cloud related information in Fig. 4. Although it is rare, if the three-dimensional coordinates calculated by the above-mentioned calculation match, it may match any of the multiple three-dimensional points shown in the point cloud related information in Fig. 4.

[0087] ==Second identification method== Fig. 11 is a display example of the point cloud relation display screen, and Fig. 12 is an explanatory diagram of the specification method. In the second specification method, for example, the following first to third steps are performed.

[0088] =1st Step= When identification method information specifying the second identification method is input, the control unit 23 of the server device 2 identifies one or more planes (each of which includes three or more three-dimensional points that are not on the same line) defined by the three-dimensional points (Figure 8) displayed on the point cloud related display screen displayed at SA2 in Figure 7 based on the three-dimensional coordinate information (Figure 4) of the point cloud related information acquired at SA1 as planes corresponding to the surface of the object, and displays a point cloud related display screen including the planar image on the display 13 of the terminal device 1 by generating and transmitting screen information for displaying a planar image showing the identified planes.

[0089] In addition, as a method for identifying one or more planes defined by three-dimensional points based on the three-dimensional coordinate information of the point cloud related information acquired by SA1, any method including known methods can be applied. For example, a method for estimating a plane using an estimation algorithm such as RANSAC (Random Sample Consensus) and identifying the estimated plane may be used, or a method using an algorithm other than RANSAC may be used.

[0090] Here, for example, for the three-dimensional point in Fig. 8, based on the three-dimensional coordinate information (Fig. 4) of the point cloud related information acquired by SA1, one or more planes indicated by the three-dimensional point are specified as planes corresponding to the floor 901 in Fig. 2, planes corresponding to each surface of the wall member 902 (for example, the surface on the +X side, the surface on the +Z side, the surface on the -Y side, the surface on the -X side (not shown), the surface on the +Y side (not shown), etc. in Fig. 2), and information for displaying planar images 721-724 (Fig. 11) indicating each of the specified planes is generated and transmitted, thereby displaying the point cloud related display screen of Fig. 11 on the display 13 of the terminal device 1. Note that here, a configuration may be made to display guidance message information such as "Which surface would you like to select?".

[0091] In addition, although a planar image showing the -X side surface of the wall member 902 in FIG. 2 is not shown in the point cloud related display screen in FIG. 11, it is possible to check the planar image by performing an operation to change the viewpoint of the point cloud related display screen.

[0092] In addition, in the example of Figure 11, a case is illustrated in which only planar images 721-724 are displayed on the point cloud association display screen, but the planar images 721-724 may be configured to be displayed in addition to the numerous three-dimensional points in Figure 8.

[0093] =Second Step= On the other hand, the user performs an operation to specify a planar image corresponding to the three-dimensional point with which the user desires to associate information from among the planar images displayed on the point cloud association display screen displayed in the first step. In this case, information indicating the specified planar image is transmitted to the server device 2, so that the server device 2 can know which planar image the user specified.

[0094] Here, for example, the user performs an operation of tapping and designating the planar image 722 in FIG.

[0095] =Third Step= Meanwhile, the control unit 23 of the server device 2 identifies three-dimensional points included in the plane corresponding to the planar image designated by the user in the second step, and specifies the identified three-dimensional points as target points.

[0096] It should be noted that the "three-dimensional points corresponding to the planar image specified by the user in the second step" may be interpreted as corresponding to "a portion of three-dimensional points corresponding to a predetermined surface of the object".

[0097] Here, for example, when the planar image 722 in Fig. 11 is designated in the second step, in the third step, a three-dimensional point 802 in Fig. 12 is identified as a three-dimensional point included in the plane corresponding to the planar image 722, and the identified three-dimensional point 802 is identified as a target point. Note that although many three-dimensional points corresponding to the planar image 722 are illustrated in Fig. 12, only some of them are labeled with reference symbols.

[0098] ==Third identification method== Fig. 13 is a display example of the point cloud relation display screen, and Fig. 14 is an explanatory diagram of the specification method. In the third specification method, for example, the following first to third steps are performed.

[0099] =1st Step= When identification method information specifying the third identification method is input, the control unit 23 of the server device 2 identifies some characteristic three-dimensional points from among the three-dimensional points (Figure 8) displayed on the point cloud related display screen displayed at SA2 in Figure 7 based on the three-dimensional coordinate information (Figure 4) of the point cloud related information acquired at SA1, and displays the point cloud related display screen including the characteristic three-dimensional points on the display 13 of the terminal device 1 by generating and transmitting screen information for displaying the identified characteristic three-dimensional points.

[0100] In addition, as a method of identifying some characteristic three-dimensional points from among the three-dimensional points based on the three-dimensional coordinate information of the point cloud related information acquired by SA1, any method including a known method can be applied. For example, a method of identifying each mesh (polygon) using a technique of converting point cloud data into mesh data (polygon data) (i.e., a meshing technique) and identifying the vertices of the mesh as characteristic three-dimensional points may be used. In this case, a known technique such as a technique related to Delaunay triangulation may be used. In addition, after a user is configured to input numerical information corresponding to the number of vertices, each mesh may be identified so that the number of vertices corresponds to the numerical information input by the user, and the vertices of the mesh may be identified as characteristic three-dimensional points. Alternatively, the vertices may be identified using a method other than meshing (for example, a method of thinning out some three-dimensional points based on a predetermined rule and then identifying the 3D points that are not thinned out).

[0101] In addition, in this embodiment, a case will be described in which "identifying a vertex of a mesh as a characteristic three-dimensional point" is defined as the criterion (predetermined criterion) for identifying a characteristic three-dimensional point.

[0102] Here, for example, for the 3D points in Fig. 8, based on the 3D coordinate information (Fig. 4) of the point cloud related information acquired by SA1, 3D points 811-813, etc. in Fig. 13 are specified as some characteristic 3D points from among the 3D points, and screen information for displaying the specified characteristic 3D points 811-813, etc. is generated and transmitted, thereby displaying the point cloud related display screen of Fig. 13 on the display 13 of the terminal device 1. Note that here, a configuration may be made to also display guidance message information such as "Which point would you like to select?"

[0103] In addition, although many characteristic 3D points are illustrated in Fig. 13, only some of them are labeled with symbols. Also, the 3D points illustrated in Fig. 13 are for convenience of explanation, and in reality, 3D points corresponding to corners or uneven parts of the object may be easily identified.

[0104] =Second Step= On the other hand, the user performs an operation of designating a characteristic 3D point corresponding to the 3D point with which the user desires to associate information from among the characteristic 3D points displayed on the point cloud association display screen displayed in the first step. In this case, information indicating the designated 3D point is transmitted to the server device 2, so that the server device 2 can know which 3D point the user designated.

[0105] Here, for example, the user performs an operation of tapping and specifying a three-dimensional point 812 in FIG.

[0106] =Third Step= On the other hand, the control unit 23 of the server device 2 specifies the three-dimensional point designated by the user in the second step as a representative point, and specifies the specified representative point as a target point.

[0107] Here, for example, when a 3D point 812 in FIG. 13 is specified in the second step, the 3D point 812 is identified as a representative point in the third step, and the identified 3D point 812 is identified as a target point.

[0108] It should be noted that, if a plurality of three-dimensional points are designated in the second step, the plurality of three-dimensional points are specified as representative points and then specified as target points in the third step.

[0109] =Variations= As a variation, a part of the processing in the first step and the processing in the second step may be omitted, and the characteristic three-dimensional points identified in the first step may be identified as representative points, and the identified representative points may then be identified as target points.

[0110] ==Fourth specific method== 15 is an explanatory diagram of the identification method. In the fourth identification method, for example, the following first to third steps are performed.

[0111] =1st Step= When identification method information specifying the fourth identification method is input, the control unit 23 of the server device 2 sends request information to the terminal device 1 requesting specification of the color range of the three-dimensional point to which information association is desired, and displays it on the display 13.

[0112] The "color range of a three-dimensional point" is a concept that indicates a range of colors. In this embodiment, for example, it is a concept that indicates the range of numerical values ​​that indicate the intensity of the three primary colors shown in the "R:", "G:", and "B:" columns of the color information in Figure 4.

[0113] Here, for example, the control unit 23 of the server device 2 transmits guidance message information stating “Please specify a color range” as request information, and displays it on the terminal device 1.

[0114] =Second Step= On the other hand, the user visually recognizes the message information and then specifies a color range. In this case, information indicating the specified color range is transmitted to the server device 2, so that the server device 2 can know which color range the user specified.

[0115] The method for specifying the color range is arbitrary, but for example, we will explain a method in which the color range is specified by referring to the colors of the three-dimensional points displayed on the point cloud related display screen (Figure 8) displayed in SA2 in Figure 7.

[0116] In this case, when a user designates a three-dimensional point by tapping on the point cloud related display screen, numerical information of the three primary colors (e.g., "R:1, G:1, B:1") is displayed as information indicating the color of the designated three-dimensional point, and a predetermined color range input user interface (hereinafter the user interface is also referred to as "UI") is displayed on the display 13 to allow the user to designate a color range based on the numerical information.

[0117] As a method for specifying a color range using a specified color range input UI, for example, a method may be adopted in which the color range is specified by inputting relative values ​​for "R", "G", and "B" based on the numerical information of the three primary colors of a three-dimensional point specified by the user, or a method may be adopted in which the color range is specified by inputting absolute values.

[0118] The process of displaying a UI for inputting a predetermined color range is optional, and may be configured to, for example, identify color information of a three-dimensional point designated by a user by referring to the point cloud related information in Fig. 4 through communication between the terminal device 1 and the server device 2, and perform processing based on the identified color information. The processing entity is also optional, and may, for example, realize the above-mentioned processing by the server device 2 performing processing based on a request from the terminal device 1 and then transmitting the processing result to the terminal device 1, or may realize the above-mentioned processing by the terminal device 1 downloading the point cloud related information in Fig. 4 from the server device 2 and then processing it.

[0119] 2, the floor 901 is basically gray overall, but one portion 901A is slightly concave and has more moisture on its surface than the other portions, causing the color of the portion 901A to turn darker gray (a gray close to black, also called "dark gray") than the other portions. Note that the wall member 902 is a color completely different from gray.

[0120] When the user designates one three-dimensional point (for example, the three-dimensional point marked with the symbol "800" at the bottom of FIG. 8) corresponding to a three-dimensional position within part 901A of FIG. 2 on the point cloud related display screen of FIG. 8, a predetermined color range input UI for displaying numerical information on the three primary colors of "dark gray" and for designating a color range is displayed on the display 13. The user designates the color range by inputting information indicating a color range indicating a range of colors that includes the color "dark gray" and is relatively close to the color "dark gray" into the predetermined color range input UI via the touch pad 12.

[0121] =Third Step= On the other hand, the control unit 23 of the server device 2 refers to the point cloud related information in Figure 4, identifies three-dimensional points whose colors are defined to be included in the color range specified by the user (i.e., three-dimensional points in which color information indicating colors included in the color range specified by the user is stored), and identifies the identified three-dimensional points as target points.

[0122] Here, for example, three-dimensional point 820 in FIG. 15 is identified as a three-dimensional point whose color is defined to be within a color range that indicates colors relatively close to the color "dark gray," and three-dimensional point 820 is identified as the target point.

[0123] ==Fifth specific method== 15 is an explanatory diagram of the identification method. In the fifth identification method, for example, the following first to third steps are performed.

[0124] =1st Step= When identification method information specifying the fifth identification method is input, the control unit 23 of the server device 2 sends request information to the terminal device 1 requesting specification of the three-dimensional position range of the three-dimensional point to which information is desired to be associated, and displays it on the display 13.

[0125] The "three-dimensional position range of a three-dimensional point" is a concept indicating a range of three-dimensional positions, and in this embodiment, for example, is a concept indicating a range of numerical values ​​indicating the coordinate values ​​shown in the "X:", "Y:", and "Z:" columns of the three-dimensional coordinate information in Figure 4.

[0126] Here, for example, the control unit 23 of the server device 2 transmits guidance message information stating “Please specify a three-dimensional position range” as request information, and displays it on the terminal device 1.

[0127] =Second Step= On the other hand, the user visually recognizes the message information and then specifies a three-dimensional position range. In this case, information indicating the specified three-dimensional position range is transmitted to the server device 2, so that the server device 2 can know which three-dimensional position range the user specified.

[0128] The method for specifying the three-dimensional position range is arbitrary, but for example, we will explain a method in which the range is specified by referring to the three-dimensional positions of the three-dimensional points displayed on the point cloud related display screen (Figure 8) displayed in SA2 in Figure 7.

[0129] In this case, when the user designates a three-dimensional point by tapping on the point cloud related display screen, three-dimensional coordinates (e.g., "(X:1, Y:1, Z:1")) are displayed as information indicating the three-dimensional position of the designated three-dimensional point, and a predetermined three-dimensional position range input UI for designating a three-dimensional position range based on the numerical information is displayed on the display 13.

[0130] As a method for specifying a three-dimensional position range using a specified three-dimensional position range input UI, for example, a method may be adopted in which the three-dimensional coordinates of a three-dimensional point specified by the user are used as a reference and relative values ​​for "X", "Y", and "Z" are input, or a method may be adopted in which the absolute values ​​are input.

[0131] In addition, the process of displaying a predetermined UI for inputting a three-dimensional position range is optional, and for example, the process may be configured to identify three-dimensional coordinate information of a three-dimensional point designated by a user by referring to the point cloud related information in Fig. 4 through communication between the terminal device 1 and the server device 2, and to perform processing based on the identified three-dimensional coordinate information. In addition, the processing subject is optional, as in the case described in the above-mentioned fourth identification method.

[0132] Here, for example, floor 901 in FIG. 2 is provided along the horizontal direction as described above, but part 901A is slightly recessed and uneven. It is assumed that there are

[0133] When the user designates one three-dimensional point (for example, the three-dimensional point marked with the symbol "800" at the bottom of FIG. 8) corresponding to a three-dimensional position in the part 901A of FIG. 2 on the point cloud relation display screen of FIG. 8, a predetermined three-dimensional position range input UI for displaying the three-dimensional coordinates of the three-dimensional point and for designating a three-dimensional position range is displayed on the display 13. The user designates the three-dimensional position range by inputting information indicating a three-dimensional position range corresponding to the recessed portion of the part 901A of FIG. 2 into the predetermined three-dimensional position range input UI via the touch pad 12.

[0134] =Third Step= On the other hand, the control unit 23 of the server device 2 refers to the point cloud related information in Figure 4, identifies three-dimensional points whose three-dimensional positions are included in the three-dimensional position range specified by the user (i.e., three-dimensional points in which three-dimensional coordinate information indicating three-dimensional positions included in the three-dimensional position range specified by the user is stored), and identifies the identified three-dimensional points as target points.

[0135] Here, for example, a three-dimensional point 820 in FIG. 15 that is a three-dimensional point corresponding to the part 901A in FIG. 2 is identified, and the three-dimensional point 820 is identified as a target point.

[0136] In the example of Figure 15, a case is described in which a common three-dimensional point 820 is identified by both the fourth and fifth determination methods. However, depending on the content of the color range or three-dimensional position range specified by the user, each method will identify a different three-dimensional point as the target point.

[0137] ===SA4=== 7, the control unit 23 of the server device 2 associates the input information with the target point identified in SA3 and records it (that is, performs a process of associating the input information). Although the specific process is arbitrary, for example, the first to third steps are performed.

[0138] ==First Step== The control unit 23 of the server device 2 communicates with the terminal device 1, and displays on the display 13 of the terminal device 1 an input field for inputting information.

[0139] Here, for example, an input field is displayed together with guidance message information saying "Please input the information to be associated." In this case, the input field may be displayed on the point cloud relation display screen (FIG. 8) after highlighting the target point specified in SA3 (for example, the center of gravity point 801 in FIG. 10, the three-dimensional point 802 in FIG. 12, the three-dimensional point 812 in FIG. 14, or the three-dimensional point 820 in FIG. 15). Alternatively, the input field may be displayed on a separate screen that is displayed around the point cloud relation display screen.

[0140] ==Second Step== On the other hand, the user inputs information into the input field via the touch pad 12. Specifically, for example, information about the construction of the object corresponding to the highlighted object point is input. In this case, the control unit 15 of the terminal device 1 transmits the information input into the input field (input information) to the server device 2.

[0141] Here, for example, a case where the first specifying method is specified in SA3 and a case where the second specifying method is specified in SA3 will be described.

[0142] <If the first identification method is specified> 10 (target point) identified by specifying the first identification method in SA3, when associating information indicating that installation of the corresponding wall member 902 (FIG. 2) has been completed, "Installed on January 20, 2021" is entered in the input field. In this case, the control unit 15 of the terminal device 1 transmits information indicating "Installed on January 20, 2021" to the server device 2.

[0143] <If the second identification method is specified> 12 (target point) identified by specifying the second identification method in SA3, when associating information indicating that there is no dirt on the +X side surface of the corresponding wall member 902 (FIG. 2), "no dirt" is input in the input field. In this case, the control unit 15 of the terminal device 1 transmits information indicating "no dirt" to the server device 2.

[0144] ==Third Step== Meanwhile, the control unit 23 of the server device 2 receives the information transmitted from the terminal device 1 in the second step, and records the received information in association with the target point identified in SA3 in the target point related information in Fig. 5. Specifically, although this is arbitrary, for example, a unique target point ID is generated, and three-dimensional coordinate information indicating the three-dimensional position of the target point identified in SA3 is identified, and the information transmitted and received from the terminal device 1 in the second step (i.e., input information) is acquired, and the generated target point ID, the identified three-dimensional coordinate information, and the acquired information are recorded as the target point ID, target point identification information, and input information in Fig. 5.

[0145] Here, for example, in the aforementioned "<When the first identification method is specified>", "ID901" is generated as a unique target point ID, and "P100(...)" is identified as the three-dimensional coordinate information of the center of gravity 801, which is the target point identified in SA3. Furthermore, after obtaining "Installed on January 20, 2021" transmitted and received from the terminal device 1 in the second step, the information in the top row of Figure 5 is recorded.

[0146] Here, for example, in the above-mentioned "<When the second identification method is specified>", "ID902", "ID903", etc. (target point IDs for the number of three-dimensional points 802 in Fig. 12) are generated, the three-dimensional coordinates of each of the three-dimensional points 802 in Fig. 12 are identified, and "no dirt" transmitted and received from the terminal device 1 in the second step is acquired, and each piece of information is stored in association with each other. In this case, combinations of the target point ID, target point identification information, and input information in Fig. 5 are stored for the number of three-dimensional points 802 in Fig. 12, but for these pieces of input information, "no dirt" is commonly stored.

[0147] The processing when the third to fifth specifying methods are specified in SA3 is similar, and therefore a description thereof will be omitted.

[0148] == Variations == In addition, when it is assumed that the information recording process of FIG. 7 is executed multiple times and a common target point is specified multiple times, in the third step of SA4, referring to the target point specification information of FIG. 5, it is determined whether or not there is target point information in which the same information as the three-dimensional coordinate information of the target point specified in the most recent SA3 is stored. If there is no information, the above processing is performed since no information has been associated with the target point, and if there is information, the following processing may be performed. For example, in the target point related information of FIG. 5, the information transmitted from the terminal device 1 in the second step is stored as input information associated with the target point ID in which the three-dimensional coordinate information of the target point specified in the most recent SA3 is stored. In this case, new information may be accumulated while the already stored information remains, or the already stored information may be replaced with new information and updated. This concludes the explanation of the information recording process.

[0149] (Example of using associated input information) Furthermore, the input information recorded in association with the target point in the information recording process of FIG. 7 may be used in any way. For example, the input information may be displayed on the display 13 of the terminal device 1.

[0150] For example, when a user performs a specified operation to display input information via his / her terminal device 1, the control unit 23 of the server device 2 may be configured to highlight the target point indicated by the target point related information of FIG. 5 on a point cloud related display screen displaying the three-dimensional point cloud of FIG. 8 based on the point cloud related information of FIG. 4 and the target point related information of FIG. 5, and to display on the terminal device 1 a screen displaying the input information of FIG. 5 in the vicinity of the target point.

[0151] As mentioned above, it is assumed that common information is stored as the input information in FIG. 5. In this case, the common input information may be thinned out and displayed in part in consideration of the user's visibility.

[0152] Alternatively, for example, when a user performs an operation to select one or more three-dimensional points on the three-dimensional point cloud display screen of FIG. 8, the control unit 23 of the server device 2 may be configured to refer to the point cloud related information of FIG. 4 and the target point related information of FIG. 5 to determine whether or not there is input information associated with the three-dimensional points selected by the user, and if it is determined that there is input information, to display the associated input information on the terminal device 1.

[0153] In this case, it is assumed that the information corresponding to the center of gravity is not stored in the point cloud related information of Figure 4, but is stored only in the object point related information of Figure 5. Therefore, when displaying the three-dimensional point cloud display screen of Figure 8, by referring to the object point identification information of the object point related information of Figure 5, if there is information different from the three-dimensional coordinate information of the point cloud related information of Figure 4, the different information may be treated as information indicating the center of gravity, and the three-dimensional point corresponding to the information may also be displayed on the three-dimensional point cloud display screen.

[0154] (Effects of this embodiment) According to this embodiment, by performing a process of associating input information with the target points identified by the identification means, it is possible to associate information input by a user, for example, thereby making it possible to improve convenience regarding three-dimensional point clouds.

[0155] [III] Modifications to the embodiment Although the embodiments of the present invention have been described above, the specific configurations and means of the present invention can be modified and improved as desired within the scope of the technical ideas of each invention described in the claims.

[0156] (About attributes) 7 may be configured to perform processing taking into consideration the attributes of the object. An "attribute" is a type based on the nature of the object, and is a concept that includes, for example, a part, an object, and the like.

[0157] The term "part" refers to a concept that indicates an object of a type that corresponds to the framework of a building, and includes, for example, floors, walls, ceilings, etc. The term "object" refers to a concept that indicates an object of a type that corresponds to an object that is added to a building later, and includes, for example, windows, exterior structures, etc.

[0158] The definitions of "part" and "object" here may be changed as desired. For example, as described above, "part" may be interpreted as a concept indicating an object of a type corresponding to the building structure, and "object" may be interpreted as a concept indicating an object other than the object corresponding to "part". "Part" and "object" may be interpreted as concepts relating to objects related to buildings, or as concepts relating to any other object. "Attributes" are not limited to two types, but may be three or more types, and are not limited to concepts relating to buildings.

[0159] ===Specific implementation method=== The implementation method is arbitrary, but for example, the guidance-related information is assumed to be stored in the recording unit 22 of the server device 2.

[0160] ==Guide related information== Fig. 16 is a diagram showing an example of guidance related information. The "guidance related information" is information related to guidance information for guiding input of input information, and for example, information of each item shown in Fig. 16 is related to each other.

[0161] The attribute information in FIG. 16 is information indicating an attribute (in FIG. 16, the above-mentioned "part" indicating a part and "object" indicating an object).

[0162] The item information in Fig. 16 is guidance information, for example, information displayed to the user, and indicates the item of information to be input as input information. In Fig. 16, "crack" indicates an information item indicating whether or not a crack exists in a part, and "dirt" indicates an information item indicating whether or not a dirty part exists in a part. "Tilt (number)" indicates an information item indicating the angle of tilt, and "pass / fail" indicates an information item indicating whether or not it is pass or fail.

[0163] The candidate information in Fig. 16 is guidance information, for example, information displayed to the user, and is candidate information to be input as input information. In Fig. 16, "Yes, No" indicates "Yes" indicating the presence and "No" indicating the absence, and "Pass, Fail" indicates "Pass" indicating the pass and "Fail" indicating the fail. In addition, although not specifically shown for the sake of convenience, the "candidate information" for "Tilt (Numeric Value)" may be a numeric value or a numeric range of a candidate angle to be input.

[0164] For example, the information at the top of Fig. 16 indicates that for an object point corresponding to an object whose attribute is a part, the items of input information are "crack", "dirt", etc., and the candidates of the input information are "yes", "no", etc. Note that this guidance-related information may be stored by the administrator entering the information.

[0165] ==Processing== Regarding the processing, for example, the management means of the control unit 23 of the server device 2 is configured to perform a first process of identifying attributes of an object corresponding to a target point identified by the identification means based on a predetermined attribute identification method, and displaying guidance information that guides the input of input information based on the identified attributes, and a second process of associating input information entered by the user based on the guidance information displayed in the first process with the target point identified by the identification means.

[0166] =1st Step= Specifically, although this is arbitrary, for example, in the first step of SA4 in FIG. 7, the control unit 23 of the server device 2 identifies the attribute of the object corresponding to the target point identified in SA3 based on a predetermined attribute identification method.

[0167] Regarding the process of identifying the attribute, it is assumed that, for the three-dimensional coordinate information (FIG. 4) included in the point cloud related information acquired by SA1, as shown in FIG. 8, the three-dimensional position of each three-dimensional point can be identified, and for example, the three-dimensional position range corresponding to the part, such as a floor or a wall, is recorded in the recording unit 22. Then, if the target point identified by SA3 is a three-dimensional point within the recorded three-dimensional position range, a "part" is identified as the attribute, and if the target point identified by SA3 is not a three-dimensional point within the recorded three-dimensional position range (i.e., if the target point identified by SA3 is a three-dimensional point outside the recorded three-dimensional position range), an "object" is identified as the attribute.

[0168] Alternatively, for example, when the user performs various operations or inputs information in SA3 of FIG. 7 (when executing the second steps of the first to fifth identification methods), information indicating a part may be input, and the part indicated by the input information may be identified.

[0169] Alternatively, for example, information that defines the relative positional relationship of the objects of the "part" such as the floor being provided along the horizontal direction, the walls being provided along the vertical direction from the floor to the ceiling, and the ceiling being provided parallel to the floor and provided above the floor at a predetermined distance is stored in the recording unit 22, and based on the information, it is determined whether each three-dimensional point indicated by the point cloud related information acquired in SA1 corresponds to the object of the "part" or not. Then, if the object point identified in SA3 is a three-dimensional point corresponding to the object of the aforementioned "part", the "part" is identified as the attribute of the object corresponding to the object, and on the other hand, if the object point identified in SA3 is not a three-dimensional point corresponding to the object of the aforementioned "part", the "object" is identified as the attribute of the object corresponding to the object.

[0170] In addition, in the case where the center of gravity is specified as the target point by the first specifying method, it is assumed that the center of gravity is not shown in the point cloud related information of FIG. 4, and the above-mentioned processing is difficult to apply. In this case, the outer shape of the object of the "part" may be estimated based on the three-dimensional point corresponding to the object of the "part" among the three-dimensional points shown in the point cloud related information of FIG. 4, and the target point may be specified based on the estimated outer shape. Specifically, when the target point specified by SA3 is a three-dimensional point on the surface of the estimated outer shape or a three-dimensional point inside the estimated outer shape, the "part" may be specified as the attribute of the object corresponding to the target point, and on the other hand, when the target point specified by SA3 is a three-dimensional point outside the estimated outer shape, the "object" may be specified as the attribute of the object corresponding to the target point. This method may also be applied to the target points specified by the second to fifth specifying methods.

[0171] For example, when the three-dimensional point 812 in FIG. 14 is specified as the target point in SA3 in FIG. 7, "part" is specified as the attribute of the wall member 902, which is the target corresponding to the three-dimensional point 812.

[0172] =Second Step= Next, for example, in the second step of SA4 in FIG. 7, the control unit 23 of the server device 2 refers to the guidance-related information in FIG. 16 to identify item information and candidate information associated with the attribute information indicating the attribute identified in the first step, and displays the identified item information and candidate information on the terminal device 1, thereby guiding the input of input information.

[0173] Here, for example, if in the first step, a "part" is identified as an attribute of an object corresponding to three-dimensional point 812 in FIG. 14, in the second step, "item information" = "crack" and "stain" etc. associated with the "attribute information" = "part" in FIG. 16 and "candidate information" = "present, absent" etc. associated therewith are identified, and this identified information is transmitted to terminal device 1 and displayed on display 13.

[0174] As for the display method, for example, "crack" and an input field for information about the crack (e.g., a rectangular input field) may be displayed as the specified item information, and when the input field is tapped via the touch pad 12, either of the candidate information "yes" or "no" may be selected. The same applies to "dirt" and the like. As for "fall (number)" and the like in FIG. 16, when the input field is tapped, a tool for inputting a number may be displayed, and information may be input using the tool.

[0175] =Third Step= Next, for example, in the third step of SA4 in Figure 7, when a user inputs information into the input field displayed in the second step, the control unit 23 of the server device 2 receives the input information and the item information corresponding to the information, and then performs processing similar to that described in the embodiment, thereby storing the target point related information of Figure 5.

[0176] Here, for example, when a user inputs "no" for "crack", "crack" and "no" are transmitted to the server device 2 in a mutually associated state, and the server device 2 receives the "crack" and "no" and stores the "crack" and "no" as input information in Fig. 5. Note that the same applies to the processing of information related to "stain" and the like.

[0177] (About target point related information) In the above embodiment, the input information is stored in the target point related information in Fig. 5, but the present invention is not limited to this. For example, the point cloud related information in Fig. 4 may be configured to further include an "input information" item, and the input information may be stored in SA4 in Fig. 7 as information corresponding to the "input information" item.

[0178] In addition, when the center of gravity is identified as the target point using the first identification method, it is possible that the center of gravity is not shown in the point cloud related information of Figure 4 and the above-mentioned processing may be difficult to apply. In this case, the point cloud related information of Figure 4 may be configured to add new information regarding the center of gravity and then store the input information.

[0179] (About the center of gravity) Also, the first identification method in Fig. 6 described in SA3 in Fig. 7 may be configured to identify the center of gravity by another method. For example, focusing on the fact that the material type (wood, concrete, iron, etc.) of an object may have its own unique color in a three-dimensional space (Fig. 2), the material type may be identified based on the color, and the center of gravity may be identified by performing a calculation to calculate the average of the three-dimensional coordinates while taking into account the weighting corresponding to the unit volume weight of the identified material type.

[0180] In this case, for example, information relating to the color, the material type corresponding to that color, and the unit volume weight corresponding to that material type may be pre-recorded in the recording unit 22, and processing may be performed based on that information.

[0181] Also, for example, by noting that in a three-dimensional space (FIG. 2), each object may have its own unique color, the system may be configured to identify a three-dimensional point corresponding to the object based on color.

[0182] In this case, for example, the user inputs information indicating a color range corresponding to the target object without performing an operation of surrounding a 3D point as shown by the polygon of reference numeral 71 in Fig. 9. In this case, the server device 2 identifies a 3D point whose color is included in the color range indicated in the input information from among the 3D points in Fig. 8, and identifies a center of gravity point based on the identified 3D point.

[0183] (About the surface) In addition, in the second identification method of Fig. 6 described in SA3 of Fig. 7, a plane corresponding to the surface of the object may be identified by another method. For example, in consideration of the fact that the configuration (position, size, and shape) of the object may be determined, such as the ceiling height being 2.4 m, information indicating the threshold value of the three-dimensional position range indicating each object is recorded in the recording unit 22. Then, the server device 2 may be configured to identify a plane corresponding to the surface of the object at the three-dimensional point of Fig. 8 based on the information.

[0184] In addition, for example, when the concrete surface of the floor is exposed and the formwork remains under the floor above, the system may be configured to automatically classify the inside corners / outside corners of the floor and column walls (=corners of the column walls) by setting a threshold value for the color range that matches the gray points that are in contact with the three surfaces.

[0185] (About color range) In addition, in the fourth identification method of FIG. 6 described in SA3 of FIG. 7, focusing on the fact that it may be assumed that three-dimensional points whose three primary colors of the RGB color model are close to (R: 255, G: 0, B: 0) (red) are almost always points corresponding to color cones (registered trademark) at a construction site, color cones can be easily automatically searched for in point cloud data obtained by scanning a construction site by setting a color range. This makes it easy to link information (input information) to color cones in a construction site. Note that "linking" may be interpreted as a concept corresponding to "associating". Furthermore, by utilizing this, color cones can be installed in advance at points to which information (input information) is to be linked (associated), information can be easily linked to a three-dimensional point of a specific point. Note that, similar to the above, for example, by painting a specific color on temporary construction materials, the specific materials can be automatically searched for among three-dimensional points, and information can be easily linked to them.

[0186] Also, for example, focusing on the fact that 3D points with the three primary colors of the RGB color model near (R: 125, G: 125, B: 125) (gray) are assumed to be concrete surfaces in most cases at construction sites, it is possible to easily automatically search for only concrete surfaces from point cloud data scanned at construction sites by setting a color range. This can also be used to easily distinguish between concrete surfaces (building structures) and other information (input information) and link them.

[0187] (About three-dimensional position range) In addition, in the fifth identification method of Fig. 6 explained in SA3 of Fig. 7, it is possible to automatically classify low and high areas (floor unevenness) that exceed a threshold value based on absolute coordinates or a reference point on the floor surface, and to associate information (input information) with each of them.

[0188] For example, on a wall surface, the amount of deformation of the core material of a retaining wall or the accuracy of the steel frame of the exterior wall base can be automatically classified into areas that protrude or recede beyond a threshold value (wall unevenness) based on absolute coordinates or reference points, and information (input information) can be linked to each of them.

[0189] (About combinations) Furthermore, the techniques of the above-described embodiments and modifications may be combined in any manner. [Explanation of symbols]

[0190] 1 Terminal equipment 2. Server equipment 11 Communications Department 12 Touchpad 13. Display 14 Recording section 15 Control section 21 Communications Department 22 Recording Section 23 Control Unit 71 sign 100 Information Processing Systems 221 Point Cloud Related Information DB 222 Target Point Related Information DB 721 Flat portrait 722 Flat portrait 723 plane portrait 724 Plane portrait 800 3D points 801 Center of gravity 802 3D points 901 bed 901A Section 1 902 Wall material

Claims

1. An acquisition means for acquiring point cloud information indicating a three-dimensional point cloud corresponding to an outer shape of an object provided in a three-dimensional space; an identification means for identifying a target point to which input information input by a user is to be associated based on the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, the target point corresponding to the target object; a management means for performing a process of associating the input information with the target point identified by the identification means; A management system comprising:

2. the point cloud information indicates at least a three-dimensional position of the three-dimensional point cloud, The identification means identifies a center of gravity point corresponding to a position of a center of gravity of the object based on the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, and identifies the identified center of gravity point as the object point. The management system according to claim 1 .

3. the point cloud information indicates at least a three-dimensional position of the three-dimensional point cloud, The identification means is Identifying a portion of three-dimensional points corresponding to a predetermined surface of the object from among a plurality of three-dimensional points included in the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, and identifying the portion of three-dimensional points identified as the object points. The management system according to claim 1 .

4. the point cloud information indicates at least a three-dimensional position of the three-dimensional point cloud, The identification means identifies, as representative points, a portion of three-dimensional points identified based on a predetermined criterion from among a plurality of three-dimensional points included in the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, and identifies the identified representative points as the target points. The management system according to claim 1 .

5. the point cloud information indicates at least a three-dimensional position and a color of the three-dimensional point cloud; The identification means identifies a portion of three-dimensional points having a color included in a specified color range from among a plurality of three-dimensional points included in the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, and identifies the identified portion of three-dimensional points as the target points. The management system according to claim 1 .

6. the point cloud information indicates at least a three-dimensional position of the three-dimensional point cloud, The identification means identifies a portion of three-dimensional points having three-dimensional positions included in a specified three-dimensional position range from among a plurality of three-dimensional points included in the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, and identifies the identified portion of three-dimensional points as the target points. The management system according to claim 1 .

7. the point cloud information indicates at least a three-dimensional position of the three-dimensional point cloud, The management means includes: a first process of identifying an attribute of the object corresponding to the object point identified by the identification means based on a predetermined attribute identification method, and displaying guidance information for guiding input of the input information based on the identified attribute; a second process of associating the input information input by the user based on the guidance information displayed in the first process with the target point identified by the identification means; The management system according to any one of claims 1 to 6.

8. Computer, An acquisition means for acquiring point cloud information indicating a three-dimensional point cloud corresponding to an outer shape of an object provided in a three-dimensional space; an identification means for identifying a target point to which input information input by a user is to be associated based on the three-dimensional point cloud indicated by the point cloud information acquired by the acquisition means, the target point corresponding to the target object; a management means for performing a process of associating the input information with the target point identified by the identification means; A management program that acts as a

Images