Information processing device, measurement device, information processing method, and information processing program

Abstract

To enable easily specifying a position of a measurement target in a three-dimensional space represented by three-dimensional information.SOLUTION: An information processing device 10 displays a two-dimensional image of a three-dimensional space represented by three-dimensional information viewed from a virtual viewpoint of a virtual camera on a display unit 14A. The information processing device 10 comprises a derivation unit 20C and a display control unit 20D. The derivation unit 20C acquires first positional information related to a first position in the three-dimensional space by a user performing input to an input unit 14B, and acquires positional information related to a second position or calculates the same based on the first positional information by the user performing input to the input unit 14B. The display control unit 20D displays the two-dimensional image including the first position and the second position on the display unit 14A.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a measurement device, an information processing method, and an information processing program. [Background technology]

[0002] In recent years, laser scanners, LiDAR, and other such devices have been used to acquire three-dimensional information about real-world three-dimensional spaces. Three-dimensional information is used in industries such as construction, facility maintenance, and real estate for tasks such as confirming dimensions while visually checking locations on a screen. Prior art discloses a system that displays a two-dimensional image of a three-dimensional space represented by three-dimensional information viewed from a virtual viewpoint of a virtual camera, and measures distance and area by accepting input of multiple measurement points from a user via the display screen of the two-dimensional image. Summary of the Invention [Problem to be solved by the invention]

[0003] However, with conventional technology, if the position of the measurement target in the three-dimensional space represented by the three-dimensional information is in the blind spot of the virtual camera, the position of the measurement target is not displayed on the screen, making it difficult for the user to specify the desired position.

[0004] The present invention has been made in consideration of the above, and aims to provide an information processing device, a measurement device, an information processing method, and an information processing program that enable easy specification of the position of a measurement target within a three-dimensional space represented by three-dimensional information. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems and achieve the object, the present invention provides an information processing device that displays, on a display unit, a two-dimensional image of a three-dimensional space represented by three-dimensional information viewed from a virtual viewpoint of a virtual camera, and that acquires first position information relating to a first position in the three-dimensional space by a user's input to an input unit, and acquires position information relating to a second position. ,beforeThe apparatus includes a derivation unit that performs calculation based on the first position information, and a display control unit that displays the two-dimensional image including the first position and the second position on the display unit. [Effects of the Invention]

[0006] According to the present invention, it is possible to easily specify the position of a measurement target in a three-dimensional space represented by three-dimensional information. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram of an example of a measurement system according to this embodiment. [Figure 2] FIG. 2 is a schematic diagram showing an example of a two-dimensional image displayed on the display unit. [Figure 3] FIG. 3 is a schematic diagram showing an example of a scene in which the user specifies two measurement points. [Figure 4] FIG. 4 is a schematic diagram showing an example of a scene in which a two-dimensional image including a blind spot is displayed. [Figure 5] FIG. 5 is a schematic diagram of a three-dimensional space represented by three-dimensional information. [Figure 6] FIG. 6 is an explanatory diagram of an example of calculating position information of a candidate position. [Figure 7] FIG. 7 is an explanatory diagram of an example of generating a two-dimensional image in which the position of the virtual viewpoint is moved. [Figure 8] FIG. 8 is a schematic diagram showing an example of a scene in which both a first two-dimensional image and a second two-dimensional image are displayed on the display unit. [Figure 9] FIG. 9 is an explanatory diagram of an example of generating a two-dimensional image in which obstacles are not displayed. [Figure 10] FIG. 10 is a flowchart illustrating an example of the flow of information processing executed by the information processing apparatus according to the embodiment. [Figure 11] FIG. 11 is a schematic diagram showing an example of a measurement system according to a modified example. [Figure 12] FIG. 12 is a diagram showing the hardware configuration. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of an information processing device, a measurement device, an information processing method, and an information processing program will be described in detail with reference to the accompanying drawings.

[0009] FIG. 1 is a block diagram of an example of a measurement system 1 according to this embodiment.

[0010] The measurement system 1 includes an information processing device 10 and an imaging device 11. The information processing device 10 and the imaging device 11 are connected so as to be able to communicate with each other.

[0011] The image capturing device 11 is a device that obtains three-dimensional information of a three-dimensional space, which is a real space. In this embodiment, a device that obtains three-dimensional information by capturing images is used as the image capturing device 11. The image capturing device 11 is, for example, a ToF (Time-of-Flight) camera, a stereo camera, or the like. The ToF method is a method that irradiates an object to be measured with infrared light and determines the distance from the time it takes for the reflected light to return. A stereo camera is a camera that obtains depth information as distance information using the distance between two cameras and parallax information of the images obtained by each of the two cameras.

[0012] The image capturing device 11 may be any device that can obtain three-dimensional information, such as a system that uses LiDAR (Light Detection and Ranging), photogrammetry, etc. The image capturing device 11 may also be a smartphone equipped with a laser scanner, LiDAR, or the like.

[0013] Three-dimensional information is information that represents a three-dimensional space, which is a real space. The file format of the three-dimensional information is not limited. The three-dimensional information is represented, for example, by shape data that represents the three-dimensional shape of a three-dimensional object contained in the real space. The three-dimensional information is, for example, a point cloud format file that represents the three-dimensional space with discrete points, or a polygon mesh format file that represents the three-dimensional space with vertices and faces. A point cloud format file may be called a depth map, a distance image, or the like.

[0014] Examples of point cloud files include files with extensions such as ".xyz", ".e57", and ".ply". Examples of polygon mesh files include files with extensions such as ".obj", ".fbx", and ".stl".

[0015] The imaging device 11 outputs the obtained three-dimensional information to the information processing device 10.

[0016] The information processing device 10 is an information processing device that displays a two-dimensional image obtained by viewing a three-dimensional space represented by three-dimensional information from a virtual viewpoint of a virtual camera. The information processing device 10 is, for example, a smartphone, a tablet terminal, a personal computer, etc.

[0017] The information processing device 10 includes a communication unit 12, a UI (user interface) unit 14, a storage unit 16, and a control unit 20. The communication unit 12, the UI unit 14, the storage unit 16, and the control unit 20 are connected to each other so as to be able to communicate with each other.

[0018] The communication unit 12 communicates with an external information processing device via a network, etc. In this embodiment, the communication unit 12 communicates with the image capturing device 11.

[0019] The UI unit 14 includes a display unit 14A and an input unit 14B. The display unit 14A is a display that displays various types of information. The input unit 14B accepts operation instructions from a user. The input unit 14B is, for example, a keyboard, a pointing device, a mouse, etc. The display unit 14A and the input unit 14B may be an integrated touch panel. The memory unit 16 stores various types of information.

[0020] The control unit 20 executes information processing. A measurement application program is pre-installed in the control unit 20. The measurement application program displays a two-dimensional image of a three-dimensional space represented by three-dimensional information viewed from a virtual viewpoint of a virtual camera on the display unit 14A, and measures distance and area by receiving input of multiple measurement points from the user via the display surface of the two-dimensional image. Hereinafter, the measurement application program may be referred to as a measurement application.

[0021] In this embodiment, the control unit 20 includes an acquisition unit 20A, a reception unit 20B, a derivation unit 20C, a display control unit 20D, and a distance calculation unit 20E. The acquisition unit 20A, the reception unit 20B, the derivation unit 20C, the display control unit 20D, and the distance calculation unit 20E are realized, for example, by one or more processors. For example, each of the above units may be realized by causing a processor such as a CPU (Central Processing Unit) to execute a program, i.e., by software. Each of the above units may be realized by a processor such as a dedicated IC (Integrated Circuit), i.e., by hardware. Each of the above units may be realized by a combination of software and hardware. When multiple processors are used, each processor may realize one of the units, or two or more of the units.

[0022] The acquisition unit 20A acquires three-dimensional information. In this embodiment, the acquisition unit 20A acquires the three-dimensional information from the imaging device 11 via the communication unit 12. The acquisition unit 20A may acquire the three-dimensional information from another information processing device communicatively connected to the information processing device 10 via the communication unit 12 and a network, etc. The other information processing device may be, for example, a smartphone capable of acquiring three-dimensional information, but is not limited to a smartphone. The acquisition unit 20A may also acquire the three-dimensional information from the storage unit 16.

[0023] The three-dimensional information acquired by the acquisition unit 20A may be either monochrome information or color information including color information. The information processing device 10 uses color information as the three-dimensional information, converts the three-dimensional information into a two-dimensional image, and displays the two-dimensional image, thereby providing the user with a more visually understandable two-dimensional image.

[0024] The acquisition unit 20A outputs the acquired three-dimensional information to the derivation unit 20C. Note that the acquisition unit 20A may convert the three-dimensional information acquired from the imaging device 11 into an arbitrary file format and then output it to the derivation unit 20C. For example, the acquisition unit 20A may convert a file in point cloud format into a file in polygon mesh format using the method disclosed in Japanese Patent Application Laid-Open No. 2019-049533, and output the converted three-dimensional information to the derivation unit 20C. This file format conversion process may also be performed by the derivation unit 20C.

[0025] The reception unit 20B receives an instruction from the user to operate the input unit 14B.

[0026] Here, an example of the basic functions of the measurement application will be described.

[0027] 2 is a schematic diagram showing an example of a two-dimensional image 40A displayed on the display unit 14A. The two-dimensional image 40A is an example of the two-dimensional image 40 displayed on the display unit 14A. The two-dimensional image 40 is a two-dimensional image of the three-dimensional space S represented by the three-dimensional information 30 viewed from a virtual viewpoint of a virtual camera.

[0028] By operating the input unit 14B while viewing the displayed two-dimensional image 40, the user can freely move the virtual viewpoint in the three-dimensional space S and switch the display to a two-dimensional image 40 in which the three-dimensional space S is viewed from any virtual viewpoint.

[0029] FIG. 3 is a schematic diagram showing an example of a scene in which the user has designated two measurement points Q.

[0030] The measurement point Q is a point used to specify a measurement target that the user desires to measure, and is a point on the two-dimensional image 40 displayed on the display unit 14A.

[0031] For example, consider a situation in which a user operates the input unit 14B to specify a first measurement point Q1 and a second measurement point Q2 as measurement points Q at any positions on the two-dimensional image 40A displayed on the display unit 14A.

[0032] The first measurement point Q1 refers to the measurement point Q that is specified first among a plurality of measurement points Q specified by the user. The second measurement point Q2 refers to the measurement point Q that is specified next.

[0033] In this case, the measurement application identifies a first position P1 and a second position P2, which are positions in the three-dimensional space S represented by the three-dimensional information 30, corresponding to each of the two specified measurement points Q (first measurement point Q1, second measurement point Q2). The measurement application then calculates and displays the distance between the first position P1 and the second position P2. FIG. 3 shows, as an example, a state in which "2.0 [m]" is displayed as the distance between the first position P1 and the second position P2 in the three-dimensional space S, which correspond to each of the two specified measurement points Q (first measurement point Q1, second measurement point Q2).

[0034] As shown in FIG. 3, when a point in the three-dimensional space S that the user desires to measure is displayed in the two-dimensional image 40, the user can easily specify the measurement point Q.

[0035] However, there are cases where a point in the three-dimensional space S that the user wishes to measure is in the blind spot of the virtual camera. In this case, the point in the three-dimensional space S that the user wishes to measure is not displayed in the two-dimensional image 40 obtained by viewing the three-dimensional space S from the virtual viewpoint of the virtual camera.

[0036] FIG. 4 is a schematic diagram showing an example of a scene in which a two-dimensional image 40B including a blind spot BS is displayed. FIG. 4 shows an example in which the two-dimensional image 40B includes a blind spot BS caused by an obstacle 50. For example, suppose a user designates a first measurement point Q1 and then a second measurement point Q2. Also, suppose a position in three-dimensional space S corresponding to second measurement point Q2 is a position that is a blind spot BS when the three-dimensional space S is viewed from the virtual viewpoint used when creating the two-dimensional image 40B. In this case, as shown in FIG. 4, points within the blind spot BS included in the three-dimensional space S are not displayed in the two-dimensional image 40B.

[0037] In this case, in the prior art, the user had to move the virtual viewpoint in the three-dimensional space S by operating the input unit 14B while viewing the displayed two-dimensional image 40B, and perform viewpoint manipulation for viewpoint conversion so that the desired point was displayed within the two-dimensional image 40.

[0038] However, such viewpoint manipulation may increase the operation time or result in incorrect manipulation for a user who is unfamiliar with manipulating the three-dimensional information 30. Furthermore, the user may lose sight of the desired point during manipulation. Therefore, in the prior art, it may be difficult for the user to easily specify the position of the measurement target within the three-dimensional space S represented by the three-dimensional information 30.

[0039] Therefore, the control unit 20 of this embodiment displays a two-dimensional image 40 of the three-dimensional space S represented by the three-dimensional information 30 viewed from the virtual viewpoint of the virtual camera on the display unit 14A. Then, when a first measurement point Q1, which is one of the multiple measurement points Q, is designated by the user via the two-dimensional image 40, the control unit 20 controls the display unit 14A to always display a candidate point R, which is a candidate for a second measurement point Q2, which is another measurement point Q. Therefore, with the information processing device 10 of this embodiment, the user can complete measurement of the distance or area between multiple measurement points Q with minimal or no viewpoint manipulation of the virtual viewpoint.

[0040] The derivation unit 20C, the display control unit 20D, and the distance calculation unit 20E included in the control unit 20 will be described in detail.

[0041] FIG. 5 is a schematic diagram of a three-dimensional space S represented by three-dimensional information 30. As shown in FIG.

[0042] The derivation unit 20C acquires first position information relating to a first position P1 in the three-dimensional space S represented by the three-dimensional information 30 acquired by the acquisition unit 20A through input by the user to the input unit 14B, and acquires position information relating to a second position P2 from input by the user to the input unit 14B or calculates it based on the first position P1.

[0043] The first position P1 represents a position in the three-dimensional space S represented by the three-dimensional information 30, which corresponds to the first measurement point Q1 designated by the user via the displayed two-dimensional image 40.

[0044] The position information related to the first position P1 is information that represents the first position P1 in the three-dimensional information 30. The position information related to the first position P1 is represented by, for example, position coordinates in the three-dimensional space S represented by the three-dimensional information 30.

[0045] The second position P2 represents a position in the three-dimensional space S represented by the three-dimensional information 30, corresponding to the second measurement point Q2 designated by the user after the first measurement point Q1 via the displayed two-dimensional image 40, and the candidate point R for the second measurement point Q2. The candidate point R is a point that is a candidate for the second measurement point Q2. When the user inputs an instruction to confirm the candidate point R by operating the input unit 14B, the candidate point R is confirmed as the second position P2 (described in detail later).

[0046] The position information related to the second position P2 is information representing the second position P2 in the three-dimensional information 30 and a candidate position CP of the candidate point R. The candidate position CP represents a position in the three-dimensional space S represented by the three-dimensional information 30, which corresponds to the candidate point R displayed in the two-dimensional image 40. That is, when an instruction to confirm the candidate point R is input by the user operating the input unit 14B, the position information of the candidate position CP corresponding to the candidate point R is confirmed as the position information related to the second position P. The position information related to the second position P2 is represented by, for example, position coordinates in the three-dimensional space S represented by the three-dimensional information 30.

[0047] When the derivation unit 20C receives the three-dimensional information 30 from the acquisition unit 20A, it determines a first virtual viewpoint VP1, which is a virtual viewpoint serving as an initial position, and a viewing direction in which the three-dimensional space S is viewed from the first virtual viewpoint VP1. For example, the storage unit 16 stores in advance initial viewpoint information representing the first virtual viewpoint VP1 and the initial direction. The derivation unit 20C may read the initial viewpoint information stored in the storage unit 16 and determine the first virtual viewpoint VP1 and the initial direction represented by the read initial viewpoint information.

[0048] Then, the derivation unit 20C generates a two-dimensional image 40 in which the three-dimensional space S represented by the three-dimensional information 30 received from the acquisition unit 20A is viewed from the first virtual viewpoint VP1 in the initial direction. The display control unit 20D displays the two-dimensional image 40 generated by the derivation unit 20C on the display unit 14A. Therefore, for example, the two-dimensional image 40A shown in FIG. 2 is displayed on the display unit 14A.

[0049] The user specifies the first measurement point Q1 by operating the input unit 14B while viewing the two-dimensional image 40A displayed on the display unit 14A. In response to the user's specification, the receiving unit 20B outputs information representing the position of the specified first measurement point Q1 in the displayed two-dimensional image 40A to the derivation unit 20C.

[0050] When the derivation unit 20C receives information representing the position of the first measurement point Q1 in the two-dimensional image 40A, it derives first position information relating to a first position P1, which is a position corresponding to the first measurement point Q1 in the three-dimensional space S represented by the three-dimensional information 30, by a known coordinate transformation process.

[0051] The derivation unit 20C may perform coordinate conversion using a known coordinate conversion method such as ray casting, which converts the screen coordinates of the display unit 14A into coordinates in the three-dimensional space S represented by the three-dimensional information 30, for example.

[0052] Through these processes, the derivation unit 20C acquires first position information relating to the first position P1 in the three-dimensional space S represented by the three-dimensional information 30, based on the user's input to the input unit 14B.

[0053] The derivation unit 20C derives a position Q1' corresponding to the first position P1 in the two-dimensional image 40B. The position Q1' in the two-dimensional image 40B corresponding to the first position P1 is a position obtained by converting the first position P1, which is a coordinate in the three-dimensional space S, into a position in the two-dimensional image 40B viewed from the virtual viewpoint VP1.

[0054] The display control unit 20D displays first position identification information, which is information identifying the first measurement point Q1, at position Q1' in the two-dimensional image 40B. The first position identification information is displayed as a circular image hatched with diagonal lines, which indicates the first measurement point Q1 in FIGS. 3 and 4. The first position identification information is not limited to a circular image, as long as it can identify the position Q1' corresponding to the first position P1 in the two-dimensional image 40B. Other examples of the first identification information include a symbol (such as an arrow or polygonal icon) and text information indicating the first position P1 (for example, "measurement point 1"), or a combination of a symbol and text information.

[0055] When the virtual viewpoint is moved by a user's input to the input unit 14B, the display control unit 20D updates the position of the first identification information in the two-dimensional image 40B in the above-mentioned step in accordance with the changes accompanying the movement of the virtual viewpoint in the two-dimensional image 40B.

[0056] When a first measurement point Q1 is specified via the displayed two-dimensional image 40, the derivation unit 20C derives position information related to a second position P2 corresponding to a second measurement point Q2. In this embodiment, the derivation unit 20C derives position information of a candidate position CP in the three-dimensional space S that corresponds to a candidate point R for the second measurement point Q2 on the two-dimensional image 40 as the position information related to the second position P2.

[0057] As described above, candidate point R is a point on the two-dimensional image 40 that is a candidate for the second measurement point Q2. Then, candidate position CP represents a position in the three-dimensional space S represented by the three-dimensional information 30 that corresponds to candidate point R displayed on the two-dimensional image 40. In other words, candidate position CP is a position that is a candidate for the second position P2.

[0058] The derivation unit 20C derives the position information of the candidate position CP by either acquiring it from an input by the user to the input unit 14B or calculating it based on the first position information related to the first position P1.

[0059] For example, the derivation unit 20C determines a method for deriving the position information of the candidate position CP based on the input form of the user via the input unit 14B.

[0060] For example, assume that the input form of the user to the input unit 14B is a pointing device such as a mouse, or that the input form is a drag operation on a touch panel, and that the pointer position PP is clear on the display unit 14A displaying the two-dimensional image 40. A case where the pointer position PP is clear means that the receiving unit 20B has already received the pointer position PP on the display unit 14A from the input unit 14B.

[0061] In this case, the derivation unit 20C derives the position information of the candidate position CP by acquiring it from the user's input to the input unit 14B.

[0062] In detail, the derivation unit 20C identifies a pointer position PP on the two-dimensional image 40 that is specified by a user's operation using the input unit 14B. The derivation unit 20C sets the identified pointer position PP as a candidate point R, and identifies a position in the three-dimensional space S that corresponds to the candidate point R as a candidate position CPa. The candidate position Cpa is an example of a candidate position CP. The derivation unit 20C then derives position information of the candidate position CPa in the three-dimensional space S.

[0063] Through these processes, the derivation unit 20C acquires the position information of the candidate position CP, which is the position information related to the second position P2, from the user's input to the input unit 14B.

[0064] The pointer position PP, which is input to the input unit 14B by the user, is always displayed on the display unit 14A. Therefore, in this case, the derivation unit 20C can derive the position information of the candidate position CP, which is the position information related to the second position P2, in real time. Therefore, in this case, the user can specify the candidate position CP more intuitively and simply.

[0065] The derivation unit 20C may set as the candidate position CP any point on an extension of a straight line L1 connecting the first position P1 and a position in the three-dimensional space S corresponding to the pointer position PP. Fig. 5 shows a case where a candidate position CPb, which is any point on the straight line L1, is set as the candidate position CP. In this case, the derivation unit 20C derives position information of the candidate position CPb in the three-dimensional space S.

[0066] On the other hand, suppose that the input form of the user to the input unit 14B does not satisfy the above condition, for example, when the input form of the user to the input unit 14B is a touch operation on the UI unit 14, which is a touch panel, and the pointer position PP is not clear.

[0067] In this case, the derivation unit 20C derives the position information of the candidate position CP by calculating it based on the first position information.

[0068] 6 is an explanatory diagram of an example of calculating the position information of the candidate position CP. In FIG. 6, a schematic diagram of a three-dimensional space S represented by three-dimensional information 30 is shown.

[0069] In this case, the derivation unit 20C derives the position information of the candidate position CP based on the virtual viewpoint VP of the two-dimensional image 40 and the position information of the first position P1.

[0070] In detail, the derivation unit 20C identifies a straight line L2, which is a line segment connecting the virtual viewpoint VP and the first position P1. Then, the derivation unit 20C identifies an arbitrary position within a cylindrical region 56 of a predetermined radius with the straight line L2 as its central axis as a candidate position CP. Fig. 6 shows an example of a scene in which the derivation unit 20C identifies a candidate position CPc within the cylindrical region 56. The candidate position CPc is an example of the candidate position CP.

[0071] The derivation unit 20C identifies an area BS2 within the cylindrical area 56 that overlaps with the blind spot BS of the virtual camera 52 placed at the virtual viewpoint VP, and identifies any position within the overlapping area as the candidate position CPc. Alternatively, the derivation unit 20C may identify the center of gravity position within the overlapping area BS2 as the candidate position CPc.

[0072] Here, the blind spot BS in FIG. 6 will be described. The space FV within the field of view of the virtual camera in the three-dimensional space S has a conical shape with the virtual viewpoint VP of the virtual camera as its apex and the line of sight as its central axis. The space FV within the field of view in FIG. 6 is a diagram illustrating its cross-sectional shape. The blind spot BS in FIG. 6 is a region of the space FV within the field of view of the virtual camera 52 that is hidden from view due to an obstacle 50. Note that the blind spot BS can be identified using publicly known techniques. By setting a candidate position CPc within the region BS2 where the cylindrical region 56 and the blind spot BS overlap, the user can view the blind spot region that the user cannot view in the two-dimensional image 40 from the virtual viewpoint VP using the processing described below. By setting the candidate position CPc within the overlapping region BS2, it can be determined to be a candidate position that is not displayed in the two-dimensional image 40 from the virtual viewpoint VP2 and that is likely to be designated as the second position.

[0073] Then, the derivation unit 20C derives by identifying the position information of the identified candidate position CP in the three-dimensional space S represented by the three-dimensional information 30. That is, through these processes, the derivation unit 20C calculates the position information of the candidate position CP, which is position information related to the second position P2, based on the position information of the first position P1.

[0074] It is preferable that the radius of the cylindrical region 56 is larger than the predicted distance between the first position P1 and the second position P2, for example. When the radius of the cylindrical region 56 satisfies this condition, a candidate point R corresponding to the candidate position CP is displayed visibly within the two-dimensional image 40. Therefore, by visually recognizing the two-dimensional image 40, the user can designate the candidate point R displayed on the two-dimensional image 40 as the second measurement point Q2.

[0075] The derivation unit 20C may calculate a value that satisfies the above condition as the radius of the cylindrical region 56 in advance and store the value in the storage unit 16 in advance.

[0076] For example, the derivation unit 20C calculates the radius of the cylindrical region 56 in advance using the maximum and average dimensions of the three-dimensional object OB included in the three-dimensional information 30. The derivation unit 20C also calculates the radius of the cylindrical region 56 in advance based on the acquisition environment of the three-dimensional information 30 acquired by the acquisition unit 20A. For example, assume that the acquisition environment of the three-dimensional information 30 is an office environment including one or more pieces of furniture as the three-dimensional object OB. In this case, the derivation unit 20C may acquire in advance the maximum and average dimensions of the three-dimensional object OB that may be included in the office environment using a known method, and use these dimensions to calculate the radius of the cylindrical region 56. The derivation unit 20C may acquire the maximum and average dimensions of the three-dimensional object OB from specification information or standard information defining the standard specifications of the three-dimensional object OB that may be included in the office environment.

[0077] The derivation unit 20C may calculate the radius of the cylindrical region 56 based on selected standard information from the standard information corresponding to each of the multiple acquisition environments of the three-dimensional information 30. As another example of the acquisition environment and standard information, if the acquisition environment is a construction site of a house or building, standard information used for building materials and construction methods may be used. The user may also specify the acquisition environment or standard information in advance. The user may input the desired acquisition environment or standard information into the input unit 14B, and the derivation unit 20C may calculate the radius of the cylindrical region 56 based on the acquisition environment or standard information entered by the user. The user may also input the radius of the cylindrical region 56 itself in advance. By the user specifying the acquisition environment, standard information, radius, etc., the candidate point R can be more accurately displayed on the two-dimensional image 40. The standard information may be stored in a database in the information processing device (e.g., ROM 90B in FIG. 12) or acquired from an external database.

[0078] Furthermore, the derivation unit 20C may calculate the radius of the cylindrical region 56 based on the two-dimensional image 40. For example, the derivation unit 20C may set the range of the first measurement point Q1, which corresponds to ¼ of the image size of the two-dimensional image 40 when displayed, as the radius of the cylindrical region 56. In this case, the derivation unit 20C can identify, as the candidate position CP, an area around the first measurement point Q1 in the two-dimensional image 40, which corresponds to approximately ½ of the image size.

[0079] Furthermore, the derivation unit 20C prepares, as training data, a combination of the three-dimensional information 30 and the radius of the cylindrical region 56 used when identifying the candidate position CP for the three-dimensional information 30. The derivation unit 20C may then derive the radius of the cylindrical region 56 from the three-dimensional information 30 using a learning model that has been machine-learned in advance using multiple pieces of training data. The learning of the learning model may be performed by the control unit 20, or may be performed by an external information processing device connected via the communication unit 12.

[0080] Note that the three-dimensional information 30 may include information related to the three-dimensional object OB. That is, the three-dimensional information 30 may be information segmented for each three-dimensional object OB. Also, assume that the first position P1 corresponding to the first measurement point Q1 input by the user operating the input unit 14B is a position on the three-dimensional object OB.

[0081] In this case, the derivation unit 20C preferably sets the candidate position CP for the second position P2 at a position on the three-dimensional object OB that is the farthest from the first position P1. Alternatively, in this case, the derivation unit 20C may set the candidate position CP at a position on the three-dimensional object OB that is the farthest from the first position P1, that is a vertex of the three-dimensional object OB.

[0082] In detail, the derivation unit 20C identifies a three-dimensional object OB at which a first position P1 exists in the three-dimensional space S represented by the three-dimensional information 30, corresponding to a first measurement point Q1 specified by a user's input to the input unit 14B. Then, the derivation unit 20C identifies a blind spot BS on the identified three-dimensional object OB from a first virtual viewpoint VP of the two-dimensional image 40 that the user used when specifying the first measurement point Q1. The derivation unit 20C may set a candidate position CP at the position of the center of gravity within the identified blind spot BS or at the position of the vertex on the three-dimensional object OB that is farthest from the first position P1.

[0083] When the derivation unit 20C sets the candidate position CP at the position of the vertex on the three-dimensional object OB that is farthest from the first position P1, the display control unit 20D, which will be described later, can display a two-dimensional image 40 including the entire three-dimensional object OB where the first measurement point Q1 is located, on the display unit 14A. This allows the display control unit 20D to display the two-dimensional image 40 from which the second measurement point Q2 can be easily selected.

[0084] Through the above process, the derivation unit 20C derives the position information of the candidate position CP, which is position information related to the second position P2, by acquiring it from the user's input to the input unit 14B or by calculating it based on the first position information.

[0085] Here, if the candidate position CP derived by the derivation unit 20C is located in the blind spot BS of the virtual camera 52 placed at the first virtual viewpoint VP1, the candidate point R of the candidate position CP will not be displayed in the two-dimensional image 40 of the three-dimensional space S viewed from the first virtual viewpoint VP1.

[0086] Therefore, the derivation unit 20C changes the display form of the two-dimensional image 40 so that both the first position P1 and the candidate position CP of the second position P2 are visible within the two-dimensional image 40. In other words, the derivation unit 20C changes the display form of the two-dimensional image 40 so that the candidate point R corresponding to the candidate position CP is always visible within the two-dimensional image 40.

[0087] In this embodiment, the derivation unit 20C changes the display form by either generating a two-dimensional image 40 by moving the virtual viewpoint VP or by hiding an obstacle 50 that forms a blind spot BS.

[0088] First, we will explain the case where a two-dimensional image 40 is generated by moving the position of the virtual viewpoint VP so that both the first position P1 and the candidate position CP of the second position P2 are visible within the two-dimensional image 40.

[0089] 7 is an explanatory diagram of an example of generating a two-dimensional image 40 in which the position of the virtual viewpoint VP is moved. In FIG. 7, a schematic diagram of a three-dimensional space S represented by three-dimensional information 30 is shown.

[0090] For example, assume that before the derivation unit 20C acquires the second position P2, the display control unit 20D displays a two-dimensional image 40 (corresponding to the first two-dimensional image) on the display unit 14A, with the first virtual viewpoint VP1 as the virtual viewpoint VP. That is, assume that the virtual viewpoint VP of the two-dimensional image 40 used by the user when specifying the first measurement point Q1 is the first virtual viewpoint VP1.

[0091] Then, if the candidate point R corresponding to the derived candidate position CP is not included in the two-dimensional image 40 (first two-dimensional image), the derivation unit 20C generates a second two-dimensional image, which is the two-dimensional image 40 with the second virtual viewpoint VP2 as the virtual viewpoint VP.

[0092] In detail, the derivation unit 20C sets the second virtual viewpoint VP2 and the direction in which the three-dimensional space S is viewed from the second virtual viewpoint VP2 so that both the first position P1 and the candidate position CP are within the field of view of the virtual camera 52 and so that both the first position P1 and the candidate position CP are outside the blind spot BS of the virtual camera 52.

[0093] Specifically, the derivation unit 20C places the second virtual viewpoint VP2 within a range E of 90 degrees left and right from the center of a vector representing the direction in which the virtual camera 52 placed at the first virtual viewpoint VP1 views the three-dimensional space S, and at a position where both the first position P1 and the candidate position CP are outside the blind spot BS of the virtual camera 52.

[0094] In addition, if the three-dimensional object OB placed at the first position P1 can be identified, the derivation unit 20C may place the second virtual viewpoint VP2 at a position where the entire view of the three-dimensional object OB is within the angle of view of the virtual camera 52.

[0095] Furthermore, the derivation unit 20C may set the direction in which the three-dimensional space S is viewed from the second virtual viewpoint VP2 to a direction corresponding to the midpoint between the first position P1 and the candidate position CP.

[0096] Then, the derivation unit 20C generates, as a second two-dimensional image, a two-dimensional image 40 obtained by viewing the three-dimensional space S represented by the three-dimensional information 30 from a second virtual viewpoint VP2, which is the virtual viewpoint VP after the viewpoint conversion. Note that a known method may be used to move the virtual viewpoint VP in the three-dimensional space S and to generate the two-dimensional image 40 obtained by viewing the three-dimensional space S from the virtual viewpoint VP after the movement. Therefore, the second two-dimensional image, which is the two-dimensional image 40 obtained by viewing the three-dimensional space S from the second virtual viewpoint VP2, includes both the first position P1 and the candidate position CP.

[0097] The display control unit 20D displays the two-dimensional image 40 generated by the derivation unit 20C on the display unit 14A. That is, when the candidate position CP is located in the blind spot BS of the virtual camera 52, the display control unit 20D displays the two-dimensional image 40 on the display unit 14A, in which the virtual viewpoint VP is viewpoint-converted from the first virtual viewpoint VP1 to the second virtual viewpoint VP2 so that the candidate position CP is outside the blind spot BS of the virtual camera 52.

[0098] Alternatively, the display control unit 20D may generate a first two-dimensional image and a second two-dimensional image and display them on the display unit 14A. In this case, the display control unit 20D acquires, for example, camera environment information such as the position, rotation, and angle of view of the virtual camera 52 of the first virtual viewpoint VP1 from the derivation unit 20C. Then, the display control unit 20D displays, on the display unit 14A, a first two-dimensional image of the three-dimensional space S viewed from the position of the virtual camera 52 represented by the camera environment information. Then, the display control unit 20D may change the camera environment information of the first virtual viewpoint VP1 to the camera environment information of the second virtual viewpoint VP2, thereby displaying the second two-dimensional image on the display unit 14A.

[0099] Therefore, the display unit 14A displays the two-dimensional image 40 in which the first position P1 and the candidate position CP are displayed so that they can be seen.

[0100] The derivation unit 20C calculates a position Q1' on the two-dimensional image 40 corresponding to the first position P1 and a position CP' on the two-dimensional image corresponding to the candidate position CP in the two-dimensional image 40 of the three-dimensional space S viewed from the second virtual viewpoint VP2.

[0101] The display control unit 20D displays first identification information indicating the first position P1 at position Q1' on the two-dimensional image 40, and displays second identification information indicating the candidate position CP at position CP' on the two-dimensional image 40. The first identification information and the second identification information correspond to the circular symbols shown in FIG. 3. The display control unit displays third identification information, which is a line segment (dashed line in FIG. 3) connecting position Q1' and position CP'.

[0102] The first identification information and the second identification information may be any information that allows the first position P1 and the candidate position CP to be visually recognized in the two-dimensional image 40, and are not limited to the circular symbols shown in FIG. 3. For example, other examples of the first identification information and the second identification information may include symbols (arrows, polygonal icons, etc.), text information indicating the first position P1 (e.g., "measurement point 1"), or a combination of symbols and text information. Furthermore, the first identification information and the second identification information may be displayed with different shapes, colors, etc.

[0103] The third identification information is not limited to the dashed line segment connecting the first identification information and the second identification information in Fig. 3. Another example of the third identification information may be an arrow pointing from position Q1' to position CP'.

[0104] The display control unit 20D displays the first identification information, the second identification information, and the third identification information on the two-dimensional image 40 according to the positions Q1' and CP', which are updated in accordance with changes in the two-dimensional image as the virtual viewpoint VP2 moves.

[0105] Since the first identification information, the second identification information, and the third identification information are displayed in the two-dimensional image 40D, the user can visually confirm the first position P1 and the candidate point CP in the two-dimensional image 40D, and can therefore intuitively confirm and specify the second position P2.

[0106] In addition, when a second two-dimensional image is displayed on the display unit 14A instead of the first two-dimensional image, it is preferable that the display control unit 20D displays on the display unit 14A a two-dimensional image 40 in which the virtual viewpoint VP is continuously moved from the first virtual viewpoint VP1 toward the second virtual viewpoint VP2.

[0107] That is, the display control unit 20D may sequentially display, on the display unit 14A, a plurality of two-dimensional images 40 obtained by viewing the three-dimensional space S from each of a plurality of virtual viewpoints VP between the first virtual viewpoint VP1 and the second virtual viewpoint VP2. The generation of the plurality of two-dimensional images 40 may be performed by the derivation unit 20C or the display control unit 20D.

[0108] In addition, the display control unit 20D may display first identification information and second identification information that identify the first position P1 and the candidate position CP on each of multiple two-dimensional images 40 of the three-dimensional space S viewed from each of multiple virtual viewpoints VP between the first virtual viewpoint VP1 and the second virtual viewpoint VP2.

[0109] Specifically, for example, the display control unit 20D continuously changes the camera environment information, such as the position, rotation, and angle of view, of the virtual camera 52 of the first virtual viewpoint VP1 toward the camera environment information of the second virtual viewpoint VP2 at a predetermined interval (for example, one second).Then, when the camera environment information of the virtual camera 52 of the first virtual viewpoint VP1 and the camera environment information of the second virtual viewpoint VP2 match, the display control unit 20D switches the display to the second two-dimensional image.

[0110] The display control unit 20D sequentially displays on the display unit 14A a plurality of two-dimensional images 40 of the three-dimensional space S viewed from each of a plurality of virtual viewpoints VP between the first virtual viewpoint VP1 and the second virtual viewpoint VP2, thereby making it possible to provide the user with an easy-to-understand view of the movement of the virtual viewpoint VP. Furthermore, sudden changes in the displayed two-dimensional images 40 can be prevented, improving the user's recognition.

[0111] The display controller 20D may display both the first and second two-dimensional images on the display 14A.

[0112] FIG. 8 is a schematic diagram showing an example of a scene in which both the first two-dimensional image and the second two-dimensional image are displayed on the display unit 14A.

[0113] For example, assume that two-dimensional image 40B is the first two-dimensional image, and that two-dimensional image 40C is the second two-dimensional image.

[0114] In this case, for example, the display control unit 20D may display on the display unit 14A a two-dimensional image in which the two-dimensional image 40C is superimposed on the two-dimensional image 40B. As shown in FIG. 8, for example, the two-dimensional image 40B, which is an example of a first two-dimensional image, includes the first position P1, but the candidate position CP is hidden because it is in a blind spot BS of the obstacle 50. On the other hand, the two-dimensional image 40C, which is an example of a second two-dimensional image, displays both the first position P1 and the candidate position CP so that they can be seen. This allows the user to always see the candidate position CP.

[0115] Furthermore, the display control unit 20D displays on the display unit 14A a two-dimensional image in which the two-dimensional image 40C is superimposed on the two-dimensional image 40B, allowing the user to check both the two-dimensional images 40 before and after the viewpoint conversion. This makes it easier for the user to check the candidate position CP. Furthermore, confusion caused by the sudden switching of the two-dimensional image 40 can be reduced.

[0116] The display control unit 20D may superimpose first identification information for identifying the first position P1 and second identification information for identifying the second position P2 on the two-dimensional image 40. For example, as shown in Fig. 8, the display control unit 20D may superimpose icon images representing the first position P1 and the second position P2 or the candidate position CP on the two-dimensional image 40. In Fig. 8, the icon images are shown as circles as an example.

[0117] The first identification information for identifying the first position P1 and the second identification information for identifying the second position P2 may be any information that can identify the first position P1, the second position P2, or the candidate position CP, and is not limited to an icon image. The identification information may be, for example, a character image.

[0118] The above describes the case where there is one second position P2 or candidate position CP, but the derivation unit 20C may calculate multiple second positions P2 or candidate positions CP, and the display control unit 20D may display the two-dimensional image 40 on the display unit 14A so that the two-dimensional image 40 includes positions corresponding to the first position P1 and all of the multiple candidate positions CP (or multiple second positions P2).

[0119] Next, a case will be described in which the derivation unit 20C hides the obstacle 50 that forms the blind spot BS so that both the first position P1 and the candidate position CP are visible within the two-dimensional image 40. Hiding the obstacle 50 means that at least a part of the obstacle 50 is displayed transparently or semi-transparently.

[0120] In detail, the derivation unit 20C generates a two-dimensional image 40 in which an obstacle 50 that exists between the virtual viewpoint VP and the candidate position CP and that obstructs the display of the candidate position CP is made transparent or semi-transparent. Then, the display control unit 20D displays the two-dimensional image 40 generated by the derivation unit 20C on the display unit 14A. Note that the generation of the two-dimensional image 40 may be performed by the display control unit 20D.

[0121] FIG. 9 is an explanatory diagram of an example of generating a two-dimensional image 40 in which the obstacle 50 is not displayed.

[0122] The derivation unit 20C derives a straight line L3 connecting a candidate position CP in the three-dimensional space S and a virtual viewpoint VP of the virtual camera 52. Then, the derivation unit 20C makes the three-dimensional information of an object that intersects with the derived straight line L3 in the three-dimensional space S represented by the three-dimensional information 30 transparent or semi-transparent. FIG. 9 shows an example in which the three-dimensional information representing an obstacle 50 that intersects with the straight line L3 is made transparent or semi-transparent. The degree of semi-transparency may be such that the candidate position CP is visibly displayed when a two-dimensional image 40 of the three-dimensional space S viewed from the virtual viewpoint VP is displayed on the display unit 14A.

[0123] By displaying the two-dimensional image 40 in which the obstacle 50 is transparent or semi-transparent on the display unit 14A, the movement of the virtual viewpoint VP can be reduced and the candidate position CP can be displayed so that it is easily visible to the user.

[0124] The derivation unit 20C may hide or make transparent the entire object that intersects with the straight line L3 in the three-dimensional space S, or may hide or make transparent a part of the object. When hiding a part of the object, the derivation unit 20C may hide a part of the obstacle 50 so that the candidate position CP, which is located within the blind spot BS formed by the obstacle 50 arranged at a position that intersects with the straight line L3, is outside the blind spot BS of the virtual camera 52. By hiding a part of the obstacle 50, it is possible to prevent an area that is not intended by the user from being hidden.

[0125] Through these processes, the display control unit 20D displays on the display unit 14A a two-dimensional image 40 in which an obstacle 50 that exists between the virtual viewpoint VP and the candidate position CP and that obstructs the display of the candidate position CP is made transparent or semi-transparent.

[0126] Therefore, both the first position P1 and the candidate position CP are displayed so as to be visible on the display unit 14A.

[0127] Although the above description has been given of the case where there is one second position P2 or one candidate position CP, the derivation unit 20C may calculate a plurality of second positions P2 or candidate positions CP. Furthermore, the display control unit 20D may display the two-dimensional image 40 on the display unit 14A so that the two-dimensional image 40 includes positions corresponding to all of the first position P1 and the plurality of candidate positions CP (or the plurality of second positions P2).

[0128] Then, when the derivation unit 20C acquires a confirmation instruction for the candidate point R of the displayed candidate position CP through a user's input to the input unit 14B, the derivation unit 20C confirms the candidate position CP as the second position P2.

[0129] Returning to Figure 1, the explanation will be continued. The distance calculation unit 20E calculates the distance between the first position P1 and the second position P2. The distance calculation unit 20E specifies first position information related to the first position P1 and position information related to the second position P2 in the three-dimensional space S, thereby specifying the position coordinates of each of the first position P1 and the second position P2 in the three-dimensional space S. Then, the distance calculation unit 20E calculates the Euclidean distance between the first position P1 and the second position P2 using the specified position coordinates, thereby calculating the distance between the first position P1 and the second position P2.

[0130] Furthermore, when the derivation unit 20C further derives position information of a new candidate position CP and the candidate position CP is determined as a second position P2, the distance calculation unit 20E may use the first position P1, the second position P2, and one or more other newly determined second positions P2 to calculate the area of ​​the region defined by these positions.

[0131] Next, the flow of information processing executed by the information processing device 10 of this embodiment will be described.

[0132] FIG. 10 is a flowchart showing an example of the flow of information processing executed by the information processing device 10 of this embodiment.

[0133] The acquisition unit 20A acquires the three-dimensional information 30 from the imaging device 11 (step S100).

[0134] The derivation unit 20C determines a first virtual viewpoint VP1, which is a virtual viewpoint as an initial position, and a viewing direction in which the three-dimensional space S is viewed from the first virtual viewpoint VP1 (step S102).

[0135] Next, the derivation unit 20C generates a two-dimensional image 40 obtained by viewing the three-dimensional space S represented by the three-dimensional information 30 acquired in step S100 from the first virtual viewpoint VP1 determined in step S102 in the initial direction (step S104). The processing of step S104 generates the two-dimensional image 40, which is the first two-dimensional image.

[0136] The display control unit 20D displays the two-dimensional image 40 generated in step S104 on the display unit 14A (step S106). Therefore, for example, the two-dimensional image 40A shown in FIG.

[0137] The derivation unit 20C acquires first position information relating to the first position P1 in the three-dimensional space S represented by the three-dimensional information 30 acquired in step S100, from a user's input to the input unit 14B (step S108).

[0138] In detail, the user specifies a first measurement point Q1 by operating the input unit 14B while viewing the two-dimensional image 40A displayed on the display unit 14A in step S106. In response to the user's specification, the receiving unit 20B outputs information representing the position of the specified first measurement point Q1 in the displayed two-dimensional image 40A to the derivation unit 20C. Upon receiving the information representing the position of the first measurement point Q1 in the two-dimensional image 40A, the derivation unit 20C derives first position information relating to a first position P1, which is a position corresponding to the first measurement point Q1, in the three-dimensional space S represented by the three-dimensional information 30, by a known coordinate conversion process. Through these processes, the derivation unit 20C acquires the first position information relating to the first position P1.

[0139] Next, the derivation unit 20C determines whether the user designates a candidate position CP (step S110). In step S110, the derivation unit 20C makes the determination of step S110 by determining whether position information of the candidate position CP is acquired from an input by the user to the input unit 14B. In this embodiment, the derivation unit 20C determines whether the input form by the user to the input unit 14B satisfies a predetermined condition. The predetermined condition is when the input form is a pointing device such as a mouse, or when the input form is a drag operation on a touch panel, and the pointer position PP is clear on the display unit 14A displaying the two-dimensional image 40.

[0140] If the determination in step S110 is affirmative (step S110: Yes), the process proceeds to step S112.

[0141] In step S112, the derivation unit 20C acquires position information of a candidate position CP from the pointer position PP input by the user to the input unit 14B (step S112). In step S112, the derivation unit 20C identifies the pointer position PP on the two-dimensional image 40 that is specified by the user's operation using the input unit 14B. The derivation unit 20C sets the identified pointer position PP as a candidate point R, and identifies a position in the three-dimensional space S that corresponds to the candidate point R as a candidate position CP. Then, the derivation unit 20C acquires position information of the candidate position CP in the three-dimensional space S. Then, the process proceeds to step S116, which will be described later.

[0142] On the other hand, if the determination in step S110 is negative (step S110: No), the process proceeds to step S114.

[0143] In step S114, the derivation unit 20C calculates the position information of the candidate position CP based on the first position information (step S114). The derivation unit 20C calculates the position information of the candidate position CP based on the virtual viewpoint VP of the two-dimensional image 40 displayed on the display unit 14A and the position information of the first position P1. Then, the process proceeds to step S116.

[0144] In step S116, the derivation unit 20C determines whether the candidate position CP is within a blind spot BS based on the derived position information of the candidate position CP (step S116). The derivation unit 20C determines whether the candidate position CP is located within a blind spot BS of the virtual camera 52 placed at the first virtual viewpoint VP1, which is the virtual viewpoint VP of the two-dimensional image 40 currently displayed on the display unit 14A.

[0145] If it is determined that the candidate position CP is located outside the blind spot BS (step S116: No), the process proceeds to step S118.

[0146] In step S118, the display control unit 20D displays, on the display unit 14A, a two-dimensional image 40 of the three-dimensional space S represented by the three-dimensional information 30 viewed from the first virtual viewpoint VP1 used in the determination of step S116 (step S118). By the processing of step S118, the two-dimensional image 40 in which both the first position P1 and the candidate position CP derived in step S112 or step S114 are displayed so as to be visible is displayed on the display unit 14A. Then, the process proceeds to step S124, which will be described later.

[0147] On the other hand, if it is determined that the candidate position CP is located within the blind spot BS (step S116: Yes), the process proceeds to step S120.

[0148] In step S120, the derivation unit 20C changes the display form of the two-dimensional image 40 so that both the first position P1 and the candidate position CP are visible in the two-dimensional image 40 (step S120). In step S120, the derivation unit 20C changes the display form by one of the following processes: generating a two-dimensional image 40 in which the virtual viewpoint VP is moved to a second virtual viewpoint VP2, or hiding an obstacle 50 that forms a blind spot BS, so that the candidate point R corresponding to the candidate position CP is always visible in the two-dimensional image 40. Then, the display control unit 20D displays the two-dimensional image 40 whose display form has been changed in step S120 on the display unit 14A (step S122).

[0149] By the processes of step S120 and step S122, the two-dimensional image 40 is displayed on the display unit 14A, in which both the first position P1 and the candidate position CP derived in step S112 or step S114 are visually recognizable. Then, the process proceeds to step S124.

[0150] In step S124, the derivation unit 20C determines whether a confirmation instruction for the candidate point R of the candidate position CP displayed in step S118 or step S122 has been acquired by a user's input to the input unit 14B (step S124). If the determination in step S124 is negative (step S124: No), the process returns to step S110. If the determination in step S124 is positive (step S124: Yes), the derivation unit 20C confirms the candidate position CP for which the confirmation instruction has been accepted as the second position P2 (step S126).

[0151] The distance calculation unit 20E calculates the distance between the first position P1 and the second position P2 (step S128). The distance calculation unit 20E calculates the distance between the first position P1 and the second position P2 using the first position information acquired in step S108 and the position information of the second position P2 located on the seabed in step S126.

[0152] The display control unit 20D displays the distance between the first position P1 and the second position P2 calculated in step S128 on the display unit 14A (step S130), and then ends this routine.

[0153] As described above, the information processing device 10 of this embodiment displays, on the display unit 14A, a two-dimensional image 40 of the three-dimensional space S represented by the three-dimensional information 30 viewed from the virtual viewpoint VP of the virtual camera 52. The information processing device 10 includes a derivation unit 20C and a display control unit 20D. The derivation unit 20C acquires first position information related to a first position P1 in the three-dimensional space S through a user's input to the input unit 14B, and acquires position information related to a second position P2 from the user's input to the input unit 14B or calculates it based on the first position information. The display control unit 20D displays, on the display unit 14A, a two-dimensional image 40 including the first position P1 and the second position P2.

[0154] For this reason, in the information processing device 10 of this embodiment, even if the second position P2 is within the blind spot BS of the virtual camera 52 in the three-dimensional space S, the display unit 14A displays the two-dimensional image 40 including the first position P1 and the second position P2. That is, the information processing device 10 of this embodiment displays the two-dimensional image 40 including the position of the measurement target in a visibly recognizable manner on the display unit 14A.

[0155] Therefore, the user can specify the desired second position P2 by visually viewing the displayed two-dimensional image 40 without performing viewpoint manipulation to change the viewpoint so that the desired point is displayed within the two-dimensional image 40.

[0156] Therefore, the information processing device 10 of this embodiment can easily specify the position of the measurement target in the three-dimensional space S represented by the three-dimensional information 30.

[0157] Furthermore, with the information processing device 10 of this embodiment, the user can complete measurement of the distance between desired points or the area with minimal viewpoint manipulation of the virtual viewpoint VP or without manipulating the viewpoint at all.

[0158] (Variation 1) In the above embodiment, an example has been described in which the measurement system 1 includes the imaging device 11 and the information processing device 10. However, the information processing device 10 may include at least the derivation unit 20C and the display control unit 20D, and a device including the information processing device 10 and the distance calculation unit 20E may be configured as the measurement device.

[0159] (Variation 2) In the above embodiment, an example has been described in which information processing device 10 includes acquisition unit 20A, reception unit 20B, derivation unit 20C, distance calculation unit 20E, and distance calculation unit 20F. However, at least some of the functions included in information processing device 10 may be installed in a server device or the like, which is an external information processing device connected to a network or the like.

[0160] FIG. 11 is a schematic diagram showing an example of a measurement system 1B of this modified example.

[0161] The measurement system 1B includes an imaging device 11, a storage unit 17, an information processing device 10A, and a server device 10B. The imaging device 11 and the storage unit 17 are communicatively connected. The storage unit 17, the information processing device 10A, and the server device 10B are communicatively connected via a network NW.

[0162] The storage unit 17 stores the three-dimensional information 30 obtained by the imaging device 11. The information processing device 10A includes a communication unit 12, a UI unit 14, and a control unit 21. The communication unit 12, the UI unit 14, and the control unit 21 are connected to each other so that they can communicate with each other. The control unit 21 includes a reception unit 20B and a display control unit 20D. The communication unit 12, the UI unit 14, the reception unit 20B, and the display control unit 20D are the same as those in the above embodiment.

[0163] Server device 10B includes a communication unit 13 and a control unit 23. Communication unit 13 communicates with information processing device 10A and storage unit 17 via network NW. Control unit 23 includes an acquisition unit 20A, a derivation unit 20C, and a distance calculation unit 20E. Acquisition unit 20A, derivation unit 20C, and distance calculation unit 20E are the same as those in the above embodiment.

[0164] As shown in FIG. 11, at least some of the functions of the information processing device 10 may be installed in a server device 10B.

[0165] Next, the hardware configurations of the information processing device 10, the information processing device 10A, and the server device 10B in the above-described embodiment and modification will be described.

[0166] FIG. 12 is a diagram showing an example of a hardware configuration of the information processing device 10, the information processing device 10A, and the server device 10B according to the above embodiment and the modified example.

[0167] The information processing device 10, information processing device 10A, and server device 10B of the above embodiment and modified examples include a control device such as a CPU 90A, a storage device such as a ROM (Read Only Memory) 90B and a RAM (Random Access Memory) 90C, an HDD (Hard Disk Drive), an I / F 90D that connects to a network and communicates, and a bus 90E that connects each part.

[0168] The programs executed by the information processing device 10, the information processing device 10A, and the server device 10B in the above-described embodiment and modified examples are provided in a state that they are pre-installed in the ROM 90B or the like.

[0169] The programs executed by the information processing device 10, information processing device 10A, and server device 10B of the above-described embodiments and variations of the above-described embodiments may be configured to be provided as a computer program product by being recorded in an installable or executable format on a computer-readable recording medium such as a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk Recordable), or a DVD (Digital Versatile Disk).

[0170] Furthermore, the programs executed by information processing device 10, information processing device 10A, and server device 10B of the above-described embodiment and modified examples may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the programs executed by information processing device 10, information processing device 10A, and server device 10B of the above-described embodiment and modified examples may be provided or distributed via a network such as the Internet.

[0171] The programs executed by the information processing device 10, the information processing device 10A, and the server device 10B of the above-described embodiment and modified example can cause a computer to function as each of the information processing device 10, the information processing device 10A, and the server device 10B of the above-described embodiment and modified example. In this computer, the CPU 90A can read the programs from a computer-readable storage medium onto a main storage device and execute the programs.

[0172] Furthermore, the information processing device 10, the information processing device 10A, and the server device 10B in the above-described embodiment and modified examples may be realized as virtual machines operating on a cloud system.

[0173] Although the embodiments and modifications of the present invention have been described above, they are presented as examples and are not intended to limit the scope of the invention. These novel embodiments and modifications can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and modifications are included within the scope and spirit of the invention, and are also included in the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0174] 10, 10A Information processing equipment 20C Derivation part 20D Display control unit 20E Distance calculation unit [Prior art documents] [Patent documents]

[0175] [Patent Document 1] Japanese Patent Application Publication No. 2017-026488

Claims

1. An information processing device that displays, on a display unit, a two-dimensional image obtained by viewing a three-dimensional space represented by three-dimensional information from a virtual viewpoint of a virtual camera, a derivation unit that acquires first position information related to a first position in the three-dimensional space by a user's input to an input unit, and calculates position information related to a second position based on the first position information; a display control unit that displays the two-dimensional image including the first position and the second position on the display unit; An information processing device comprising:

2. The second position is The information processing device according to claim 1 , wherein the position is determined based on an input specifying position information calculated by the derivation unit based on the first position information with respect to the two-dimensional image displayed on the display unit.

3. The second position is a position determined based on the virtual viewpoint and the first position, a position within a cylindrical region of a predetermined radius with a line segment connecting the virtual viewpoint and the first position as its central axis; 3. The information processing device according to claim 1.

4. The information processing device according to claim 3 , wherein the derivation unit calculates the predetermined radius based on the three-dimensional information.

5. the three-dimensional information includes information related to a three-dimensional object; the first position is a position on the three-dimensional object; the second position is a position on the three-dimensional object that is the farthest from the first position; 5. The information processing device according to claim 1.

6. the three-dimensional information includes information related to a three-dimensional object; the first position is a position on the three-dimensional object; 6. The information processing device according to claim 1, wherein the second position is a position on the three-dimensional object that is a vertex of the three-dimensional object that is farthest from the first position.

7. The display control unit before the derivation unit acquires the second position, a first two-dimensional image, which is the two-dimensional image with a first virtual viewpoint as the virtual viewpoint, is displayed on the display unit; When the second position is not included in the first two-dimensional image, a second two-dimensional image is displayed on the display unit, the second two-dimensional image being the two-dimensional image with a second virtual viewpoint as the virtual viewpoint; the second two-dimensional image includes the first position and the second position; 7. The information processing device according to claim 1.

8. The display control unit displaying the first two-dimensional image and the second two-dimensional image on the display unit; The information processing device according to claim 7 .

9. The display control unit a plurality of two-dimensional images of the three-dimensional space viewed from each of a plurality of virtual viewpoints between the first virtual viewpoint and the second virtual viewpoint are sequentially displayed on the display unit; The information processing device according to claim 7 .

10. The display control unit displaying the two-dimensional image in such a manner that an obstacle that exists between the virtual viewpoint and the second position and that obstructs the display of the second position is made transparent or semi-transparent; 7. The information processing device according to claim 1.

11. The display control unit identification information for identifying the first position and identification information for identifying the second position are displayed superimposed on the two-dimensional image; The information processing device according to any one of claims 1 to 10.

12. An information processing device according to any one of claims 1 to 11; a distance calculation unit that calculates a distance between the first position and the second position; A measuring device comprising:

13. An information processing method for displaying, on a display unit, a two-dimensional image obtained by viewing a three-dimensional space represented by three-dimensional information from a virtual viewpoint of a virtual camera, the method comprising: acquiring first position information relating to a first position in the three-dimensional space by a user's input to an input unit, and calculating position information relating to a second position based on the first position information; displaying the two-dimensional image including the first position and the second position on the display unit; An information processing method including:

14. An information processing program executed by an information processing device that displays, on a display unit, a two-dimensional image of a three-dimensional space represented by three-dimensional information viewed from a virtual viewpoint of a virtual camera, acquiring first position information relating to a first position in the three-dimensional space by a user's input to an input unit, and calculating position information relating to a second position based on the first position information; displaying the two-dimensional image including the first position and the second position on the display unit; An information processing program including:

Images