Information processing device, information processing method, and recording medium

Abstract

In order to resolve the problem of more accurately estimating the orientation of a mobile body in a three-dimensional space, this information processing device (1) comprises an acquiring unit (11) that acquires a three-dimensional model for a mobile body, and an identifying unit (12) that refers to the three-dimensional model and identifies candidates for a plurality of measurement points necessary to estimate the orientation of the mobile body.

Description

[Technical field]

[0001] The present invention relates to a technique for estimating the attitude of a moving object. [Background technology]

[0002] Patent Document 1 describes a technique for estimating the attitude of a moving object by using parameters detected by a sensor provided on the moving object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2021-21673 Summary of the Invention [Problem to be solved by the invention]

[0004] When the ground surface of the moving body is not a two-dimensional plane, such as including a slope or a depression, it is required to accurately estimate the attitude of the moving body in three-dimensional space. In the technology described in Patent Document 1, for example, when a six-axis sensor is provided on the moving body, it is difficult to accurately estimate the attitude in three-dimensional space because the yaw angle cannot be detected with high accuracy. In addition, for example, when a nine-axis sensor is provided on the moving body, it becomes necessary to measure the difference between the earth's axis and the on-site coordinate system, which makes the process of accurately estimating the attitude in three-dimensional space complicated.

[0005] One aspect of the present invention has been made in consideration of the above problems, and provides a technique for more accurately estimating the orientation of a moving body in three-dimensional space. [Means for solving the problem]

[0006] An information processing device according to one aspect of the present invention includes an acquisition means for acquiring a three-dimensional model of a moving body, and an identification means for identifying candidates for a plurality of measurement points required for estimating the posture of the moving body by referring to the three-dimensional model.

[0007] An information processing method according to one aspect of the present invention includes obtaining a three-dimensional model of a moving body, and referring to the three-dimensional model, identifying a plurality of candidate measurement points required for estimating the posture of the moving body.

[0008] According to one aspect of the present invention program is a program that makes a computer function as an information processing device. In The computer is caused to function as an acquisition unit that acquires a three-dimensional model of a moving object, and as a specification unit that refers to the three-dimensional model and specifies candidates for a plurality of measurement points required for estimating the posture of the moving object. do. In addition, the program Recording medium on which also falls within the scope of one aspect of the present invention. Effect of the Invention

[0009] According to one aspect of the present invention, the posture of a moving object in a three-dimensional space can be estimated with higher accuracy. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to a first exemplary embodiment. [Diagram 2] 1 is a flow chart showing the flow of an information processing method according to the first exemplary embodiment. [Diagram 3] FIG. 11 is a block diagram illustrating a configuration of an information processing device according to a second exemplary embodiment. [Figure 4] 11 is a schematic diagram for explaining a specific example of a three-dimensional model in the exemplary embodiment 2. FIG. [Diagram 5] 13 is a schematic diagram illustrating an example of coordinate system axis information in the exemplary embodiment 2. FIG. [Figure 6] FIG. 11 is a flow diagram illustrating the flow of an information processing method according to the second exemplary embodiment. [Figure 7] 11A to 11C are schematic diagrams illustrating specific examples of the identification process and the extraction process in the second exemplary embodiment. [Figure 8]13A to 13C are schematic diagrams illustrating a specific example of an extraction process in the second exemplary embodiment. [Figure 9] FIG. 11 is a diagram showing a specific example of display data in the second exemplary embodiment. [Figure 10] FIG. 11 is a diagram showing a specific example of a display screen in exemplary embodiment 2. [Figure 11] FIG. 11 is a block diagram illustrating a configuration of an information processing device according to a third exemplary embodiment. [Figure 12] FIG. 11 is a flow diagram illustrating the flow of an information processing method according to the third exemplary embodiment. [Figure 13] 13 is a schematic diagram for explaining an outline of a specific example of an estimation process in exemplary embodiment 3. FIG. [Figure 14] 13A to 13C are diagrams illustrating details of a specific example of an estimation process in the exemplary embodiment 3. [Figure 15] FIG. 2 is a block diagram illustrating an example of a hardware configuration of an information processing device according to each exemplary embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Example 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described later.

[0012] <Configuration of information processing device 1> The configuration of an information processing device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1.

[0013] 1, the information processing device 1 includes an acquisition unit 11 and an identification unit 12. The acquisition unit 11 acquires a three-dimensional model of a moving object. The identification unit 12 refers to the three-dimensional model and identifies candidates for a plurality of measurement points required for estimating the posture of the moving object.

[0014] <Flow of information processing method S1> The information processing device 1 configured as above executes an information processing method S1 according to this exemplary embodiment. The flow of the information processing method S1 will be described with reference to Fig. 2. Fig. 2 is a flow diagram illustrating the flow of the information processing method S1.

[0015] 2, information processing method S1 includes steps S11 to S12. In step S11, acquisition unit 11 acquires a three-dimensional model of a moving object. In step S12, identification unit 12 refers to the three-dimensional model and identifies candidates for a plurality of measurement points required for estimating the posture of the moving object.

[0016] <Example of program implementation> When the information processing device 1 is configured by a computer, the following program is stored in a memory (recording medium) referenced by the computer. The program is a program for causing the computer to function as the information processing device 1, and causes the computer to function as an acquisition unit 11 that acquires a three-dimensional model of a moving body, and an identification unit 12 that refers to the three-dimensional model and identifies candidates for a plurality of measurement points required for estimating the posture of the moving body.

[0017] The computer reads the program from the memory and executes it, thereby implementing the information processing method S1 described above.

[0018] <Advantages of this exemplary embodiment> As described above, in this exemplary embodiment, a configuration is adopted in which a three-dimensional model of a moving body is acquired, and a plurality of candidates for measurement points required for estimating the posture of the moving body is identified by referring to the three-dimensional model. Such a plurality of candidates for measurement points can be used to more accurately estimate the posture of the moving body in three-dimensional space. Therefore, the posture of the moving body in three-dimensional space can be more accurately estimated.

[0019] Exemplary embodiment 2 A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and the description thereof will be omitted as appropriate.

[0020] <Overview of information processing device 1A> The information processing device 1A according to this exemplary embodiment is a device that presents measurement points to be measured by an external measurement device TS in order to estimate the attitude of a construction machine MV. The construction machine MV is an example of a moving body according to each exemplary embodiment of the present application.

[0021] For example, the user operates the external measuring device TS to perform measurements on the measurement points presented by the information processing device 1A. In addition, if the construction machine MV is equipped with an attitude sensor, the user corrects the measurement value of the attitude sensor using the measurement results by the external measuring device TS. The attitude sensor is, for example, a six-axis sensor (IMU: Inertial Measurement Unit) that measures three-dimensional angular velocity and acceleration. Inertial The external measurement device TS may be, but is not limited to, a total station that measures the distance and angle to a measurement point.

[0022] <Configuration of information processing device 1A> The configuration of the information processing device 1A according to this exemplary embodiment will be described with reference to FIG. 3. FIG. 3 is a block diagram illustrating the configuration of the information processing device 1A. As shown in FIG. 3, the information processing device 1A includes a control unit 10A, a storage unit 20A, an input / output unit 30A, and a communication unit 40A. The input / output unit 30A receives input to the information processing device 1A from an input device (not shown) such as a mouse or a touchpad. The input / output unit 30A also outputs information output from the information processing device 1A to a display device (not shown) such as a liquid crystal display. The input / output unit 30A may be connected to a device in which an input device and an output device are integrally formed, such as a touch panel. The communication unit 40A communicates with other devices via a network.

[0023] The control unit 10A controls each unit of the information processing device 1A. The control unit 10A also includes an acquisition unit 11A, a specification unit 12A, an extraction unit 13A, and a presentation unit 14A. The acquisition unit 11A and the specification unit 12A are configured almost similarly to the acquisition unit 11 and the specification unit 12 in the exemplary embodiment 1, but the details are different. The extraction unit 13A may realize the extraction means described in the claims, but is not limited to this. The presentation unit 14A may realize the presentation means described in the claims, but is not limited to this. Details of each unit included in the control unit 10A will be described in the "Flow of information processing method S1A" described later.

[0024] The storage unit 20A stores various data used by the control unit 10A. For example, the storage unit 20A stores a three-dimensional model MD, coordinate system axis information CAI, measurement point candidates MC, and display data DI. For example, the three-dimensional model MD and the coordinate system axis information CAI are stored in advance in the storage unit 20A. Also, for example, the measurement point candidates MC and the display data DI are generated in the "flow of information processing method S1A" described later.

[0025] (3D model MD) An example of the three-dimensional model MD will be described with reference to FIG. 4. FIG. 4 is a schematic diagram for explaining a specific example of the three-dimensional model MD of the construction machine MV. As shown in FIG. 4, the three-dimensional model MD is data representing the three-dimensional shape of the construction machine MV shown in a three-sided drawing. Note that, hereinafter, for ease of explanation, the three-dimensional shape of the construction machine MV may be described as the solid MV-1 shown in FIG. 4. In this case, the construction machine MV whose three-dimensional shape is the solid MV-1 and its three-dimensional model MD are also described as the construction machine MV-1 and the three-dimensional model MD-1. However, when there is no need to distinguish between these, they will simply be described as the construction machine MV and the three-dimensional model MD. Also, for example, the three-dimensional model MD is represented by CAD (computer aided design) data and point cloud data, but is not limited to these.

[0026] (Coordinate system axis information CAI) The coordinate system axis information CAI is information indicating the relationship between the axes of the on-site coordinate system and the attitude of the construction machine MV. The on-site coordinate system is an orthogonal coordinate system used in measurements using measurement points. The on-site coordinate system is determined according to the layout surface of the site where the construction machine MV is placed. For example, the on-site coordinate system may be an orthogonal coordinate system in which a plane that approximates the layout surface of the site is the XY plane. In this case, the coordinate system axis information CAI may be information that represents the three-dimensional model MD placed on the XY plane using the on-site coordinate system. The three-dimensional model MD is placed on the XY plane so as to represent the construction machine MV in an upright position. The upright position is a state in which the construction machine MV is placed without tilt.

[0027] An example of the coordinate system axis information CAI will be described with reference to Fig. 5. Fig. 5 is a schematic diagram for explaining an example of the coordinate system axis information CAI, which is a top view of the three-dimensional model MD as viewed from the positive direction of the Z axis of the on-site coordinate system. In the coordinate system axis information CAI, the three-dimensional model MD is placed upright on the XY plane. Furthermore, the coordinate system axis information CAI-1 indicates the relationship between the three-dimensional model MD-1 and the on-site coordinate system. In the coordinate system axis information CAI-1, the three-dimensional model MD-1 is placed upright on the XY plane.

[0028] <Flow of information processing method S1A> The information processing device 1A configured as above executes an information processing method S1A according to this exemplary embodiment. The flow of the information processing method S1A will be described with reference to Fig. 6. Fig. 6 is a flow diagram illustrating the flow of the information processing method S1A. As shown in Fig. 6, the information processing method S1A includes steps S11A to S15A.

[0029] (Step S11A) In step S11A, the acquisition unit 11A acquires a three-dimensional model MD of the construction machine MV (moving body). In this exemplary embodiment, the acquisition unit 11A acquires the three-dimensional model MD by reading it from the storage unit 20A. However, the acquisition unit 11A may acquire the three-dimensional model MD via the input / output unit 30A or the communication unit 40A.

[0030] (Step S12A) In step S12A, the identification unit 12A refers to the three-dimensional model MD of the construction machine MV (moving body) to identify multiple candidates for measurement points required for estimating the posture of the construction machine MV. For example, the identification unit 12A may identify candidates for measurement reference line segments required for posture estimation. However, identifying candidates for measurement reference line segments is substantially synonymous with identifying "multiple candidates for measurement points" at least on both ends of the line segment. Also, for example, the identification unit 12A may further refer to the coordinate system axis information CAI to identify pairs of candidates for multiple measurement points arranged parallel to any one of the axes of the site coordinate system.

[0031] (Specific examples of specific processing) A specific example of the identification process for identifying multiple measurement point candidates will be described with reference to Fig. 7. Fig. 7 is a schematic diagram for explaining a specific example of the identification process and the extraction process described below. As shown in Fig. 7, the identification unit 12A identifies measurement point candidates MC (points a1 to a12) as multiple measurement point candidates with reference to the three-dimensional model MD-1. These points a1 to a12 (multiple measurement point candidates) include pairs of measurement point candidates arranged parallel to at least one of the axes of the on-site coordinate system (orthogonal coordinate system) when the construction machine MV-1 (mobile body) is in an upright position in the on-site coordinate system.

[0032] For example, points a1 and a2 are a pair arranged parallel to the X-axis of the site coordinate system. Points a1 and a2 are included in a line segment X1 parallel to the X-axis. In addition, as shown in the figure, the measurement point candidates MC include pairs of measurement point candidates included in line segments X1, X2, Y1, Y2, Z1, and Z2 parallel to any of the X-axis, Y-axis, and Z-axis. In other words, the identification unit 12A identifies line segments X1, X2, Y1, Y2, Z1, and Z2 that are candidates for measurement reference lines parallel to at least one of the axes of the site coordinate system.

[0033] For example, since the line segment X1 is parallel to the X-axis, the y-coordinate and z-coordinate of the measurement coordinates of points a1 and a2 included in the line segment X1 should be equal to each other when the machine is placed upright. Therefore, an estimation process can be performed to estimate the posture of the construction machine MV based on the difference between the measurement coordinates of points a1 and a2. In this way, the measurement coordinates of multiple measurement points included in a measurement reference line segment parallel to at least one axis of the site coordinate system can be used in the estimation process of the posture of the construction machine MV.

[0034] (Step S13A) In step S13A, the acquisition unit 11A acquires information regarding the relative position of the construction machine MV (moving body) with respect to the external measurement device TS. For example, the information regarding the relative position includes information indicating the position and orientation of the construction machine MV with reference to the line of sight direction of the external measurement device TS. Furthermore, the information regarding the relative position includes information regarding the viewing angle of the external measurement device TS. Note that the acquisition unit 11A may acquire the information regarding the relative position with reference to outputs from the external measurement device TS, an external camera (not shown), and the like. Here, the line of sight direction of the external measurement device TS refers to, for example, the direction in which the sensor of the device faces. Furthermore, the information indicating the viewing angle is, for example, information indicating the sensing range of the sensor.

[0035] (Step S14A) In step S14A, the extraction unit 13A refers to the information on the relative position and extracts a plurality of candidates of measurement points that fall within the measurable range of the external measurement device TS from a plurality of candidates of measurement points. For example, the extraction unit 13A refers to information indicating the line of sight and the viewing angle of the external measurement device TS, which is included in the information on the relative position, to specify an area that is the measurable range of the external measurement device TS. The extraction unit 13A also refers to the position and orientation of the construction machine MV based on the line of sight, which is included in the information on the relative position, to narrow down a plurality of candidates of measurement points that fall within the measurable range. Here, the measurable range of the external measurement device TS can also be considered as a range having a spatial spread that can be measured by the optical system (not shown) of the device. The measurable range may be expressed as, for example, "viewing field" or "angle of view". Hereinafter, the measurable range will also be referred to as "viewing field".

[0036] (Example 1 of extraction process) A specific example 1 of the extraction process for extracting a plurality of candidates for measurement points will be described with reference to FIG. 7. In the example shown in FIG. 7, measurement point candidates MC (points a1 to a12) are identified for the construction machine MV-1. In this example, points a1, a2, a3, a5, a6, a7, a9, a11, and a12 are within the field of view of the external measurement device TS. Points a4, a8, and a10 are not within the field of view. Therefore, the extraction unit 13A extracts these points within the field of view, thereby extracting line segments X1-1, X1-2, X2-1, Y1-1, Y2-1, Z1-1, Z2-1, and Z2-2 as candidates for the measurement reference line segment. Line segments X1-1 and X1-2 are parts of line segment X1. Line segment X2-1 is part of line segment X2. Line segment Y1-1 is part of line segment Y1. The line segment Y2-1 is a part of the line segment Y2. The line segment Z1-1 is a part of the line segment Z1. Z 2-2 is a part of the line segment Z2. In other words, the extraction unit 13A refers to the information on the relative position described above and extracts a part or the whole of the measurement reference line segment that falls within the field of view of the external measuring device TS from the candidates for the measurement reference line segment.

[0037] (Specific example of extraction process 2) A second specific example of the extraction process for extracting candidates for a plurality of measurement points will be described with reference to FIG. Fig. 8 is a schematic diagram for explaining a specific example 2 of the extraction process. In this example, measurement point candidates MC (points a21 to a27) are identified for the construction machine MV. In Fig. 8, relative positions pos1, pos2 indicate the relative position of the construction machine MV with respect to the external measuring device TS. Note that, although Fig. 8 shows the relative positions pos1, pos2 two-dimensionally, it is preferable that the information indicating the relative positions pos1, pos2 represents a three-dimensional relative position.

[0038] As shown in FIG. 8, the extraction unit 13A specifies the field of view SR1 based on information about the relative position pos1. The field of view SR1 is a cone-shaped area with the position of the external measuring device TS as the apex. In this case, the extraction unit 13A extracts points a21 to a23 included in the field of view SR1. In other words, the extraction unit 13A extracts a measurement reference line segment including points a21 and a22 included in the field of view SR1, and a measurement reference line segment including points a22 and a23. In addition, the extraction unit 13A does not extract points a24 to a27 that are not included in the field of view SR1. In other words, the extraction unit 13A does not extract a measurement reference line segment including points a24 and a25 that are not included in the field of view SR1, and a measurement reference line segment including points a26 and a27.

[0039] The extraction unit 13A also identifies the field of view SR2 based on information about the relative position pos2. The field of view SR2 is a cone-shaped area with the external measuring device TS as its apex. In this case, the extraction unit 13A extracts points a21 to a25 included in the field of view SR2. In other words, the extraction unit 13A extracts a measurement reference line segment including points a21 and a22 included in the field of view SR2, a measurement reference line segment including points a22 and a23, and a measurement reference line segment including points a24 and a25. The extraction unit 13A also does not extract points a26 to a27 that are not included in the field of view SR2. In other words, the extraction unit 13A does not extract a measurement reference line segment including points a26 and a27 that are not included in the field of view SR2.

[0040] (Step S15A) In step S15A, the presentation unit 14A presents at least one of the multiple measurement point candidates identified by the identification unit 12A and the multiple measurement point candidates extracted by the extraction unit 13A. For example, the presentation unit 14A may present a measurement reference line segment candidate including at least two of the multiple measurement points identified by the identification unit 12A. Also, for example, the presentation unit 14A may present a measurement reference line segment candidate including at least two of the multiple measurement point candidates extracted by the extraction unit 13A. Here, an example will be described in which the presentation unit 14A generates display data DI for displaying multiple measurement point candidates (or multiple measurement reference line segment candidates) on a display device and outputs the display data DI to the display device.

[0041] (Example of display screen) A specific example of a display screen on which the display data DI is presented will be described with reference to FIG.

[0042] Fig. 9 is a diagram showing display screen examples G1 and G2. As shown in Fig. 9, the display data DI presented on the display screen example G1 includes a captured image including the construction machine MV, points a31 and a32, and a line segment X3. The captured image is an image of the construction machine MV in operation. The captured image is captured, for example, by a camera (not shown) arranged so that the construction machine MV in operation is included in the angle of view. The points a31 and a32 indicate candidates for a plurality of measurement points extracted by the extraction unit 13A. The line segment X3 includes the points a31 and a32, and is parallel to the X-axis of the site coordinate system.

[0043] Moreover, the display data DI presented on the display screen example G2 includes a photographed image including the construction machine MV, points a33, a34, a35, and a36, and line segments Y3 and Z3. The points a33, a34, a35, and a36 indicate a plurality of measurement point candidates extracted by the extraction unit 13A. The line segment Y3 includes the points a33 and a34, and is parallel to the Y axis of the site coordinate system. The line segment Z3 includes the points a35 and a36, and is parallel to the Z axis of the site coordinate system.

[0044] In this way, the presentation unit 14A presents a plurality of measurement point candidates (or measurement reference line segments) according to the relative position of the construction machine MV with respect to the external measuring device TS. This concludes the explanation of the information processing method S1A.

[0045] <Advantages of this exemplary embodiment> As described above, according to this exemplary embodiment, in addition to the configuration of exemplary embodiment 1, a configuration is adopted in which information regarding the relative position of the construction machine MV (moving body) with respect to the external measuring device TS is further acquired, and by referring to the information regarding the relative position, a plurality of candidate measuring points that fall within the field of view of the external measuring device TS is extracted from a plurality of candidate measuring points.

[0046] Therefore, according to this exemplary embodiment, multiple candidate measurement points on the construction machine MV (moving body) that should be measured using an external measuring device TS to estimate the posture of the construction machine MV can be accurately identified according to the relative position.

[0047] In addition, in this exemplary embodiment, a configuration is adopted in which at least one of the multiple measurement point candidates identified by the identification unit 12A and the multiple measurement point candidates extracted by the extraction unit 13A is presented.

[0048] Therefore, according to this exemplary embodiment, the user can perform measurements using the external measuring device TS on the presented measurement point candidates.

[0049] In addition, in this exemplary embodiment, the multiple candidate measurement points are configured to include pairs of candidate measurement points that are arranged parallel to at least one of the axes of the site coordinate system (Cartesian coordinate system) when the construction machine MV (moving body) is in an upright position in the site coordinate system.

[0050] Here, if the construction machine MV is in a normal position, the measurement coordinates of each measurement point measured for the pair should be equal to each other in coordinates other than the axis component. By utilizing this, it is possible to perform accurate estimation processing based on the difference in the measurement coordinates of each measurement point. Therefore, by performing measurements using the multiple measurement point candidates presented in this exemplary embodiment, the attitude of the construction machine MV can be estimated with high accuracy. In addition, by performing measurements using the multiple measurement point candidates presented in this exemplary embodiment, the detection value of the attitude sensor mounted on the construction machine MV can be accurately corrected.

[0051] [Variation 1] This exemplary embodiment can be modified to a mode in which a simulation is performed in which the user checks measurement point candidates while changing the relative position of the construction machine MV with respect to the external measuring device TS on the display screen.

[0052] In this modification, the information processing device 1A repeats the processes of steps S13A to S15A after executing steps S11A to S12A.

[0053] In step S13A, the acquisition unit 11A acquires information regarding the relative position in virtual space, instead of acquiring information regarding the relative position of the construction machine MV with respect to the external measuring device TS in real space.

[0054] Moreover, in step S15A, the presentation unit 14A displays display data DI including an image showing a virtual space, instead of displaying display data DI including a photographed image of the construction machine MV. In the virtual space, an object showing the construction machine MV and an object showing the external measurement device TS are arranged. Furthermore, the display data DI includes a first graphical user interface and a second graphical user interface. The first graphical user interface accepts a user input related to the position of the construction machine MV (mobile body). Furthermore, the second graphical user interface accepts a user input related to the position of the external measurement device TS. The user changes the relative position in the virtual space by operating the first graphical user interface and the second graphical user interface.

[0055] (Example of display screen) A specific example of a display screen on which such display data DI is presented will be described with reference to Fig. 10. Fig. 10 is a diagram showing display screen examples G3 and G4. As shown in Fig. 10, the display data DI presented on the display screen example G3 includes an image showing a virtual space SP and GUI objects g1 and g2.

[0056] In the virtual space SP, an object representing a construction machine MV, an object representing a plurality of measurement point candidates MC, and an object representing an external measurement device TS are arranged. The GUI object g1 is an example of a first graphical user interface. The GUI object g2 is an example of a second graphical user interface.

[0057] For example, the control unit 10A updates the position and orientation of the object representing the construction machine MV in the virtual space SP by moving (i.e., dragging) the GUI object g1 while superimposing it on the object representing the construction machine MV based on the user input. Also, for example, the control unit 10A updates the position and orientation of the object representing the external measurement device TS in the virtual space SP by moving (i.e., dragging) the GUI object g2 while superimposing it on the object representing the external measurement device TS based on the user input. This changes the relative position of the construction machine MV in the virtual space SP with respect to the external measurement device TS.

[0058] In this way, according to this modified example, when the user virtually changes the relative position of the external measuring device TS with respect to the construction machine MV, the user can recognize candidates for measuring points that change.

[0059] [Modification 2] In the present exemplary embodiment, the presentation unit 14A has been described as presenting a plurality of candidates for measurement points by outputting the display data DI to a display device. However, the presenter 14A may generate voice data DI indicating at least one of the plurality of candidates for measurement points identified by the identification unit 12A and the plurality of candidates for measurement points extracted by the extraction unit 13A. In this case, the presentation unit 14A may present the plurality of candidates for measurement points by outputting the voice data DI to a voice output device.

[0060] Exemplary embodiment 3 A third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first and second exemplary embodiments are denoted by the same reference numerals, and the description thereof will not be repeated.

[0061] <Overview of information processing device 1B> The information processing device 1B according to this exemplary embodiment is a device that estimates the attitude of the construction machine MV by correcting the detection value of an attitude sensor mounted on the construction machine MV using measurement results measured using an external measuring device TS.

[0062] <Configuration of information processing device 1B> The configuration of the information processing device 1B according to this exemplary embodiment will be described with reference to FIG. 11. FIG. 11 is a block diagram illustrating the configuration of the information processing device 1B. As shown in FIG. 11, the information processing device 1B includes a control unit 10B, a storage unit 20B, an input / output unit 30A, and a communication unit 40A. The input / output unit 30A and the communication unit 40A are as described in the exemplary embodiment 2. In addition, the information processing device 1B is connected to an external measurement device TS so as to be able to communicate with the external measurement device TS. The external measurement device TS is as described in the exemplary embodiment 2.

[0063] The control unit 10B includes an acquisition unit 11A, a specification unit 12A, an extraction unit 13A, a measurement control unit 15B, and an estimation unit 16B. The acquisition unit 11A, the specification unit 12A, and the extraction unit 13A are as described in the exemplary embodiment 2. The measurement control unit 15B controls the external measurement device TS. The measurement control unit 15B may realize the measurement means described in the claims, but is not limited to this. The estimation unit 16B may realize the estimation means described in the claims, but is not limited to this. Details of each unit included in the control unit 10B will be described in the "Flow of information processing method S1B" below.

[0064] The storage unit 20B stores the three-dimensional model MD, the coordinate system axis information CAI, and the measurement point candidates MC. Details of these data are as described in the second exemplary embodiment. In this exemplary embodiment, the storage unit 20B stores the measurement point candidates MC in advance in addition to the three-dimensional model MD and the coordinate system axis information CAI. The measurement point candidates MC are generated by the same processes as steps S11A and S12A in the second exemplary embodiment. In addition, marker parts that can be tracked by the external measuring device TS are installed in advance on the construction machine MV at the multiple measurement point candidates indicated by the measurement point candidates MC.

[0065] <Flow of Information Processing Method S1B> The information processing device 1B configured as above executes an information processing method S1B according to this exemplary embodiment. The flow of the information processing method S1B will be described with reference to Fig. 12. Fig. 12 is a flow diagram illustrating the flow of the information processing method S1B. As shown in Fig. 12, the information processing method S1B includes steps S13A to S14A and S15B to S16B. Steps S13A to S14A are as described in the exemplary embodiment 2.

[0066] (Step S15B) In step S15B, the measurement control unit 15B performs measurement using at least one of the multiple measurement point candidates that fall within the field of view extracted by the extraction unit 13A as a measurement point. The measurement control unit 15B may also determine the measurement point to be measured from the multiple measurement point candidates that fall within the field of view extracted by the extraction unit 13A by referring to the construction machine information related to the construction machine MV.

[0067] For example, the measurement control unit 15B controls the external measurement device TS to measure the coordinates of the measurement points to be measured in the on-site coordinate system. The measurement can be performed using the above-mentioned marker parts installed at multiple measurement point candidates.

[0068] (Step S16B) In step S16B, the estimation unit 16B refers to the measurement results from the measurement control unit 15B and estimates the attitude of the construction machine MV (moving body).

[0069] For example, the estimation process for estimating the attitude may be a process for estimating the attitude of the construction machine MV based on the difference between the measurement coordinates measured for each pair of candidate measurement points. The candidate measurement point pairs are multiple points included in a line segment parallel to any axis in the site coordinate system when the construction machine MV is in an upright position.

[0070] (Specific example of estimation process) Here, a specific example of the estimation process will be described with reference to Fig. 13 and Fig. 14. Fig. 13 is a schematic diagram for explaining an outline of a specific example of the estimation process. Fig. 14 is a diagram for explaining details of the specific example of the estimation process.

[0071] 13, the measurement point candidates MC include points A and B. When the construction machine MV is in an upright position, points A and B are included in a line segment parallel to the Z axis of the site coordinate system. Therefore, the coordinate values ​​(x coordinate and y coordinate) of points A and B other than the Z axis component are equal to each other.

[0072] Here, the posture of the construction equipment MV is represented using a roll angle r, a pitch angle p, and a yaw angle y. For example, the posture when the construction equipment MV is upright is represented as POSE0 (0,0,0) where r, p, and y are zero. The posture of the construction equipment MV to be estimated is represented as POSE1 (r,p,y). The points in POSE1 corresponding to points A and B are represented as points A1 and B1. It is assumed that measurement coordinates (ax1, ay1, az1) and measurement coordinates (bx1, by1, bz1) have been acquired for points A1 and B1.

[0073] The estimation process is a process for estimating POSE1 from the measured coordinates of points A1 and B1. As the posture of the construction machine MV changes from POSE0 to POSE1, the line segment AB rotates and changes to the line segment A1-B1. When the line segment A1-B1 rotates in the opposite direction, it returns to the line segment AB.

[0074] Here, the rotation matrix R representing the rotation from POSE0 to POSE1 (r, p, y) can be expressed as in the following equation (1).

[0075]

number

[0076] For example, the coordinates of points A1 and B1 can be considered to be calculated by multiplying the coordinates of points A and B by the rotation matrix R, respectively. Therefore, points A2 and B2 obtained by multiplying points A1 and B1 by the inverse matrix of rotation matrix R should be included in a line segment parallel to the Z axis, just like the original points A and B. Therefore, POSE1(r, p, y) can be estimated by finding a rotation matrix R that causes points A2 and B2 to be included in a line segment parallel to the Z axis.

[0077] As shown in Fig. 14, the coordinates of points A2 and B2 obtained by rotating the line segment A1-B1 inversely are expressed as (ax2, ay2, az2) and (bx2, by2, bz2). As described above, rotating inversely is equivalent to multiplying by the inverse matrix of the rotation matrix R. For example, the coordinates of point A2 can be calculated by the following formula (2).

[0078]

number

[0079] Similarly, assume that a pair of points C and D included in a line segment parallel to the Y axis is specified. Also assume that the measurement coordinates of points C1 and D1 measured for each are (cx1, cy1, cz1) and (dx1, dy1, dz1). In this case, the coordinates of points C2 and D2 obtained by rotating the line segment C1-D1 in the reverse direction are expressed as (cx2, cy2, cz2) and (dx2, dy2, dz2). At this time, the error of the pair is calculated based on the difference between cx2 and dx2, which are coordinate values ​​other than the Y component, and the difference between cz2 and dz2. If points C2 and D2 are included in a line segment parallel to the Y axis, the error of the pair should be zero.

[0080] Similarly, assume that a pair of points E and F included in a line segment parallel to the X-axis is specified. Also assume that the measured coordinates of points E1 and F1 measured for each are (ex1, ey1, ez1) and (fx1, fy1, fz1). In this case, the coordinates of points E2 and F2 obtained by rotating the line segment E1-F1 in the reverse direction are expressed as (ex2, ey2, ez2) and (fx2, fy2, fz2). At this time, the error of the pair is calculated based on the difference between ey2 and fy2, which are coordinate values ​​other than the X component, and the difference between ez2 and fz2. If points E2 and F2 are included in a line segment parallel to the X-axis, the error of the pair should be zero.

[0081] Therefore, POSE1 can be estimated by finding a rotation matrix R that minimizes the sum of the errors of each pair.

[0082] Furthermore, the information processing device 1B may repeatedly execute the information processing method S1B. This makes it possible to estimate changes in the attitude of the construction machine MV in real time. This concludes the explanation of the information processing method S1B.

[0083] <Advantages of this exemplary embodiment> As described above, according to this exemplary embodiment, in addition to the configuration related to exemplary embodiment 2, a configuration is adopted in which an external measuring device TS is used to perform measurements using at least one of multiple candidate measurement points within the field of view as a measurement point.

[0084] Therefore, measurements can be performed on measurement points that should be used to accurately estimate the attitude of the construction machine MV with reduced effort on the part of the user.

[0085] Furthermore, according to this exemplary embodiment, a configuration is adopted in which the attitude of the construction machine MV is estimated by referring to the measurement results from the measurement control unit 15B.

[0086] This reduces the burden on the user while allowing measurements to be taken at more appropriate measurement points, enabling the posture of the construction machine MV to be estimated accurately in real time.

[0087] [Modification 3] Furthermore, in the third exemplary embodiment, the information processing device 1B may further have a function of executing the information processing method S1A according to the second exemplary embodiment, in addition to a function of executing the information processing method S1B.

[0088] [Modification 4] In addition, in the exemplary embodiments 2 and 3, the construction machine MV is used as an example of a moving body according to each exemplary embodiment of the present application, but other moving bodies may be used. Specific examples of such moving bodies may be, for example, robots, people, etc., but are not limited to these as long as the moving body can change its posture.

[0089] [Software implementation example] A part or all of the functions of the information processing devices 1, 1A, and 1B may be realized by hardware such as an integrated circuit (IC chip), or may be realized by software.

[0090] In the latter case, the information processing devices 1, 1A, and 1B are realized, for example, by a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter, referred to as computer C) is shown in FIG. 15. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the information processing devices 1, 1A, and 1B. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the information processing devices 1, 1A, and 1B.

[0091] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination of these. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these.

[0092] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may further include a communication interface for transmitting and receiving data to and from other devices. The computer C may further include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0093] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can obtain the program P via such a recording medium M. Furthermore, the program P can be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also obtain the program P via such a transmission medium.

[0094] [Additional Note 1] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiment are also included in the technical scope of the present invention.

[0095] [Additional Note 2] Some or all of the above-described embodiments can be described as follows. However, the present invention is not limited to the following described aspects.

[0096] (Appendix 1) An acquisition means for acquiring a three-dimensional model of a moving object; a specifying means for specifying candidates of a plurality of measurement points required for estimating the posture of the moving object by referring to the three-dimensional model; An information processing device comprising:

[0097] (Appendix 2) the acquiring means further acquires information regarding a relative position of the moving object with respect to an external measuring device; The present invention further includes an extraction unit that refers to information about the relative positions and extracts, from the plurality of measurement point candidates, a plurality of measurement point candidates that fall within a measurable range of the external measurement device. 2. An information processing device according to claim 1.

[0098] (Appendix 3) The measuring device further includes a measuring means for performing a measurement using at least one of a plurality of measurement point candidates within the measurable range extracted by the extracting means as a measurement point. 3. An information processing device according to claim 2.

[0099] (Appendix 4) The vehicle further includes an estimation unit that estimates the attitude of the moving object by referring to the measurement result by the measurement unit. 4. The information processing device according to claim 3.

[0100] (Appendix 5) A plurality of measurement point candidates identified by the identification means; and The present invention further includes a presentation unit that presents at least one of the plurality of measurement point candidates extracted by the extraction unit. 5. An information processing device according to any one of claims 2 to 4.

[0101] (Appendix 6) The display screen presented by the presentation means includes: a first graphical user interface for accepting user input regarding a location of the mobile object; and a second graphical user interface for accepting user input regarding the location of the external metrology device; Including, The acquiring means acquires information about the relative position by referring to a user input via the first graphical user interface and a user input via the second graphical user interface. 6. The information processing device according to claim 5.

[0102] (Appendix 7) The plurality of measurement point candidates identified by the identifying means include a pair of measurement point candidates arranged parallel to at least one of the axes of a Cartesian coordinate system when the moving body is in a normal position in the Cartesian coordinate system. 5. An information processing device according to any one of claims 1 to 4.

[0103] (Appendix 8) Obtaining a three-dimensional model of the moving object; Identifying a plurality of candidates for measurement points required for estimating the posture of the moving object by referring to the three-dimensional model; An information processing method comprising:

[0104] (Appendix 9) A recording medium having a program recorded thereon for causing a computer to function as an information processing device, the computer comprising: An acquisition means for acquiring a three-dimensional model of a moving object; a specifying means for specifying candidates of a plurality of measurement points required for estimating the posture of the moving object by referring to the three-dimensional model; A recording medium on which a program that functions as a

[0105] [Additional Note 3] A part or all of the above-described embodiments can be further expressed as follows.

[0106] An information processing device comprising at least one processor that executes an acquisition process for acquiring a three-dimensional model of a moving body, and an identification process for identifying candidates for multiple measurement points required for estimating the posture of the moving body by referring to the three-dimensional model. The information processing device may further include a memory that stores a program for causing the processor to execute the acquisition process and the identification process. The program may be recorded in a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]

[0107] 1, 1A, 1B Information processing device 10A, 10B Control section 11, 11A Acquisition Department 12, 12A specific part 13A Extraction part 14A Presentation section 15B Measurement control section 16B Estimation part 20A, 20B storage section 30A input / output section 40A Communication section TS External measurement device C1 Processor C2 Memory

Claims

1. An acquisition means for acquiring a three-dimensional model of a moving object; a specifying means for specifying candidates of a plurality of measurement points required for estimating the posture of the moving object by referring to the three-dimensional model; An information processing device comprising:

2. the acquiring means further acquires information regarding a relative position of the moving object with respect to an external measuring device; The present invention further includes an extraction unit that refers to information about the relative positions and extracts, from the plurality of measurement point candidates, a plurality of measurement point candidates that fall within a measurable range of the external measurement device. The information processing device according to claim 1 .

3. The measuring device further includes a measuring means for performing a measurement using at least one of a plurality of measurement point candidates within the measurable range extracted by the extracting means as a measurement point. The information processing device according to claim 2 .

4. The vehicle further includes an estimation unit that estimates the attitude of the moving object by referring to the measurement result by the measurement unit. The information processing device according to claim 3 .

5. A plurality of measurement point candidates identified by the identification means; and The present invention further includes a presentation unit that presents at least one of the plurality of measurement point candidates extracted by the extraction unit. The information processing device according to claim 2 .

6. The display screen presented by the presentation means includes: a first graphical user interface for accepting user input regarding a location of the mobile object; and A second graphical user interface that accepts user input regarding the location of the external metrology device. Including, The acquiring means acquires information about the relative position by referring to a user input via the first graphical user interface and a user input via the second graphical user interface. The information processing device according to claim 5 .

7. The plurality of measurement point candidates identified by the identifying means include a pair of measurement point candidates arranged parallel to at least one of the axes of a Cartesian coordinate system when the moving body is in a normal position in the Cartesian coordinate system. The information processing device according to claim 1 .

8. Obtaining a three-dimensional model of the moving object; Identifying candidates for a plurality of measurement points required for estimating the posture of the moving object by referring to the three-dimensional model; An information processing method comprising:

9. A program for causing a computer to function as an information processing device, the program comprising: An acquisition means for acquiring a three-dimensional model of a moving object; a specifying means for specifying candidates of a plurality of measurement points required for estimating the posture of the moving object by referring to the three-dimensional model; A program that functions as a

Images