Information processing device, information processing method, and program

Abstract

In order to resolve the problem of more accurately estimating the orientation of a mobile body in a three-dimensional space, this information processing device (1) comprises an acquiring unit (11) that acquires a three-dimensional model for a mobile body, and an identifying unit (12) that refers to the three-dimensional model and identifies candidates for a plurality of measurement points necessary to estimate the orientation of the mobile body.

Description

[Technical Field]

[0001] The present invention relates to a technique for estimating the attitude of a moving object. [Background technology]

[0002] Patent Document 1 describes a technique for estimating the attitude of a moving body using parameters detected by a sensor provided on the moving body. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2021-21673 Summary of the Invention [Problem to be solved by the invention]

[0004] When the contact surface of a moving body is not a two-dimensional plane, such as when it includes a slope or a depression, it is necessary to accurately estimate the attitude of the moving body in three-dimensional space. With the technology described in Patent Document 1, for example, if a six-axis sensor is installed on the moving body, it is difficult to accurately estimate the attitude in three-dimensional space because the yaw angle cannot be detected with high accuracy. Furthermore, for example, if a nine-axis sensor is installed on the moving body, it becomes necessary to measure the difference between the earth's axis and the on-site coordinate system, which makes the process of accurately estimating the attitude in three-dimensional space complicated.

[0005] One aspect of the present invention has been made in consideration of the above-mentioned problems, and aims to provide a technique for more accurately estimating the posture of a moving body in three-dimensional space. [Means for solving the problem]

[0006] An information processing device according to one aspect of the present invention includes an acquisition means for acquiring a three-dimensional model of a moving body, and an identification means for identifying, by referring to the three-dimensional model, multiple candidate measurement points required for estimating the posture of the moving body.

[0007] An information processing method according to one aspect of the present invention includes acquiring a three-dimensional model of a moving body, and referring to the three-dimensional model, identifying a plurality of candidate measurement points required for estimating the posture of the moving body.

[0008] According to one aspect of the present invention program is a program that makes a computer function as an information processing device. In The computer is caused to function as an acquisition means for acquiring a three-dimensional model of a moving body, and as an identification means for identifying candidates of a plurality of measurement points required for estimating the posture of the moving body by referring to the three-dimensional model. do. In addition, the program Recording media that records This also falls within the scope of one aspect of the present invention. [Effects of the Invention]

[0009] According to one aspect of the present invention, the posture of a moving body in three-dimensional space can be estimated with higher accuracy. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to a first exemplary embodiment. [Figure 2] 1 is a flowchart showing the flow of an information processing method according to the first exemplary embodiment. [Figure 3] FIG. 10 is a block diagram illustrating the configuration of an information processing device according to a second exemplary embodiment. [Figure 4] 10A and 10B are schematic diagrams illustrating a specific example of a three-dimensional model in the second exemplary embodiment. [Figure 5] 10 is a schematic diagram illustrating an example of coordinate system axis information in the second exemplary embodiment. FIG. [Figure 6] FIG. 10 is a flow chart illustrating the flow of an information processing method according to the second exemplary embodiment. [Figure 7] 10A to 10C are schematic diagrams illustrating specific examples of the identification process and extraction process in the second exemplary embodiment. [Figure 8]10A and 10B are schematic diagrams illustrating a specific example of an extraction process in the second exemplary embodiment. [Figure 9] FIG. 10 is a diagram showing a specific example of display data in the second exemplary embodiment. [Figure 10] FIG. 10 is a diagram showing a specific example of a display screen in the second exemplary embodiment. [Figure 11] FIG. 10 is a block diagram illustrating the configuration of an information processing device according to a third exemplary embodiment. [Figure 12] FIG. 10 is a flow chart illustrating the flow of an information processing method according to the third exemplary embodiment. [Figure 13] 10 is a schematic diagram for explaining an outline of a specific example of an estimation process in exemplary embodiment 3. FIG. [Figure 14] 10A to 10C are diagrams illustrating details of a specific example of an estimation process in the third exemplary embodiment. [Figure 15] FIG. 2 is a block diagram illustrating an example of a hardware configuration of an information processing device according to each exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Exemplary Embodiment 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described below.

[0012] <Configuration of information processing device 1> The configuration of an information processing device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1.

[0013] 1, the information processing device 1 includes an acquisition unit 11 and an identification unit 12. The acquisition unit 11 acquires a three-dimensional model of a moving object. The identification unit 12 refers to the three-dimensional model and identifies candidates for a plurality of measurement points required for estimating the posture of the moving object.

[0014] <Flow of information processing method S1> The information processing device 1 configured as above executes an information processing method S1 according to this exemplary embodiment. The flow of the information processing method S1 will be described with reference to Fig. 2. Fig. 2 is a flow diagram illustrating the flow of the information processing method S1.

[0015] 2, information processing method S1 includes steps S11 to S12. In step S11, acquisition unit 11 acquires a three-dimensional model of a moving object. In step S12, identification unit 12 refers to the three-dimensional model and identifies candidates for a plurality of measurement points required for estimating the posture of the moving object.

[0016] <Example of program implementation> When the information processing device 1 is configured by a computer, the following program is stored in a memory (recording medium) referenced by the computer. The program is a program for causing the computer to function as the information processing device 1, and causes the computer to function as an acquisition unit 11 that acquires a three-dimensional model of a moving body, and an identification unit 12 that refers to the three-dimensional model and identifies multiple measurement point candidates required for estimating the posture of the moving body.

[0017] The computer reads the program from memory and executes it, thereby realizing the information processing method S1 described above.

[0018] <Advantages of this exemplary embodiment> As described above, this exemplary embodiment employs a configuration in which a three-dimensional model of a moving body is acquired, and a plurality of candidate measurement points required for estimating the posture of the moving body are identified by referring to the three-dimensional model. These candidate measurement points can be used to more accurately estimate the posture of the moving body in three-dimensional space. This allows for more accurate estimation of the posture of the moving body in three-dimensional space.

[0019] Exemplary Embodiment 2 A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and their description will be omitted as appropriate.

[0020] <Overview of information processing device 1A> The information processing device 1A according to this exemplary embodiment is a device that presents measurement points to be measured by an external measurement device TS in order to estimate the attitude of a construction machine MV. The construction machine MV is an example of a moving body according to each exemplary embodiment of the present application.

[0021] For example, the user operates the external measurement device TS to perform measurements on the measurement points presented by the information processing device 1A. In addition, if the construction machine MV is equipped with an attitude sensor, the user corrects the measurement value of the attitude sensor using the measurement results from the external measurement device TS. The attitude sensor is, for example, a six-axis sensor (IMU: Inertial Measurement Unit) that measures three-dimensional angular velocity and acceleration. Inertial The external measurement device TS may be, but is not limited to, a total station that measures the distance and angle to a measurement point.

[0022] <Configuration of information processing device 1A> The configuration of an information processing device 1A according to this exemplary embodiment will be described with reference to FIG. 3. FIG. 3 is a block diagram illustrating the configuration of the information processing device 1A. As shown in FIG. 3, the information processing device 1A includes a control unit 10A, a storage unit 20A, an input / output unit 30A, and a communication unit 40A. The input / output unit 30A receives input to the information processing device 1A from an input device (not shown) such as a mouse or a touchpad. The input / output unit 30A also outputs information output from the information processing device 1A to a display device (not shown) such as a liquid crystal display. The input / output unit 30A may be connected to a device in which an input device and an output device are integrated, such as a touch panel. The communication unit 40A communicates with other devices via a network.

[0023] The control unit 10A controls each unit of the information processing device 1A in an integrated manner. The control unit 10A also includes an acquisition unit 11A, an identification unit 12A, an extraction unit 13A, and a presentation unit 14A. The acquisition unit 11A and the identification unit 12A are configured substantially similarly to the acquisition unit 11 and the identification unit 12 in exemplary embodiment 1, but the details are different. The extraction unit 13A may realize an extraction means recited in the claims, but is not limited to this. The presentation unit 14A may realize a presentation means recited in the claims, but is not limited to this. Details of each unit included in the control unit 10A will be described in the "Flow of information processing method S1A" section below.

[0024] The storage unit 20A stores various data used by the control unit 10A. For example, the storage unit 20A stores a three-dimensional model MD, coordinate system axis information CAI, measurement point candidates MC, and display data DI. For example, the three-dimensional model MD and the coordinate system axis information CAI are stored in advance in the storage unit 20A. Also, for example, the measurement point candidates MC and the display data DI are generated in the "flow of information processing method S1A" described below.

[0025] (3D model MD) An example of the three-dimensional model MD will be described with reference to FIG. 4. FIG. 4 is a schematic diagram illustrating a specific example of the three-dimensional model MD of a construction machine MV. As shown in FIG. 4, the three-dimensional model MD is data representing the three-dimensional shape of the construction machine MV shown in a three-view drawing. Note that, hereinafter, for simplicity of explanation, the three-dimensional shape of the construction machine MV may be described as the three-dimensional MV-1 shown in FIG. 4. In this case, the construction machine MV whose three-dimensional shape is the three-dimensional MV-1 and its three-dimensional model MD will also be referred to as the construction machine MV-1 and the three-dimensional model MD-1. However, when there is no need to distinguish between them, they will simply be referred to as the construction machine MV and the three-dimensional model MD. Furthermore, for example, the three-dimensional model MD is represented by CAD (computer-aided design) data or point cloud data, but is not limited to these.

[0026] (Coordinate system axis information CAI) The coordinate system axis information CAI is information that indicates the relationship between the axes of the site coordinate system and the attitude of the construction machine MV. The site coordinate system is a Cartesian coordinate system used in measurements using measurement points. The site coordinate system is determined according to the layout surface of the site where the construction machine MV is placed. For example, the site coordinate system may be a Cartesian coordinate system in which a plane that approximates the layout surface of the site is the XY plane. In this case, the coordinate system axis information CAI may be information that represents the three-dimensional model MD placed on the XY plane using the site coordinate system. The three-dimensional model MD is placed on the XY plane so as to represent the construction machine MV in an upright position. The upright position is a state in which the construction machine MV is placed without tilt.

[0027] An example of the coordinate system axis information CAI will be described with reference to Fig. 5. Fig. 5 is a schematic diagram illustrating an example of the coordinate system axis information CAI, and is a top view of the three-dimensional model MD as viewed from the positive direction of the Z axis of the site coordinate system. In this coordinate system axis information CAI, the three-dimensional model MD is placed upright on the XY plane. Furthermore, coordinate system axis information CAI-1 indicates the relationship between the three-dimensional model MD-1 and the site coordinate system. In this coordinate system axis information CAI-1, the three-dimensional model MD-1 is placed upright on the XY plane.

[0028] <Flow of information processing method S1A> The information processing device 1A configured as above executes an information processing method S1A according to this exemplary embodiment. The flow of the information processing method S1A will be described with reference to Fig. 6. Fig. 6 is a flow diagram illustrating the flow of the information processing method S1A. As shown in Fig. 6, the information processing method S1A includes steps S11A to S15A.

[0029] (Step S11A) In step S11A, the acquisition unit 11A acquires a three-dimensional model MD of the construction machine MV (mobile body). In this exemplary embodiment, the acquisition unit 11A acquires the three-dimensional model MD by reading it from the storage unit 20A. However, the acquisition unit 11A may also acquire the three-dimensional model MD via the input / output unit 30A or the communication unit 40A.

[0030] (Step S12A) In step S12A, the identification unit 12A refers to the three-dimensional model MD of the construction machine MV (mobile body) to identify multiple candidate measurement points required for estimating the posture of the construction machine MV. For example, the identification unit 12A may identify candidates for measurement reference line segments required for posture estimation. However, identifying candidates for measurement reference line segments is substantially synonymous with identifying "multiple candidate measurement points" at least on both ends of the line segment. Also, for example, the identification unit 12A may further refer to the coordinate system axis information CAI to identify pairs of candidate measurement points arranged parallel to any axis of the site coordinate system.

[0031] (Specific examples of specific processing) A specific example of the identification process for identifying multiple measurement point candidates will be described with reference to Fig. 7. Fig. 7 is a schematic diagram for explaining a specific example of the identification process and the extraction process described below. As shown in Fig. 7, the identification unit 12A refers to the three-dimensional model MD-1 and identifies measurement point candidates MC (points a1 to a12) as multiple measurement point candidates. These points a1 to a12 (multiple measurement point candidates) include pairs of measurement point candidates that are arranged parallel to at least one of the axes of the site coordinate system (Cartesian coordinate system) when the construction machine MV-1 (mobile body) is in an upright position in the site coordinate system.

[0032] For example, points a1 and a2 are a pair arranged parallel to the X-axis of the on-site coordinate system. Points a1 and a2 are included in line segment X1, which is parallel to the X-axis. In addition, as shown in the figure, measurement point candidates MC include pairs of measurement point candidates included in line segments X1, X2, Y1, Y2, Z1, and Z2, which are parallel to any of the X-axis, Y-axis, and Z-axis. In other words, the identification unit 12A identifies line segments X1, X2, Y1, Y2, Z1, and Z2, which are candidates for measurement reference line segments parallel to at least one of the axes of the on-site coordinate system.

[0033] For example, because line segment X1 is parallel to the X-axis, the y-coordinates and z-coordinates of the measurement coordinates of points a1 and a2 included in line segment X1 should be equal to each other when the machine is in the upright position. Therefore, an estimation process can be performed to estimate the attitude of the construction machine MV based on the difference between the measurement coordinates of points a1 and a2. In this way, the measurement coordinates of multiple measurement points included in a measurement reference line segment that is parallel to at least one axis of the site coordinate system can be used in the estimation process of the attitude of the construction machine MV.

[0034] (Step S13A) In step S13A, the acquisition unit 11A acquires information about the relative position of the construction machine MV (mobile body) with respect to the external measurement device TS. For example, the information about the relative position includes information indicating the position and orientation of the construction machine MV with reference to the line of sight direction of the external measurement device TS. The information about the relative position also includes information about the field of view angle of the external measurement device TS. Note that the acquisition unit 11A may acquire the information about the relative position by referring to outputs from the external measurement device TS, an external camera (not shown), etc. Here, the line of sight direction of the external measurement device TS refers, for example, to the direction in which the sensor of the device is facing. Furthermore, the information indicating the field of view angle is, for example, information indicating the sensing range of the sensor.

[0035] (Step S14A) In step S14A, the extraction unit 13A references information about the relative position and extracts, from among multiple candidate measurement points, multiple candidate measurement points that fall within the measurable range of the external measurement device TS. For example, the extraction unit 13A references information about the relative position that indicates the line-of-sight direction and field of view angle of the external measurement device TS to identify an area that is within the measurable range of the external measurement device TS. The extraction unit 13A also references the position and orientation of the construction machine MV relative to the line-of-sight direction, which are included in the information about the relative position, to narrow down the multiple candidate measurement points that fall within the measurable range. Here, the measurable range of the external measurement device TS can also be considered as a range having a spatial extent that can be measured by the optical system (not shown) of the device. The measurable range may also be expressed as, for example, "field of view" or "angle of view." Hereinafter, the measurable range will also be referred to as "field of view."

[0036] (Example 1 of extraction processing) A first specific example of extraction processing for extracting multiple measurement point candidates will be described with reference to FIG. 7. In the example shown in FIG. 7, measurement point candidates MC (points a1 to a12) have been identified for construction machine MV-1. In this example, points a1, a2, a3, a5, a6, a7, a9, a11, and a12 fall within the field of view of external measurement device TS. Points a4, a8, and a10 do not fall within the field of view. Therefore, the extraction unit 13A extracts these points that fall within the field of view, thereby extracting line segments X1-1, X1-2, X2-1, Y1-1, Y2-1, Z1-1, Z2-1, and Z2-2 as candidates for the measurement reference line segment. Line segments X1-1 and X1-2 are part of line segment X1. Line segment X2-1 is part of line segment X2. Line segment Y1-1 is part of line segment Y1. The line segment Y2-1 is a part of the line segment Y2. The line segment Z1-1 is a part of the line segment Z1. The line segment Z2-1, Z 2-2 is a part of the line segment Z2. In other words, the extraction unit 13A refers to the information on the relative position described above and extracts a part or all of the candidates for the measurement reference line segment that falls within the field of view of the external measuring device TS.

[0037] (Example of extraction process 2) A second specific example of the extraction process for extracting candidates for a plurality of measurement points will be described with reference to FIG. FIG. 8 is a schematic diagram illustrating a specific example 2 of the extraction process. In this example, measurement point candidates MC (points a21 to a27) are identified for the construction machine MV. In FIG. 8, relative positions pos1 and pos2 indicate the relative positions of the construction machine MV with respect to the external measuring device TS. Note that while FIG. 8 shows the relative positions pos1 and pos2 two-dimensionally, it is desirable that the information indicating the relative positions pos1 and pos2 represent three-dimensional relative positions.

[0038] As shown in FIG. 8, the extraction unit 13A identifies the field of view SR1 based on information about the relative position pos1. The field of view SR1 is a cone-shaped region with the position of the external measurement device TS as its apex. In this case, the extraction unit 13A extracts points a21 to a23 included in the field of view SR1. In other words, the extraction unit 13A extracts a measurement reference line segment including points a21 and a22, which are included in the field of view SR1, and a measurement reference line segment including points a22 and a23. Furthermore, the extraction unit 13A does not extract points a24 to a27, which are not included in the field of view SR1. In other words, the extraction unit 13A does not extract a measurement reference line segment including points a24 and a25, which are not included in the field of view SR1, and a measurement reference line segment including points a26 and a27.

[0039] The extraction unit 13A also identifies the field of view SR2 based on information about the relative position pos2. The field of view SR2 is a conical region with the external measurement device TS at its apex. In this case, the extraction unit 13A extracts points a21 to a25 that are included in the field of view SR2. In other words, the extraction unit 13A extracts a measurement reference line segment that includes points a21 and a22, a measurement reference line segment that includes points a22 and a23, and a measurement reference line segment that includes points a24 and a25 that are included in the field of view SR2. The extraction unit 13A also does not extract points a26 to a27 that are not included in the field of view SR2. In other words, the extraction unit 13A does not extract a measurement reference line segment that includes points a26 and a27 that are not included in the field of view SR2.

[0040] (Step S15A) In step S15A, the presentation unit 14A presents at least one of the multiple measurement point candidates identified by the identification unit 12A and the multiple measurement point candidates extracted by the extraction unit 13A. For example, the presentation unit 14A may present a measurement reference line segment candidate that includes at least two of the multiple measurement points identified by the identification unit 12A. Furthermore, for example, the presentation unit 14A may present a measurement reference line segment candidate that includes at least two of the multiple measurement point candidates extracted by the extraction unit 13A. Here, an example will be described in which the presentation unit 14A generates display data DI for displaying the multiple measurement point candidates (or the multiple measurement reference line segment candidates) on a display device and outputs the display data DI to the display device.

[0041] (Example of display screen) A specific example of a display screen on which the display data DI is presented will be described with reference to FIG.

[0042] FIG. 9 shows display screen examples G1 and G2. As shown in FIG. 9, the display data DI presented on display screen example G1 includes a captured image including the construction machine MV, points a31 and a32, and a line segment X3. The captured image is an image of the construction machine MV in operation. The captured image is captured, for example, by a camera (not shown) positioned so that the construction machine MV in operation is included in the angle of view. Points a31 and a32 indicate multiple measurement point candidates extracted by extraction unit 13A. Line segment X3 includes points a31 and a32 and is parallel to the X-axis of the site coordinate system.

[0043] Furthermore, the display data DI presented on the display screen example G2 includes a captured image including the construction machine MV, points a33, a34, a35, and a36, and line segments Y3 and Z3. Points a33, a34, a35, and a36 indicate multiple measurement point candidates extracted by the extraction unit 13A. Line segment Y3 includes points a33 and a34 and is parallel to the Y axis of the site coordinate system. Line segment Z3 includes points a35 and a36 and is parallel to the Z axis of the site coordinate system.

[0044] In this way, the presentation unit 14A presents a plurality of measurement point candidates (or measurement reference line segments) according to the relative position of the construction machine MV with respect to the external measuring device TS. This concludes the description of the information processing method S1A.

[0045] <Advantages of this exemplary embodiment> As described above, according to this exemplary embodiment, in addition to the configuration of exemplary embodiment 1, a configuration is adopted in which information regarding the relative position of the construction machine MV (moving body) with respect to the external measuring device TS is further acquired, and by referring to the information regarding the relative position, multiple candidate measuring points that fall within the field of view of the external measuring device TS are extracted from multiple candidate measuring points.

[0046] Therefore, according to this exemplary embodiment, multiple candidate measurement points on the construction machine MV (moving body) that should be measured using an external measuring device TS to estimate the attitude of the construction machine MV can be accurately identified according to their relative positions.

[0047] Furthermore, in this exemplary embodiment, a configuration is adopted in which at least one of the multiple measurement point candidates identified by the identification unit 12A and the multiple measurement point candidates extracted by the extraction unit 13A is presented.

[0048] Therefore, according to this exemplary embodiment, the user can perform measurements using the external measurement device TS on the presented measurement point candidates.

[0049] In addition, in this exemplary embodiment, the multiple candidate measurement points are configured to include pairs of candidate measurement points that are arranged parallel to at least one axis of the site coordinate system (Cartesian coordinate system) when the construction machine MV (moving body) is in an upright position in the site coordinate system.

[0050] Here, if the construction machine MV is in an upright position, the measurement coordinates of each measurement point measured for that pair should be equal to each other in coordinates other than the relevant axis component. Utilizing this, it is possible to perform accurate estimation processing based on the difference in the measurement coordinates of each measurement point. Therefore, by performing measurements using the multiple candidate measurement points presented in this exemplary embodiment, it is possible to accurately estimate the attitude of the construction machine MV. Furthermore, by performing measurements using the multiple candidate measurement points presented in this exemplary embodiment, it is possible to accurately correct the detection value of the attitude sensor mounted on the construction machine MV.

[0051] [Variation 1] This exemplary embodiment can be modified to a mode in which a simulation is performed in which the user checks measurement point candidates while changing the relative position of the construction machine MV with respect to the external measuring device TS on the display screen.

[0052] In this modification, the information processing device 1A executes steps S11A to S12A, and then repeats the processes of steps S13A to S15A.

[0053] In step S13A, the acquisition unit 11A acquires information relating to the relative position in virtual space, instead of acquiring information relating to the relative position of the construction machine MV with respect to the external measuring device TS in real space.

[0054] Furthermore, in step S15A, the presentation unit 14A displays display data DI including an image showing a virtual space, instead of displaying display data DI including a photographed image of the construction machine MV. An object showing the construction machine MV and an object showing the external measuring device TS are arranged in the virtual space. The display data DI also includes a first graphical user interface and a second graphical user interface. The first graphical user interface accepts user input related to the position of the construction machine MV (mobile body). The second graphical user interface accepts user input related to the position of the external measuring device TS. The user changes the relative position in the virtual space by operating the first graphical user interface and the second graphical user interface.

[0055] (Example of display screen) A specific example of a display screen on which such display data DI is presented will be described with reference to Fig. 10. Fig. 10 is a diagram showing display screen examples G3 and G4. As shown in Fig. 10, the display data DI presented on display screen example G3 includes an image showing a virtual space SP and GUI objects g1 and g2.

[0056] In the virtual space SP, an object representing a construction machine MV, an object representing multiple measurement point candidates MC, and an object representing an external measurement device TS are arranged. Furthermore, the GUI object g1 is an example of a first graphical user interface. The GUI object g2 is an example of a second graphical user interface.

[0057] For example, the control unit 10A updates the position and orientation of the object representing the construction machine MV in the virtual space SP by moving (i.e., dragging) the GUI object g1 while superimposing it on the object representing the construction machine MV based on user input. Also, for example, the control unit 10A updates the position and orientation of the object representing the external measurement device TS in the virtual space SP by moving (i.e., dragging) the GUI object g2 while superimposing it on the object representing the external measurement device TS based on user input. This changes the relative position of the construction machine MV in the virtual space SP with respect to the external measurement device TS.

[0058] In this way, according to this modification, the user can recognize candidates for measurement points that change when the relative position of the external measurement device TS with respect to the construction machine MV is virtually changed.

[0059] [Variation 2] In the present exemplary embodiment, the presentation unit 14A has been described as presenting a plurality of measurement point candidates by outputting the display data DI to a display device. However, the presenting unit 14A may generate audio data DI indicating at least one of the plurality of measurement point candidates identified by the identifying unit 12A and the plurality of measurement point candidates extracted by the extracting unit 13A. In this case, the presentation unit 14A may present the plurality of measurement point candidates by outputting the audio data DI to an audio output device.

[0060] Exemplary Embodiment 3 A third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first and second exemplary embodiments are denoted by the same reference numerals, and their description will not be repeated.

[0061] <Outline of information processing device 1B> The information processing device 1B according to this exemplary embodiment is a device that estimates the attitude of a construction machine MV by correcting the detection value of an attitude sensor mounted on the construction machine MV using measurement results measured using an external measuring device TS.

[0062] <Configuration of information processing device 1B> The configuration of an information processing device 1B according to this exemplary embodiment will be described with reference to FIG. 11. FIG. 11 is a block diagram illustrating the configuration of the information processing device 1B. As shown in FIG. 11, the information processing device 1B includes a control unit 10B, a storage unit 20B, an input / output unit 30A, and a communication unit 40A. The input / output unit 30A and the communication unit 40A are as described in the exemplary embodiment 2. The information processing device 1B is also communicatively connected to an external measurement device TS. The external measurement device TS is as described in the exemplary embodiment 2.

[0063] The control unit 10B includes an acquisition unit 11A, an identification unit 12A, an extraction unit 13A, a measurement control unit 15B, and an estimation unit 16B. The acquisition unit 11A, the identification unit 12A, and the extraction unit 13A are as described in the second exemplary embodiment. The measurement control unit 15B controls the external measurement device TS. The measurement control unit 15B may realize the measurement means described in the claims, but is not limited to this. The estimation unit 16B may realize the estimation means described in the claims, but is not limited to this. Details of each unit included in the control unit 10B will be described below in the "Flow of Information Processing Method S1B."

[0064] The storage unit 20B stores the three-dimensional model MD, coordinate system axis information CAI, and measurement point candidates MC. Details of these data are as explained in the second exemplary embodiment. In this exemplary embodiment, the storage unit 20B stores in advance the measurement point candidates MC in addition to the three-dimensional model MD and coordinate system axis information CAI. The measurement point candidates MC are generated by the same processing as steps S11A and S12A in the second exemplary embodiment. Furthermore, on the construction machine MV, marker parts that can be tracked by the external measuring device TS are installed in advance at the multiple measurement point candidates indicated by the measurement point candidates MC.

[0065] <Flow of Information Processing Method S1B> The information processing device 1B configured as above executes an information processing method S1B according to this exemplary embodiment. The flow of the information processing method S1B will be described with reference to FIG. 12. FIG. 12 is a flow diagram illustrating the flow of the information processing method S1B. As shown in FIG. 12, the information processing method S1B includes steps S13A to S14A and S15B to S16B. Steps S13A to S14A are as described in the second exemplary embodiment.

[0066] (Step S15B) In step S15B, the measurement control unit 15B performs measurement using at least one of the multiple candidate measurement points that fall within the field of view extracted by the extraction unit 13A as a measurement point. Furthermore, the measurement control unit 15B may determine the measurement point to be measured from the multiple candidate measurement points that fall within the field of view extracted by the extraction unit 13A by referring to construction machine information related to the construction machine MV.

[0067] For example, the measurement control unit 15B controls the external measurement device TS to measure the coordinates of the measurement points to be measured in the on-site coordinate system. This measurement can be performed using the above-mentioned marker parts installed at multiple candidate measurement points.

[0068] (Step S16B) In step S16B, the estimation unit 16B refers to the measurement results from the measurement control unit 15B and estimates the attitude of the construction machine MV (moving body).

[0069] For example, the estimation process for estimating the attitude may be a process for estimating the attitude of the construction machine MV based on the difference between the measurement coordinates measured for each pair of candidate measurement points. The candidate measurement point pairs are multiple points included in a line segment parallel to one of the axes in the site coordinate system when the construction machine MV is in an upright position.

[0070] (Specific example of estimation process) A specific example of the estimation process will now be described with reference to Fig. 13 and Fig. 14. Fig. 13 is a schematic diagram outlining the specific example of the estimation process. Fig. 14 is a diagram illustrating the details of the specific example of the estimation process.

[0071] In the example shown in Figure 13, the measurement point candidates MC include points A and B. When the construction machine MV is in an upright position, points A and B are included in a line segment parallel to the Z axis of the site coordinate system. Therefore, the coordinate values ​​(x coordinate and y coordinate) of points A and B other than the Z axis component are equal to each other.

[0072] Here, the posture of the construction equipment MV is expressed using a roll angle r, a pitch angle p, and a yaw angle y. For example, the posture when the construction equipment MV is upright is expressed as POSE0 (0,0,0), where r, p, and y are zero. The posture of the construction equipment MV to be estimated is expressed as POSE1 (r,p,y). The points in POSE1 corresponding to points A and B are expressed as points A1 and B1. Assume that measurement coordinates (ax1, ay1, az1) and measurement coordinates (bx1, by1, bz1) have been acquired for points A1 and B1.

[0073] The estimation process is a process for estimating POSE1 from the measured coordinates of points A1 and B1. When the attitude of the construction machine MV changes from POSE0 to POSE1, the line segment AB rotates and changes to the line segment A1-B1. When the line segment A1-B1 rotates in the opposite direction, it returns to the line segment AB.

[0074] Here, the rotation matrix R representing the rotation from POSE0 to POSE1 (r, p, y) can be expressed as in the following equation (1).

[0075]

number

[0076] For example, the coordinates of points A1 and B1 can be thought of as being calculated by multiplying the coordinates of points A and B by rotation matrix R, respectively. Therefore, points A2 and B2, obtained by multiplying points A1 and B1 by the inverse matrix of rotation matrix R, should be included in a line segment parallel to the Z axis, just like the original points A and B. Therefore, POSE1(r, p, y) can be estimated by finding a rotation matrix R that causes points A2 and B2 to be included in a line segment parallel to the Z axis.

[0077] As shown in Figure 14, the coordinates of points A2 and B2 obtained by rotating the line segment A1-B1 inversely are expressed as (ax2, ay2, az2) and (bx2, by2, bz2). As mentioned above, rotating inversely is equivalent to multiplying by the inverse matrix of the rotation matrix R. For example, the coordinates of point A2 can be calculated using the following equation (2).

[0078]

number

[0079] Similarly, suppose a pair of points C and D contained in a line segment parallel to the Y axis has been identified. Furthermore, suppose the measurement coordinates of points C1 and D1 measured for each are (cx1, cy1, cz1) and (dx1, dy1, dz1). In this case, the coordinates of points C2 and D2 obtained by rotating the line segment C1-D1 in the reverse direction are expressed as (cx2, cy2, cz2) and (dx2, dy2, dz2). The error for this pair is calculated based on the difference between cx2 and dx2 and the difference between cz2 and dz2, which are coordinate values ​​other than the Y component. If points C2 and D2 are contained in a line segment parallel to the Y axis, the error for this pair should be zero.

[0080] Similarly, suppose a pair of points E and F contained in a line segment parallel to the X-axis has been identified. Furthermore, suppose the measured coordinates of points E1 and F1 measured for each are (ex1, ey1, ez1) and (fx1, fy1, fz1). In this case, the coordinates of points E2 and F2 obtained by rotating the line segment E1-F1 in the reverse direction are expressed as (ex2, ey2, ez2) and (fx2, fy2, fz2). The error for this pair is calculated based on the difference between ey2 and fy2, and the difference between ez2 and fz2, which are coordinate values ​​other than the X component. If points E2 and F2 are contained in a line segment parallel to the X-axis, the error for this pair should be zero.

[0081] Therefore, POSE1 can be estimated by finding a rotation matrix R that minimizes the sum of the errors for each pair.

[0082] Furthermore, the information processing device 1B may repeatedly execute the information processing method S1B. This makes it possible to estimate changes in the attitude of the construction machine MV in real time. This concludes the description of the information processing method S1B.

[0083] <Advantages of this exemplary embodiment> As described above, according to this exemplary embodiment, in addition to the configuration of exemplary embodiment 2, a configuration is adopted in which an external measuring device TS is used to perform measurements using at least one of multiple candidate measurement points within the field of view as the measurement point.

[0084] Therefore, measurements can be taken at measurement points that should be used to accurately estimate the attitude of the construction machine MV, with less effort on the part of the user.

[0085] Furthermore, according to this exemplary embodiment, a configuration is employed in which the attitude of the construction machine MV is estimated by referring to the measurement results from the measurement control unit 15B.

[0086] This reduces the user's workload and enables measurements to be taken at more appropriate measurement points, enabling the posture of the MV construction machine to be estimated accurately in real time.

[0087] [Variation 3] Furthermore, in the third exemplary embodiment, the information processing device 1B may further have a function of executing the information processing method S1A according to the second exemplary embodiment, in addition to a function of executing the information processing method S1B.

[0088] [Variation 4] In addition, in the exemplary embodiments 2 and 3, examples have been described in which a construction machine MV is applied as an example of a moving body according to each exemplary embodiment of the present application, but other moving bodies may also be applied. Specific examples of such moving bodies may include, for example, a robot, a person, etc., but are not limited to these as long as the moving body can change its posture.

[0089] [Software implementation example] Some or all of the functions of the information processing devices 1, 1A, and 1B may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0090] In the latter case, the information processing devices 1, 1A, and 1B are realized, for example, by a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in FIG. 15. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the information processing devices 1, 1A, and 1B. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the information processing devices 1, 1A, and 1B.

[0091] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0092] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0093] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0094] [Appendix 1] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

[0095] [Appendix 2] Some or all of the above-described embodiments can also be described as follows: However, the present invention is not limited to the following described aspects.

[0096] (Appendix 1) An acquisition means for acquiring a three-dimensional model of a moving object; an identification means for identifying candidates of a plurality of measurement points required for estimating the posture of the moving object by referring to the three-dimensional model; An information processing device comprising:

[0097] (Appendix 2) the acquiring means further acquires information regarding a relative position of the moving object with respect to an external measuring device; The present invention further includes an extraction unit that extracts, from the plurality of measurement point candidates, a plurality of measurement point candidates that fall within a measurable range of the external measurement device, by referring to information related to the relative positions. 2. The information processing device according to claim 1.

[0098] (Appendix 3) The measuring device further includes a measuring means for performing a measurement using at least one of a plurality of measurement point candidates that fall within the measurable range extracted by the extracting means as a measurement point. 3. The information processing device according to claim 2.

[0099] (Appendix 4) The vehicle further includes an estimation unit that estimates the attitude of the moving body by referring to the measurement results obtained by the measurement unit. 4. The information processing device according to claim 3.

[0100] (Appendix 5) A plurality of measurement point candidates identified by the identification means; and The measuring device further includes a presentation unit that presents at least one of the plurality of measurement point candidates extracted by the extraction unit. 5. An information processing device according to any one of claims 2 to 4.

[0101] (Appendix 6) The display screen presented by the presentation means includes: a first graphical user interface for accepting user input regarding the location of the mobile object; and a second graphical user interface for accepting user input regarding the location of the external measurement device; Including, The acquiring means acquires information about the relative position by referring to a user input via the first graphical user interface and a user input via the second graphical user interface. 6. The information processing device according to claim 5.

[0102] (Appendix 7) The plurality of measurement point candidates identified by the identifying means include pairs of measurement point candidates arranged parallel to at least one of the axes of a Cartesian coordinate system when the moving body is in a normal position in the Cartesian coordinate system. 5. An information processing device according to any one of claims 1 to 4.

[0103] (Appendix 8) Obtaining a three-dimensional model of the moving object; Identifying a plurality of candidate measurement points required for estimating the posture of the moving object by referring to the three-dimensional model; An information processing method including:

[0104] (Appendix 9) A recording medium on which a program for causing a computer to function as an information processing device is recorded, the computer comprising: An acquisition means for acquiring a three-dimensional model of a moving object; an identification means for identifying candidates of a plurality of measurement points required for estimating the posture of the moving object by referring to the three-dimensional model; A recording medium on which a program that functions as a

[0105] [Appendix 3] Some or all of the above-described embodiments can also be expressed as follows.

[0106] An information processing device comprising at least one processor, the processor executing an acquisition process for acquiring a three-dimensional model of a moving body, and an identification process for identifying candidates for multiple measurement points required for estimating the posture of the moving body by referring to the three-dimensional model. The information processing device may further include a memory that stores a program for causing the processor to execute the acquisition process and the identification process. The program may be recorded on a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]

[0107] 1, 1A, 1B Information processing equipment 10A, 10B Control section 11, 11A Acquisition Department 12, 12A specific part 13A Extraction part 14A Presentation section 15B Measurement control section 16B Estimation part 20A, 20B storage section 30A input / output section 40A Communications Department TS external measurement device C1 processor C2 Memory

Claims

1. an acquisition means for acquiring a three-dimensional model of a moving object; a specifying means for specifying candidates of a plurality of measurement points required for estimating the posture of the moving object by referring to the three-dimensional model; Equipped with the acquiring means further acquires information regarding a relative position of the moving object with respect to an external measuring device; an extraction unit that extracts, from the plurality of measurement point candidates, a plurality of measurement point candidates that fall within a measurable range of the external measurement device by referring to information about the relative positions; The extraction means extracts the candidates by referring to information indicating the line of sight or field of view of the external measurement device, which is included in the information about the relative position, and specifying an area that is within the measurable range.

2. The measuring device further includes a measuring means for performing a measurement using at least one of a plurality of measurement point candidates that fall within the measurable range extracted by the extracting means as a measurement point. The information processing device according to claim 1 .

3. The vehicle further includes an estimation unit that estimates the attitude of the moving body by referring to the measurement results obtained by the measurement unit. The information processing device according to claim 2 .

4. A plurality of measurement point candidates identified by the identification means; and The measuring device further includes a presentation unit that presents at least one of the plurality of measurement point candidates extracted by the extraction unit. The information processing device according to claim 1 .

5. The display screen presented by the presentation means includes: a first graphical user interface for accepting user input regarding the location of the mobile object; and a second graphical user interface for accepting user input regarding the location of the external measurement device; Including, The acquiring means acquires information about the relative position by referring to a user input via the first graphical user interface and a user input via the second graphical user interface. The information processing device according to claim 4 .

6. An acquisition means for acquiring a three-dimensional model of a moving object; a specifying means for specifying candidates of a plurality of measurement points required for estimating the posture of the moving object by referring to the three-dimensional model; Equipped with An information processing device, wherein the multiple measurement point candidates identified by the identification means include pairs of measurement point candidates arranged parallel to at least one axis of a Cartesian coordinate system when the moving body is in an upright position in the Cartesian coordinate system.

7. Obtaining a three-dimensional model of the moving object; Identifying a plurality of candidate measurement points required for estimating the posture of the moving object by referring to the three-dimensional model; Including, further acquiring information regarding a relative position of the moving object with respect to an external measurement device; extracting, from the plurality of measurement point candidates, a plurality of measurement point candidates that fall within a measurable range of the external measurement device, by referring to information about the relative positions; Including, In extracting the candidates for the plurality of measurement points, the candidates are extracted by identifying an area that is the measurable range with reference to information that indicates the line of sight or the viewing angle of the external measurement device, which is included in the information about the relative positions. Information processing methods.

8. A program for causing a computer to function as an information processing device, the program comprising: an acquisition means for acquiring a three-dimensional model of a moving object; a specifying means for specifying candidates of a plurality of measurement points required for estimating the posture of the moving object by referring to the three-dimensional model; It functions as the acquiring means further acquires information regarding a relative position of the moving object with respect to an external measuring device; The computer an extraction means for extracting, from the plurality of measurement point candidates, a plurality of measurement point candidates that fall within a measurable range of the external measurement device, by referring to information about the relative positions; It functions as The extraction means extracts the candidates by identifying an area that is the measurable range with reference to information that indicates a line of sight or a viewing angle of the external measurement device, which information is included in the information about the relative position. program.

Images