INFORMATION PROCESSING DEVICE AND PROGRAM

Abstract

To provide a technique capable of more appropriately performing inspection of a target object using measurement data. The inspection support device 200 according to an embodiment of the present disclosure acquires narrow-angle data D22 and wide-angle data D21 representing shapes of a relatively narrow range and a relatively wide range of an inspection target object, respectively, and for which a relationship in terms of position and orientation between sensors 110 and 120 is predefined at a time of acquiring the respective data; acquires a collection of narrow-angle data D12 and a collection of wide-angle data D11 (a narrow-angle data group DG12 and a wide-angle data group DG11) representing shapes of a relatively narrow range and a relatively wide range, respectively.a relatively wide range of a comparison target object and for which a relationship in position and orientation between the sensors 110 and 120 is predefined at a time of acquisition of the respective data; and links, based on the wide-angle data D21 and the wide-angle data group DG11, the narrow-angle data D22 with the narrow-angle data D12 representing a portion of the comparison target object that is the same portion as a portion of the inspection target object that corresponds to the narrow-angle data D22.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an information processing apparatus and the like. STATE OF THE ART

[0002] For example, a technique for inspecting an object using measurement data such as image data is disclosed (see Patent Document 1). RELATED PRIOR ART PATENT DOCUMENT

[0003] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2016-99633 SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION

[0004] For example, when using measurement data that represents the shape of a relatively narrow area, such as image data with a relatively narrow viewing angle, the data can represent a detailed shape of that area. Therefore, the condition of that area of a target object can be determined in more detail.

[0005] For measurement data representing a shape within a relatively narrow range, it may not be possible to determine which area of the entire inspection target range of the target object the data corresponds to. Therefore, for example, measurement data of a section of an inspection target cannot be adequately extracted from the measurement data group, and as a result, there is a possibility that the inspection cannot be adequately performed.

[0006] In a case where measurement data representing a shape of a relatively wide range, such as image data with a relatively wide viewing angle, is used, it is easier to determine the portion of the target object corresponding to a data range within the entire inspection target range of the target object.

[0007] However, for measurement data representing a shape of a relatively wide range, it is less likely that a detailed shape will be reflected in the data. Therefore, the condition of the range included in the measurement data cannot be determined in detail, and as a result, there is a possibility that the inspection cannot be performed adequately.

[0008] In view of the above problem, an object of the present invention is to provide a technique capable of more appropriately inspecting a target object using measurement data. MEANS TO SOLVE THE PROBLEMS

[0009] To achieve the above object, according to an embodiment of the present disclosure, an information processing apparatus is provided. The information processing apparatus includes: a first sensing part configured to acquire first measurement data and second measurement data, wherein the first measurement data represents a shape of a relatively narrow range of a first target object and the second measurement data represents a shape of a relatively wide range of the first target object, and for which a relationship in terms of position and orientation between sensors at a time of acquiring the first measurement data and the second measurement data with respect to the first measurement data is predefined; a second sensing part configured to acquire a third measurement data group and a fourth measurement data group, wherein the third measurement data group is a collection of third measurement data, the third measurement data representing a shape of a relatively narrow range of a second target object, the second target object being a comparison target of the first target object, the fourth measurement data group is a collection of fourth measurement data, the fourth measurement data representing a shape of a relatively wide range of the second target object, and for which a relationship of position and orientation between the sensors at a time of acquiring the third measurement data and the fourth measurement data with respect to the third measurement data is predefined; and a linking part configured to link the first measurement data to the third measurement data based on the second measurement data and the fourth measurement data group, wherein the third measurement data represents a portion of the second target object that is the same portion as a portion of the first target object that corresponds to the first measurement data.

[0010] According to another embodiment of the present disclosure, an information processing apparatus comprises: a first sensing part configured to acquire a first measurement data group and a second measurement data group, wherein the first measurement data group is a collection of first measurement data, the first measurement data representing a shape of a relatively narrow range of a first target object, and the second measurement data group is a collection of second measurement data, the second measurement data representing a shape of a relatively wide range of the first target object and for which a relationship of position and orientation between sensors at a time of acquiring the first measurement data and the second measurement data with respect to the first measurement data is predefined; a storage part configured to store three-dimensional shape data of a second target object as a comparison target of the first target object and a third measurement data group, wherein the third measurement data group is a collection of third measurement data, the third measurement data representing a shape of the second target object and for which a correspondence relationship with a portion of the second target object or the three-dimensional shape data is predefined; and a linking part configured to link the first measurement data and the third measurement data based on the second measurement data group and the three-dimensional shape data of the second target object, wherein the first measurement data represents a portion of the first target object and the third measurement data represents a portion of the second target object that is a same portion as the portion of the first target object.

[0011] According to yet another embodiment of the present disclosure, an information processing apparatus comprises: a first sensing part configured to acquire a first measurement data group and a second measurement data group, wherein the first measurement data group is a collection of first measurement data, the first measurement data representing a shape of a relatively wide area of a first target object, the second measurement data group is a collection of second measurement data, the second measurement data representing a shape of a relatively wide area of the first target object, and for which a relationship of position and orientation between sensors at a time of acquiring the first measurement data and the second measurement data with respect to the first measurement data is predefined; and a linking part configured to link the second measurement data to a portion of the first target object represented by the second measurement data based on the first measurement data group.

[0012] According to yet another embodiment of the present disclosure, an information processing apparatus includes a program for causing an information processing apparatus to perform: a first acquisition step of acquiring first measurement data and second measurement data, wherein the first measurement data represents a shape of a relatively narrow range of a first target object and the second measurement data represents a shape of a relatively wide range of the first target object and for which a relationship of position and orientation between sensors at a time of acquiring the first measurement data and the second measurement data with respect to the first measurement data is predefined; a second acquisition step of acquiring a third measurement data group and a fourth measurement data group, wherein the third measurement data group is a collection of third measurement data, wherein the third measurement data represents a shape of a relatively narrow range of a second target object, wherein the second target object is a comparison target of the first target object, the fourth measurement data group is a collection of fourth measurement data, wherein the fourth measurement data represents a shape of a relatively wide range of the second target object and for which a relationship of position and orientation between the sensors at a time of acquiring the third measurement data and the fourth measurement data with respect to the third measurement data is predefined; and a linking step of linking the first measurement data to the third measurement data based on the second measurement data and the fourth measurement data group, wherein the third measurement data represents a portion of the second target object that is the same portion as a portion of the first target object that corresponds to the first measurement data. EFFECTS OF THE INVENTION

[0013] According to the embodiments described above, it is possible to more appropriately inspect a target object using the measurement data. BRIEF DESCRIPTION OF THE DRAWINGS [ Fig. 1] Fig. Figure 1 is a diagram schematically showing an example of a test support system. [ Fig. 2] Fig. Figure 2 is a diagram schematically showing another example of the test support system. [ Fig. 3] Fig. 3 is a view showing a specific example of an inspection target object. [ Fig. 4] Fig. 4 is a view showing a specific example of an inspection target object. [ Fig. 5] Fig. 5 is a view showing a specific example of an inspection target object. [ Fig. 6] Fig. 6 is a diagram showing an example of a hardware configuration of a test support device. [ Fig. 7] Fig. 7 is a functional block diagram showing a first example of the configuration of the test support device. [ Fig. 8] Fig. 8 is a functional block diagram showing a second example of the configuration of the test support device. [ Fig. 9] Fig. 9 is a diagram illustrating a first example of a screen displayed on a display device. [ Fig. 10] Fig. 10 is a diagram illustrating a second example of a screen displayed on the display device. [ Fig. 11] Fig. 11 is a diagram illustrating a third example of a screen displayed on the display device. EMBODIMENT OF THE INVENTION

[0014] Embodiments are described below with reference to the accompanying drawings. [Overview of Test Support System]

[0015] First, an example of a test support system SYS according to the present embodiment will be described with reference to Fig. 1 to 5.

[0016] Fig. Figure 1 is a diagram showing an example of a test support system SYS. Fig. Figure 2 is a diagram showing another example of the test support system SYS. Fig. 3 to 5 are views that show specific examples of a test target object. In particular, Fig. 3 is a side view showing an excavator as an example of a test target. Fig. 4 is a side view showing a mobile crane (crawler crane) as another example of the inspection target. Fig. 5 is a side view showing a continuous discharger as yet another example of an inspection target.

[0017] The inspection support system SYS supports a user's inspection (hereinafter referred to as a "comparison inspection") of a difference between an inspection target object and a comparison target.

[0018] The test target object is, for example, a large structure. The large structure as a test target includes, for example, a large machine. The large machine includes, for example, a work machine such as an excavator, a mobile crane (crawler crane), or a continuous unloader, as in Fig. 3 to 5. The large machine may include a large ship. The large structure as the inspection target includes, for example, a large facility (a factory building or factory equipment). The large facility includes, for example, a building or equipment of a power plant, a building or equipment of a steel mill, and the like. The inspection target includes a large civil engineering structure and a large architectural structure. The large engineering structures include, for example, road facilities, railway facilities, dams, bridges, airport facilities, port facilities, and the like. The large architectural structures include, for example, skyscrapers, large sports venues (stadiums), etc.

[0019] For example, comparison testing includes testing for changes in the test target relative to a previous state, i.e., testing for changes in the test target over time. In this case, the comparison target itself is a previous test target. The changes in the test target over time include, for example, the occurrence of rust, deformation, scratches, loosening of a screw, missing components, modifications, discoloration, and relative displacement with respect to the surroundings due to an earthquake, etc. Comparison testing includes testing for differences between the test target and the comparison target of the same type (same design) as the test target.In this case, the reference target is, for example, a product that has passed the test with the mean tolerance among prototypes of products of the same design as the test target and initial batches thereof. The difference between the test target and the other reference target of the same type (same design) includes, for example, a difference in shape or color that goes beyond an assumed manufacturing defect, the presence or absence of a component defect, and the like.

[0020] The scope of the comparison test can be the entirety or part of the test target and the comparison test object. The following mainly describes a case where the scope of the comparison test is the entirety of the test target and the comparison test object.

[0021] Furthermore, multiple comparison targets can be used. For example, when testing a change in a test target in time series, a comparison test is performed between a current test target and multiple comparison targets corresponding to the test target at several different previous points in time.

[0022] As in Fig. 1, the test support system SYS comprises, for example, a sensor device 100 and a test support device 200. As shown in Fig. 2, the test support system SYS may further comprise a terminal device 300.

[0023] The sensor device 100 acquires measurement data related to the shape of an inspection target or a comparison target. For example, the sensor device 100 acquires measurement data related to the shape of the inspection target or the comparison target in response to an external operation. The sensor device 100 may acquire measurement data related to the shape of the inspection target or the comparison target in accordance with the movement of the worker wearing the sensor device 100. The sensor device 100 may be configured to be movable, for example, like a drone. In this case, the sensor device 100 acquires measurement data related to the shape of the inspection target or the comparison target in response to an external operation while moving, for example, in response to an external operation.The sensor device 100 can automatically acquire measurement data related to the shape of the inspection target or the comparison target while moving autonomously.

[0024] The sensor device 100 includes sensors 110 and 120.

[0025] The sensor device 100 may be a device specialized for a function of acquiring sensor data, or may be a multi-purpose device. For example, the sensor device 100 may be an information device (smart device) equipped with an imaging device, a distance sensor, or the like, such as the sensors 110 and 120. The information device includes, for example, a smartphone, a tablet terminal, and the like. The sensor device 100 may be a drone to which an imaging device, a distance sensor, or the like is attached as the sensors 110 and 120.

[0026] The sensors 110 and 120 acquire measurement data relating to the shapes of the test target and the comparison target, respectively.

[0027] Sensors 110 and 120 are, for example, an imaging device capable of capturing an image (image data) reflecting the shape of the inspection target. The imaging device is, for example, a monocular camera. The imaging device may be a stereo camera, an RGB-D camera, a TOF (Time Of Flight) camera, or the like. The camera may be a camera (hereinafter referred to simply as a "3D camera") capable of capturing depth information in addition to a two-dimensional image. Sensors 110 and 120 may be a distance sensor. The distance sensor may be, for example, a distance sensor capable of capturing point group data corresponding to the shape of the inspection target with respect to the sensor device 100 (sensors 110 and 120).The distance sensor includes, for example, a LiDAR (Light Detection and Ranging) sensor, a millimeter-wave radar, an ultrasonic sensor, and the like. Furthermore, one of the sensors 110 and 120 may be an imaging device, and the other may be a distance sensor.

[0028] Sensor 110 is a sensor with a larger measurement range than sensor 120. Thus, sensor 110 can acquire measurement data representing the shape of the target object over a larger area than sensor 120. For convenience, the measurement data acquired by sensor 110 may be referred to as "wide-angle data" below. Sensor 110 is, for example, a surround-view camera.

[0029] Sensor 120 is a sensor with a narrower measurement range than sensor 110. Thus, sensor 120 can acquire measurement data representing a detailed shape of a relatively narrow area of the inspection target. Hereinafter, the measurement data acquired by sensor 120 will be referred to as "narrow-angle data" for convenience.

[0030] Sensors 110 and 120 are fixed in their relative position and orientation. Furthermore, sensors 110 and 120 are synchronized in the acquisition time of measurement data. Thus, when sensor device 100 is located at a specific position, both wide-angle data and narrow-angle data are acquired by sensors 110 and 120.

[0031] The measurement data (the wide-angle data and the narrow-angle data) regarding the shapes of the inspection target and the comparison target may be acquired by the same sensor device 100 or may be acquired by different sensor devices 100. In the latter case, the sensor device 100 that acquires the measurement data regarding the shape of the inspection target and the sensor device 100 that acquires the measurement data regarding the shape of the comparison target may have different relative positional relationships of the sensors 110 and 120.

[0032] The measurement data (the wide-angle data and the narrow-angle data) acquired by the sensor device 100 are transferred to the inspection support device 200.

[0033] For example, the sensor device 100 transmits the measurement data to the test support device 200 via a predetermined communication network. The predetermined communication network includes, for example, a wide area network (WAN). The wide area network includes, for example, a mobile communication network having a base station as a terminal, a satellite communication network using a communication satellite, and the Internet. The predetermined communication network may include a local area network (LAN). The predetermined communication network may include a short-distance communication line based on a communication standard such as Bluetooth (registered trademark) or WiFi. As shown in Fig. 1, the sensor device 100 transmits, for example, measurement data directly to the test support device 200. As shown in Fig. 2, the sensor device 100 can transmit the measurement data to the test support device 200 via the terminal device 300 as a relay device.

[0034] The measurement data acquired by the sensor device 100 can be stored on a portable recording medium of the sensor device 100 and can be imported from the recording medium into the test support device 200. The recording medium is, for example, a flash memory such as a hard disk drive (HDD), a solid state drive (SSD), a USB memory, or an SD card.

[0035] The inspection support device 200 supports a comparison test performed by a user. For example, the inspection support device 200 provides the user with information for a comparison test based on the measurement data (the wide-angle data and the narrow-angle data) input from the sensor device 100.

[0036] The test support device 200 is, for example, a terminal device (user terminal) used by a user. The user terminal may be a stationary terminal device such as a desktop personal computer (PC). The user terminal may be a mobile (portable) terminal device (portable terminal) such as a smartphone, a tablet terminal, or a laptop PC. [Hardware configuration of test support device]

[0037] Next, a hardware configuration of the test support device 200 will be described with reference to Fig. 6 described.

[0038] Fig. 6 is a block diagram showing an example of a hardware configuration of the test support device 200.

[0039] The functions of the test support device 200 are implemented by any hardware or a combination of any hardware and software. As in Fig. 6, the test support device 200 includes, for example, an external interface (I / F) 201, an auxiliary storage device 202, a storage device 203, a central processing unit (CPU) 204, a high-speed computing device 205, a communication interface (I / F) 206, an input device 207, a display device 208, and a sound output device 209, which are connected via a bus BS2.

[0040] The external interface 201 functions as an interface for reading from and writing to the recording medium 201A. The recording medium 201A includes, for example, a flexible floppy disk, a CD (Compact Disc), a DVD (Digital Versatile Disc), a BD (Blu-ray (Trademark) Disc), an SD memory card, a USB memory, and the like. The test support device 200 can read various types of information used in processing via the recording medium 201A and store the information in the auxiliary storage device 202, and can install programs for implementing various functions.

[0041] The test support device 200 can acquire various data and programs used in processing from an external device via the communication interface 206.

[0042] The auxiliary storage device 202 stores the various installed programs and also stores files, data, and the like required for various processes. The auxiliary storage device 202 includes, for example, an HDD, an SSD, or the like.

[0043] When an instruction to activate a program is issued, the storage device 203 reads the program from the auxiliary storage device 202 and stores the program. The storage device 203 includes, for example, a dynamic random access memory (DRAM) or a static random access memory (SRAM).

[0044] The CPU 204 executes various programs loaded from the auxiliary storage device 202 into the storage device 203 and implements various functions with respect to the test support device 200 according to the programs.

[0045] The high-speed computing device 205 performs computational processing at a relatively high speed in conjunction with the CPU 204. The high-speed computing device 205 includes, for example, a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or the like.

[0046] The high-speed computing device 205 may be omitted depending on the speed of the required computational processing.

[0047] The communication interface 206 is used as an interface for connecting to the external device to be able to communicate with the external device. The test support device 200 can communicate with an external device of the test support device 200 via the communication interface 206. The communication interface 206 may include multiple types of communication interfaces depending on a communication method with a device to be connected.

[0048] The input device 207 receives various inputs from a user.

[0049] The input device 207 includes, for example, an input device (hereinafter referred to as an "operation input device") that receives a mechanical operation input from a user. The operation device for remote control may be an operation input device. The operation input device includes, for example, a button, a toggle switch, a lever, a keyboard, a mouse, a touch panel mounted on the display device 208, a touch pad provided separately from the display device 208, and the like.

[0050] The input device 207 may include a voice input device capable of receiving voice input from the user. The voice input device includes, for example, a microphone capable of recording the user's voice.

[0051] The input device 207 may include a gesture input device capable of receiving a gesture input from the user. For example, the gesture input device includes a camera capable of capturing an image of a user's gesture.

[0052] The input device 207 may include a biometric input device capable of receiving biometric input from the user. For example, the biometric input device includes a camera capable of capturing image data containing information about a user's fingerprint or iris.

[0053] The display device 208 displays an information screen and an operation screen to the user of the test support device 200. The display device 208 is, for example, a liquid crystal display or an organic electroluminescence (EL) display.

[0054] The sound output device 209 transmits various types of information to the user of the test support device 200 through sound. The sound output device 209 is, for example, a buzzer, an alarm, a speaker, or the like. [Functional configuration of test support device]

[0055] The functional configuration of the test support device 200 will be described with reference to Fig. 7 to 11.

[0056] Fig. 7 is a functional block diagram showing an example of a configuration of the test support device 200. Fig. 8 is a functional block diagram illustrating another example of a configuration of the test support device 200. Fig. 9 is a diagram illustrating a first example (screen 900) of a screen displayed on the display device 208. Fig. 10 is a diagram illustrating a second example (screen 1000) of the screen displayed on the display device 208. Fig. 11 is a diagram illustrating a third example (screen 1100) of the screen displayed on the display device 208.

[0057] It should be noted that Fig. 9 to 11 illustrate a case where testing of a change from a previous state of the excavator as the test target object is performed.

[0058] The following mainly describes a case where the measurement data is image data.

[0059] As in Fig. 7 and Fig. 8, the test support device 200 includes a data acquisition part 2001, a comparison target data generation part 2002, and a comparison test support part 2003.

[0060] The data acquisition part 2001 acquires the measurement data acquired by the sensor device 100.

[0061] The comparison target data generation part 2002 generates a data set (reference data set DS17) representing the overall shape of the comparison target object based on the wide-angle data group DG11 and the narrow-angle data group DG12 representing the shape of the comparison target object acquired by the data acquisition part 2001. The wide-angle data group DG11 is a collection of a large number of wide-angle data D11, and the narrow-angle data group DG12 is a collection of a large number of narrow-angle data D12.

[0062] For example, unique identification information such as an ID (identification) is defined for each wide-angle data D11. Similarly, unique identification information such as an ID is defined for each narrow-angle data D12. As described above, since the acquisition timings of the wide-angle data and the narrow-angle data by the sensors 110 and 120 are synchronized, the identification information of the wide-angle data D11 and the identification information of the narrow-angle data D12 acquired at the same time are linked to each other by a database or the like. Thus, the inspection support device 200 can combine the wide-angle data D11 and the narrow-angle data D12 acquired at the same time (ie,at the time when the sensor device 100 is at the same position and in the same orientation) from the wide-angle data group DG11 and the narrow-angle data group DG12.

[0063] The comparison target data generation part 2002 includes a structure from motion (SfM) processing part 2002A, a storage part 2002B, a narrow angle data position estimation part 2002C, a data set generation part 2002D, and a storage part 2002E.

[0064] The functions of the SfM processing part 2002A, the narrow angle data position estimation part 2002C, and the data set generation part 2002D are implemented, for example, by loading a program installed in the auxiliary storage device 202 into the storage device 203 and executing the program in the CPU 204. The functions of the storage parts 2002B and 2002E are implemented by storage areas defined in the auxiliary storage device 202 or the like.

[0065] The SfM processing part 2002A performs known SfM processing based on the wide-angle data group DG11 of the comparison target acquired by the data acquisition part 2001. Thus, the SfM processing part 2002A can output a three-dimensional model (3D model) D13 of the comparison target object and information data about the position and orientation of the sensor 110 corresponding to target wide-angle data D11 (wide-angle position information data D14) on a per-wide-angle data D11 basis. Hereinafter, a collection of wide-angle position information data D14 on a per-wide-angle data D11 basis is referred to as a "wide-angle position information data group DG14."

[0066] The position and orientation of the sensor 110 corresponding to the wide-angle data D11 are a relative position and a relative orientation of the sensor 110 with respect to the comparison target object when the wide-angle data D11 is acquired.

[0067] The storage part 2002B stores sensor relative position information data D15 in advance.

[0068] The sensor relative position information data D15 is information data indicating relationships between relative positions and relative orientations of the sensors 110 and 120 in the sensor device 100 that acquired the measurement data with respect to the shape of the comparison target object.

[0069] The narrow angle data position estimation part 2002C estimates a position and an orientation of the sensor 120 according to the target narrow angle data D12 on a per-narrow angle data D12 basis. The narrow angle data position estimation part 2002C outputs information data about the position and orientation of the sensor 120 corresponding to the target narrow angle data D12 (narrow angle position information data D16) on a per-narrow angle data D12 basis. Hereinafter, a collection of the narrow angle position information data D16 on a per-narrow angle data D12 basis may be referred to as a "narrow angle position information data group DG16."

[0070] The position and orientation of the sensor 120 corresponding to the narrow angle data D12 are a relative position and a relative orientation of the sensor 120 with respect to the comparison target object when the narrow angle data D12 is acquired.

[0071] For example, the narrow-angle data position estimation part 2002C extracts narrow-angle wide-angle data D11 associated with identification information on a per-narrow-angle data D12 basis from the wide-angle data group DG11. Subsequently, the narrow-angle data position estimation part 2002C extracts the wide-angle position information data D14 indicating the position and orientation of the sensor 110 corresponding to the extracted wide-angle data D11 on a per-narrow-angle data D12 basis from the wide-angle position information data group DG14. Then, the narrow-angle data position estimation part 2002C calculates the position and orientation of the sensor 120 corresponding to the narrow-angle data D12 based on the extracted wide-angle position information data D14 and the sensor relative position information data D15 on a per-narrow-angle data D12 basis.

[0072] The data set generation part 2002D generates a reference data set DS17 including a wide angle data group DG11, a narrow angle data group DG12, a 3D model D13, a wide angle position information data group DG14, and a narrow angle position information data group DG16.

[0073] The storage part 2002E stores the reference data record DS17 generated by the data record generation part 2002D.

[0074] The comparison inspection support part 2003 supports the user's comparison inspection based on wide angle data D21 and narrow angle data D22 of the inspection target and the reference data set DS17 acquired by the data acquisition part 2001.

[0075] The wide-angle data D21 and the narrow-angle data D22 are measurement data acquired by the sensors 110 and 120 at the same time, that is, when the sensor device 100 is at the same position and in the same orientation.

[0076] As in Fig. 7 and Fig. 8, the comparison check support part 2003 includes a wide-angle data position estimation part 2003A, a storage part 2003B, a narrow-angle data position estimation part 2003C, a comparison target data search part 2003D, and a display processing part 2003E. As shown in Fig. 9, the comparison inspection support part 2003 may include a difference detection part 2003F.

[0077] The functions of the wide-angle data position estimation part 2003A, the narrow-angle data position estimation part 2003C, the comparison target data search part 2003D, the display processing part 2003E, and the difference detection part 2003F are implemented, for example, by loading a program installed in the auxiliary storage device 202 into the storage device 203 and executing the program by the CPU 204. The function of the storage part 2003B is implemented by a memory area defined in the auxiliary storage device 202, or the like.

[0078] The wide-angle data position estimation part 2003A estimates the position and orientation of the sensor 110 according to the wide-angle data D21 of the inspection target based on the wide-angle data group DG11 and the wide-angle position information data group DG14 of the comparison target included in the reference data set DS17. The wide-angle data position estimation part 2003A outputs information data about the position and orientation of the sensor 110 corresponding to the wide-angle data D21 (wide-angle position information data D23).

[0079] The position and orientation of the sensor 110 corresponding to the wide-angle data D21 of the inspection target are a relative position and a relative orientation of the sensor 110 with respect to the inspection target when the wide-angle data D21 is acquired.

[0080] For example, the wide-angle data position estimation part 2003A extracts, from the comparison target wide-angle data group DG11, one or more pieces of wide-angle data of the inspection target, in which the same section (corresponding point) as the wide-angle data D21 of the inspection target is represented, using a known corresponding point search method. The wide-angle data position estimation part 2003A estimates (calculates) the position and orientation of the wide-angle data based on the position and orientation information data of the sensor 110 corresponding to the extracted wide-angle data included in the wide-angle position information data group DG14.

[0081] The storage part 2003B stores sensor relative position information data D24 in advance.

[0082] The sensor relative position information data D24 is information indicating relationships between relative positions and relative orientations of the sensors 110 and 120 in the sensor device 100 that acquired the measurement data with respect to the shape of the inspection target object.

[0083] The narrow angle data position estimation part 2003C estimates the position and orientation of the sensor 120 according to the narrow angle data D22. The narrow angle data position estimation part 2003C outputs information data (end angle position information data D25) about the position and orientation of the sensor 120 corresponding to the narrow angle data D22.

[0084] The position and orientation of the sensor 120 corresponding to the narrow angle data D22 of the inspection target are the relative position and relative orientation of the sensor 120 with respect to the inspection target when the narrow angle data D22 is acquired.

[0085] For example, the narrow angle data position estimation part 2003C calculates the position and orientation of the sensor 120 corresponding to the narrow angle data D22 based on the position information of the sensor 110 corresponding to the wide angle data D21 included in the wide angle position information data D23 and the data of the sensor relative position information data D24.

[0086] Note that when the sensor device 100 that acquires the wide-angle data group DG11 and the narrow-angle data group DG12 of the comparison target is the same as the sensor device 100 that acquires the wide-angle data D21 and the narrow-angle data D22 of the inspection target, the storage part 2003B can be omitted. In this case, the narrow-angle data position estimation part 2003C can estimate the position and orientation of the sensor 120 according to the narrow-angle data D22 using the sensor relative position information data D15. The function of the narrow-angle data position estimation part 2002C and the function of the narrow-angle data position estimation part 2003C are essentially the same functions and are implemented, for example, by the same program installed in the auxiliary storage device 202.

[0087] The comparison target data search part 2003D searches, based on the narrow angle position information data D25 and the narrow angle position information data group DG16, for suitable narrow angle data D12 as a comparison target for the narrow angle data D22 of the inspection target from the narrow angle data group DG12 of the comparison target. The suitable narrow angle data D12 as the comparison target for the narrow angle data D22 of the inspection target is narrow angle data D12 including data from the narrow angle data group DG12 of the comparison target that represents a shape of the same portion (a common portion) as a portion represented by the narrow angle data D22 of the inspection target (that is, the same portion is detected). The comparison target data search part 2003D outputs one or more narrow angle data D12 as the comparison target narrow angle data D26 as a search result.

[0088] The display processing part 2003E displays the narrow angle data D22 and the comparison target narrow angle data D26 on a display device 208 to compare them with each other. The display processing part 2003E can display an image representing an overall shape of the inspection target area of the comparison target and the inspection target based on a 3D model D13 of the comparison target, together with the narrow angle data D22 and the comparison target narrow angle data D26.

[0089] As in Fig. 9, the display processing part 2003E causes the display device 208 to display a screen 900, for example.

[0090] On the screen 900, an image 901 corresponding to the comparison target narrow angle data D26 and an image 902 corresponding to the narrow angle data D22 are displayed side by side. This allows the user to check changes to the inspection target relative to the comparison target (excavator) representing the previous state by comparing images 901 and 902 showing the same sections (common sections) of the inspection target and the comparison target.

[0091] As in Fig. 10, the display processing part 2003E may cause the display device 208 to display a screen 1000.

[0092] The screen 1000 displays an image group 1001 corresponding to the narrow angle data group DG12 of the comparison target, an image 1002 corresponding to the narrow angle data D22 of the inspection target, and an image 1003 schematically illustrating the overall shape of the inspection target area of the comparison target and the inspection target (excavator). Furthermore, an image of the 3D model D13 illustrating the overall shape of the inspection target area of the inspection target and the comparison target may be displayed on the screen 1000 instead of the image 1003.

[0093] Image group 1001 includes images 1001-1 to 1001-5, which correspond to the narrow angle data D12 contained in the narrow angle data group DG12.

[0094] Among images 1001-1 to 1001-5, image 1001-5 corresponds to the comparison target angle data D26 and is highlighted by a thick frame. This allows the user to easily identify image 1001-5, which represents the same section (common section) as image 1002 of the inspection target, from the comparison target image group 1001.

[0095] Images 1001-1 to 1001-5 are connected to sections P1 to P5 of image 1003 in a shape connected by curved lines. Furthermore, image 1002 is connected to section P5 of image 1003 in a shape connected by a curved line, similar to image 1001-5. Thus, the user can easily identify which section of the inspection target (excavator) images 1001-1 to 1001-5 and 1002 depict.

[0096] For example, the comparison inspection support part 2003 specifies, on the 3D model D13, a portion of the comparison target object reflected in the narrow angle data D12 based on the narrow angle data D12 included in the narrow angle data group DG12 based on the narrow angle position information data group DG16. Thus, the display processing part 2003E can connect the images 1001-1 to 1001-5 corresponding to the narrow angle data D12 included in the narrow angle data group DG12 to the respective portions of the image 1003 representing the overall shape of the inspection target area of the comparison target object.

[0097] As in Fig. 11, the display processing part 2003E may cause the display device 208 to display a screen 1100.

[0098] As in the case of Fig. 9, an image 1101 corresponding to the comparison target narrow angle data D26 and an image 1102 corresponding to the narrow angle data D22 are displayed side by side on the screen 1100.

[0099] In this example, unlike the case of Fig. 9, one of the comparison target narrow angle data D26 and the narrow angle data D22 is corrected so that the difference in appearance of the same portions (common portions) of the comparison target and the inspection target between the image 1101 and the image 1102 is reduced.

[0100] In this example, the scale and direction of the image corresponding to the comparison target narrow angle data D26 are corrected to generate image 1101, and the generated image 1101 is displayed on the screen 1100 so that it can be compared with the image 1102 corresponding to the narrow angle data D22. Thus, the user can more efficiently inspect a change in the previous state of the excavator by comparing images 1101 and 1102, in which the same sections (common sections) of the inspection target and the comparison target are depicted, and the difference in appearance between these sections is relatively small.

[0101] The viewing angle conversion process of the first image can be performed in such a manner that a first image corresponding to one of the narrow angle data D26 and the narrow angle data D22 matches the position of the sensor 120 corresponding to a second image corresponding to the other. Thus, the display processing part 2003E can cause the display device 208 to display images (the image after the viewing angle conversion of the first image and the second image) in which the same portions (common portions) between the inspection target and the comparison target are viewed in the same way. Therefore, the user can more efficiently perform the comparison inspection of the inspection target and the comparison target by comparing two images in which the same portions (common portions) of the inspection target and the comparison target are displayed and the appearance between these portions is the same.

[0102] Back to Fig. 8, the difference detection part 2003F detects the difference in the form of the same portion (common portion) of the comparison target and the inspection target, which is reflected in the comparison target narrow angle data D26 and the narrow angle data D22, respectively.

[0103] For example, the difference detection part 2003F detects a difference of a predefined type between the shapes of the same portions (common portions) of the comparison target and the inspection target corresponding to the comparison target narrow angle data D26 and the narrow angle data D22, respectively, by applying a discriminator based on a known image processing technique or machine learning. The predefined type difference may include, for example, scratches, dents, predetermined component attachment orientations, the presence or absence of predetermined components, and the like.

[0104] Furthermore, the comparison target angle data D26 and the final angle data D22 may be pre-labeled to represent a feature related to a shape difference. The label indicating the feature related to the shape difference includes, for example, a label indicating the presence of scratches, a label indicating the presence of dents, a label indicating the presence of components, and the like.

[0105] For example, the difference detection part 2003F uses a discriminator based on a known image processing method or machine learning to detect the presence or absence of a feature corresponding to a target mark in the target narrow angle data D12 on a per-narrow angle data D12 basis and on a per-marker type basis. If the target narrow angle data D12 has a feature corresponding to the target mark, the difference detection part 2003F associates the target mark as metadata with the target narrow angle data D12. Similarly, the difference detection part 2003F detects the presence or absence of a feature corresponding to the target mark in the narrow angle data D22 on a per-marker type basis. If the narrow angle data D22 has a feature corresponding to the target mark, the difference detection part 2003F associates the target mark as metadata with the narrow angle data D22.Thus, the difference detection part 2003F can detect the shape difference between the same portions (common portions) of the comparison target and the inspection target by the difference in label assigned to each of the comparison target narrow angle data D26 and the narrow angle data D22 of the inspection target.

[0106] The difference detection part 2003F can detect the shape difference between the same portions (common portions) of the comparison target and the inspection target by detecting the presence or absence of a difference between the comparison target narrow angle data D26 and the narrow angle data D22 on a basis per several predefined difference types.

[0107] The difference detection part 2003F can estimate the degree of shape difference between the same portions (common portions) of the comparison target and the inspection target corresponding to the comparison target narrow angle data D26 and the narrow angle data D22, respectively, by applying a discriminator based on, for example, a known image processing technique or machine learning.

[0108] When there are a plurality of narrow angle data D22, the difference detection part 2003F may select a plurality of combinations of the comparison target narrow angle data D26 and the narrow angle data D22 based on the content and degree of the shape difference between the same portions (common portions) of the comparison target and the inspection target.

[0109] When the display processing part 2003E displays the comparison target narrow angle data D26 and the narrow angle data D22 on the display device 208, the display processing part 2003E may reflect the detection result of the difference detection part 2003F on the display contents.

[0110] For example, the display processing part 2003E displays the comparison target narrow angle data D26 and the narrow angle data D22 on the display device 208 to emphasize the shape difference between the comparison target and the inspection target corresponding to the comparison target narrow angle data D26 and the narrow angle data D22, respectively. Specifically, the display processing part 2003E may display a mark at an image portion where a difference occurs between two images corresponding to the comparison target narrow angle data D26 and the narrow angle data D22. Further, when the shape of the inspection target corresponding to the narrow angle data D22 has a feature that is absent in the shape of the comparison target corresponding to the comparison target narrow angle data D26, the display processing part 2003E may cause the display device 208 to display the narrow angle data D22 to emphasize the feature portion.For example, the display processing part 2003E performs image processing to emphasize the amount of scratches, dents, or the like of the inspection target object present in the image corresponding to the narrow angle data D22, and causes the display device 208 to display the narrow angle data D22.

[0111] The display processing part 2003E may cause the display device 208 to display an image representing the overall shape of the inspection target area of the inspection target and the comparison target in association with information regarding the difference in the portion where the difference in shape between the comparison target and the inspection target is detected by the difference detection part 2003F. For example, the display processing part 2003E displays the information about the difference in association with a portion of the image representing the overall shape of the inspection target area of the inspection target and the comparison target, the portion of the image corresponding to each of the portions of the inspection target and the comparison target in which the difference in shape is detected by the difference detection part 2003F.The information about the difference includes, for example, summary information that indicates the content and degree of the difference.

[0112] When there are a plurality of narrow angle data D22, the display processing part 2003E may cause the display device 208 to display a plurality of combinations of the comparison target narrow angle data D26 and the narrow angle data D22 in a distinguishable manner according to the content and degree of difference selected by the difference detection part 2003F.

[0113] In this way, the inspection support device 200 can estimate the position and orientation of the sensor 110 corresponding to the wide-angle data D11 based on the wide-angle data group DG11 of the comparison target. Thus, the inspection support device 200 can estimate the position and orientation of the sensor 120 corresponding to the narrow-angle data D12 based on the sensor relative position information data D15, assuming that the relative positional relationship between the sensors 110 and 120 is fixed. The inspection support device 200 can estimate the position and orientation of the sensor 110 corresponding to the wide-angle data D21 of the inspection target based on the information about the wide-angle data group DG11 of the comparison target and the position and orientation of the sensor 110 corresponding to the wide-angle data D11.Thus, the inspection support device 200 can estimate the position and orientation of the sensor 120 corresponding to the narrow angle data D22 of the inspection target based on the sensor relative position information data D24, assuming that the relative positional relationship between the sensors 110 and 120 is fixed. Therefore, the inspection support device 200 can extract a combination of the narrow angle data D12 and D22 representing the shapes of the same portions (common portions) of the inspection target and the comparison target based on the information about the position and orientation of the sensor 120 corresponding to the narrow angle data D12 and D22, respectively.Therefore, the user can identify the detailed shapes of the same sections (common sections) of the comparison target and the inspection target based on the narrow-angle data D12 and D22, while identifying the overall shape of the inspection area of the comparison target and the inspection target based on the wide-angle data D11 and D21. As a result, the user can perform the comparison inspection more appropriately. [Other embodiments]

[0114] Next, further embodiments will be described.

[0115] The embodiment described above can be modified or changed as required.

[0116] In the above-described embodiment, for example, the 3D model D13 of the comparison target and the narrow angle data group DG12, in which a portion of the comparison target or a portion of the 3D model D13 is identified for each narrow angle data group D12, can be created in advance and stored in a predetermined storage area. The 3D model D13 of the comparison target is, for example, a three-dimensional computer-aided design (CAD) model with final specifications at the time of design of the comparison target.

[0117] In this case, the wide-angle data group DG11 of the comparison target object may be omitted, and the narrow-angle data position estimation part 2002C may estimate the position and orientation of the sensor 120 corresponding to the narrow-angle data D12 on a per-narrow-angle data D12 basis based on the 3D model D13. For example, the narrow-angle data position estimation part 2002C applies a known correspondence point search method based on the narrow-angle data D12 and data on the shape of the portion of the 3D model D13 corresponding to the portion of the comparison target object specified for the narrow-angle data D12 on a per-narrow-angle data D12 basis. This enables the narrow angle data position estimation part 2002C to estimate the position and orientation of the sensor 120 corresponding to the narrow angle data D12 in the coordinate system of the 3D model D13 based on the combination of the plurality of corresponding points obtained as a result of the search.

[0118] In this case, the wide-angle data position estimation part 2003A can perform known SfM processing using a wide-angle data group including a large number of wide-angle data D21 of the inspection target. This allows the wide-angle data position estimation part 2003A to generate a three-dimensional model of the inspection target and estimate the position and orientation of the sensor 110 corresponding to each of the wide-angle data D21. At this time, the coordinate system in which the three-dimensional model and the position and orientation of the sensor 120 corresponding to each of the wide-angle data D21 of the inspection target are represented is processed to be the same as the coordinate system of the 3D model D13 of the comparison target.For example, the coordinate system of the three-dimensional model of the inspection target is aligned with the coordinate system of the 3D model D13 based on the comparison between the three-dimensional model of the inspection target obtained by SfM processing and the 3D model D13 of the comparison target. This allows the comparison target data search part 2003D to search for the comparison target narrow angle data D26 corresponding to the narrow angle data D22 based on the narrow angle position information data group DG16 and the narrow angle position information data D25 represented by the coordinate system of the 3D model D13.

[0119] In the above-described embodiment, instead of the comparison inspection or as a preliminary inspection before the detailed comparison inspection, the user can perform inspection of a single inspection target without using the comparison target (hereinafter referred to as "single inspection") based on the narrow angle data D22 of the inspection target. For example, the presence or absence of scratches, dents, or missing components can also be inspected using a single inspection target.

[0120] In this case, the comparison target data generation part 2002 is omitted. The wide-angle data position estimation part 2003A can perform known SfM processing using a wide-angle data group including a large number of wide-angle data D21 of the inspection target. This enables the wide-angle data position estimation part 2003A to generate a three-dimensional model of the inspection target and estimate the position and orientation of the sensor 120 according to each of the wide-angle data D21. Furthermore, instead of the comparison target data search part 2003D, a function part is provided that specifies a portion of the inspection target corresponding to the narrow-angle data D22 based on the narrow-angle position information data D25 and links the narrow-angle data D22 to the specified portion using a database or the like.This allows, for example, the display processing part 2003E to link the narrow angle data D22 with the portion corresponding to the narrow angle data D22 on the image representing the overall shape of the inspection target area of the inspection target object when the narrow angle data D22 and the image representing the overall shape of the inspection target area are displayed on the display device 208. Therefore, the user can identify the portion of the inspection target object represented by the narrow angle data D22. Accordingly, the user can identify the detailed shape of each of the portions of the inspection target object based on the narrow angle data D22 while identifying the overall shape of the inspection target object based on the wide angle data D21. As a result, the user can perform the individual inspection more appropriately.

[0121] In the embodiment and the modifications and variations thereof, instead of or in addition to displaying the narrow angle data D22 and the comparison target angle data D26 on the display device 208, a combination of the narrow angle data D22 and the comparison target angle data D26 may be transmitted externally. For example, the inspection support device 200 transmits a combination of the narrow angle data D22 and the comparison target angle data D26 to the terminal device 300. Thus, the user can obtain, for example, information about a combination of the narrow angle data D22 and the comparison target angle data D26 by utilizing the function of the server-side inspection support device 200 via the user-side terminal device 300. Therefore, the user can perform the comparison test without being restricted by the location where the inspection support device 200 is installed.

[0122] In the above-described embodiment and the examples of modifications and variations thereof, the sensor device 100 may be incorporated into the inspection support device 200 or the terminal device 300. For example, the inspection support device 200 and the terminal device 300 are a smartphone and a tablet terminal equipped with a camera, a LiDAR, and the like as the sensor device 100. [Operation]

[0123] Next, the operation of the information processing apparatus according to the present embodiment will be described.

[0124] In the present embodiment, the information processing device includes a first acquisition part, a second acquisition part, and a linking part. The information processing device is, for example, the above-described inspection support device 200. The first acquisition part and the second acquisition part are, for example, the above-described data acquisition part 2001. The linking part is, for example, the above-described comparison target data search part 2003D. The functions of the first acquisition part, the second acquisition part, and the linking part can be implemented by a program that causes an information processing device to execute a first acquisition step, a second acquisition step, and a linking step, which correspond to the first acquisition part, the second acquisition part, and the linking part, respectively.Specifically, the first acquisition part acquires first measurement data representing a shape of a relatively narrow range of a first target object, and second measurement data representing a shape of a relatively wide range of the first target object and for which a relationship in terms of position and orientation between sensors at a time of acquisition is predefined with respect to the first measurement data. The first target object is, for example, the inspection target object described above. The first measurement data is, for example, the narrow-angle data D22 described above. The sensors include, for example, sensor 110 and sensor 120. The second measurement data is, for example, the wide-angle data D21 described above. The second acquisition part acquires a third measurement data group and a fourth measurement data group.The third measurement data group is a collection of third measurement data, each representing a shape of a relatively narrow range of a second target object to be compared with the first target object, and the fourth measurement data group is a collection of fourth measurement data, each representing a shape of a relatively wide range of the second target object and for which a relationship in position and orientation between the sensors at the time of acquiring the third measurement data and the fourth measurement data with respect to the third measurement data is predefined. The second target object is, for example, the comparison target object described above. The third measurement data is, for example, the narrow-angle data D12 described above. The sensors include, for example, sensor 110 and sensor 120. The fourth measurement data is, for example, the wide-angle data D11 described above. The third measurement data group is, for example, the narrow-angle data group DG12 described above.The fourth measurement data group is, for example, the wide-angle data group DG11 described above. Then, based on the second measurement data and the fourth measurement data group, the linking part links the first measurement data to the third measurement data, wherein the third measurement data corresponds to a portion of the second target object that is the same portion (common portion) (ie, represents the same portion) as the portion of the first target object that corresponds to the first measurement data.

[0125] For example, when using measurement data representing the shape of a relatively narrow area, such as image data with a relatively narrow viewing angle, the detailed shape of the area can be reflected in the data. Therefore, the condition of the area of the target object can be determined in more detail.

[0126] However, for measurement data representing the shape of a relatively narrow range, it may not be possible to determine which area of the entire inspection target range of the target object this data corresponds to. Therefore, for example, measurement data of a specific inspection target range cannot be properly extracted from the measurement data set, and as a result, there is a possibility that the inspection cannot be properly performed.

[0127] For example, in a case where measurement data representing a shape of a relatively wide range, such as image data with a relatively wide viewing angle, is used, it is easy to determine a portion corresponding to a range of data in the entire inspection target area of the target object.

[0128] However, for measurement data representing a shape of a relatively large area, it is less likely that a detailed shape will be reflected in the data. Therefore, the condition of the area included in the measurement data cannot be determined in detail, and there is a possibility that the inspection cannot be performed appropriately.

[0129] In contrast, the information processing device can detect the overall shape of the inspection target area of the first and second target objects based on the second measurement data and the fourth measurement data group, which represent, for example, the shapes of relatively wide areas. Further, the information processing device can identify a portion of the first target object represented by the first measurement data that represents a shape of a relatively narrow area and a portion of the second target object represented by the third measurement data that represents a shape of a relatively narrow area, for example, based on a relationship in position and orientation between the sensors. Therefore, the information processing device can distribute the first and third measurement data corresponding to the same portions (common portions) of the first and second target objects.Therefore, the user can perform the comparison test more appropriately by using the first and third measurement data, which represent the relatively narrow ranges of the first and second target objects.

[0130] In the present embodiment, the first target object and the second target object may be the same object at different times. The first target object and the second target object may be different objects of the same configuration.

[0131] Thus, the information processing apparatus can support checking a change of the same target object in time series and can also support checking a difference between different target objects having the same design.

[0132] In the present embodiment, the information processing device may include a first estimation part, a second estimation part, a third estimation part, and a fourth estimation part. The first estimation part is, for example, the above-described SfM processing part 2002A. The second estimation part is, for example, the above-described narrow-angle data position estimation part 2002C. The third estimation part is, for example, the above-described wide-angle data position estimation part 2003A. The fourth estimation part is, for example, the above-described end-angle data position estimation part 2003C. Specifically, the first estimation part estimates a position and an orientation of a sensor corresponding to the fourth measurement data on a per-fourth measurement data basis based on the fourth measurement data group.The second estimation part estimates a position and an orientation of a sensor corresponding to the third measurement data on a per-third measurement data basis based on the estimation results of the first estimation part. The third estimation part estimates the position and orientation of a sensor corresponding to the second measurement data based on the fourth measurement data group and the estimation results of the first estimation part. The fourth estimation part estimates a position and an orientation of a sensor corresponding to the first measurement data based on the estimation results of the third estimation part. The linking part may link the first measurement data with the third measurement data corresponding to a portion of the second target object that is the same portion (common portion) as a portion of the first target object corresponding to the first measurement data (i.e.which represents the same section) based on the estimation results of the position and orientation of the sensor corresponding to the first measurement data by the fourth estimation part and the estimation results of the position and orientation of the sensor corresponding to the third measurement data on a per third measurement data basis by the second estimation part.

[0133] Thus, the information processing device can identify the position and orientation of the sensor corresponding to the first and third measurement data, respectively, and link the first and third measurement data corresponding to the same portions (common portions) of the first and second target objects.

[0134] In the present embodiment, the first estimation part may estimate the position and the orientation of the sensor corresponding to the fourth measurement data on a per fourth measurement data basis based on the relationship between the fourth measurement data each representing the same portion (common portion) of the second target object included in the fourth measurement data group.

[0135] This enables the information processing device to estimate the position and orientation of the sensor according to the fourth measurement data on a per-fourth measurement data basis from the fourth measurement data group.

[0136] In the present embodiment, the information processing device may include a detection part. The detection part is, for example, the difference detection part 2003F. Specifically, the detection part may detect a difference between the same portions (common portions) of the first target object and the second target object based on the first measurement data and the third measurement data linked by the linking part.

[0137] This allows the user to perform a comparison check using the detection result. Therefore, the information processing device can improve user convenience and the efficiency of the comparison check.

[0138] In the present embodiment, the detection part may select portions between which a difference is detected among the same portions (common portions) of the first target object and the second target object according to at least one of the content of the difference and the degree of the difference.

[0139] Thus, for example, the user can be provided with information only about a combination of the first and third measurement data corresponding to a section defined by at least one of the content of the difference and the degree of difference between the same sections (common sections) of the first and second target objects. Therefore, the information processing device can improve the user's convenience and the efficiency of the comparison test.

[0140] In the present embodiment, the detection part may label the first measurement data and the third measurement data linked by the linking part to represent respective features, and detect a difference between the same portions (common portions) of the first target object and the second target object based on a difference between the labels of the data.

[0141] Thus, the information processing device can detect the difference between the same portions (common portions) of the first and second target objects.

[0142] In the present embodiment, the information processing apparatus may include a display part configured to display the first measurement data and the third measurement data linked by the linking part.

[0143] Thus, the information processing device can provide the user with information about the first and third measurement data corresponding to the same portions (common portions) of the first and second target objects through a visual method.

[0144] In the present embodiment, the display part may display the first measurement data and the third measurement data linked by the linking part, highlighting a difference in shape.

[0145] Thus, the information processing device can more easily detect the difference in shape between the first and third measurement data corresponding to the same portions (common portions) of the first and second target objects. Therefore, the information processing device can improve user convenience and the efficiency of the comparison test.

[0146] In the present embodiment, the display part may display an image representing the overall shape of the inspection target area of the first target object and the second target object.

[0147] Thus, the user can perform the comparison test while checking not only the combination of measurement data for the specific sections, but also the image representing the overall shape of the inspection target area of the first and second target objects. Therefore, the information processing device can improve the user's convenience and the efficiency of the comparison test.

[0148] In the present embodiment, the display part may display the first measurement data and the third measurement data linked by the linking part in linkage with the portions of the entire image.

[0149] This allows the user to easily identify the positions of the sections corresponding to the first and third measurement data displayed on the display part by comparing these sections with sections in the image representing the overall shape of the inspection area of the first and second objects. Therefore, the information processing device can improve user convenience and increase the efficiency of the comparison inspection.

[0150] In the present embodiment, the display part may display information about a difference between the same portions (common portions) of the first target object and the second target object corresponding to the first measurement data and the third measurement data linked by the linking part (ie, represented by these data), in linkage with a portion of the image representing the overall shape of the inspection target area of the first target object and the second target object.

[0151] This allows the user to easily identify the positions of the portions of the first and second objects corresponding to the first and third measurement data displayed on the display part by comparing these portions with the portions of the image representing the overall shape of the first and second objects. Therefore, the information processing device can improve user convenience and increase the efficiency of the comparison test.

[0152] In the present embodiment, the information processing device may include a first acquisition part, a storage part, and a linking part. Specifically, the first acquisition part acquires a first measurement data group and a second measurement data group. The first measurement data group is a collection of first measurement data representing a shape of a relatively narrow range of a first target object, and the second measurement data group is a collection of second measurement data representing a shape of a relatively wide range of the first target object and for which a relationship in position and orientation between the sensors at the time of acquisition of the first measurement data and the second measurement data with respect to the first measurement data is predefined. The storage part stores three-dimensional shape data of an inspection target region of a second target object as a comparison target of the first target object and a third measurement data group.The third measurement data group is a collection of third measurement data representing a shape of the second target object and for which a correspondence relationship with a portion of the second target object or the three-dimensional shape data is predefined. The linking part links the first measurement data and the third measurement data based on the second measurement data group and the three-dimensional shape data of the second target object, wherein the first measurement data represents a portion of the first target object and the third measurement data represents a portion of the second target object that is a same portion (common portion) as the portion of the first target object (ie, represents the same portion).

[0153] Accordingly, the information processing device can detect the overall shape of the inspection target area of the first and second target objects, for example, based on the second measurement data group and the three-dimensional shape data. Further, the information processing device can identify a portion of the first target object represented by the first measurement data that represents a shape of a relatively narrow area and a portion of the second target object represented by the third measurement data that represents a shape of a relatively narrow area, for example, based on a relationship in position and orientation between the sensors. Therefore, the information processing device can link the first and third measurement data corresponding to the same portions (common portions) of the first and second target objects.Therefore, the user can perform the comparison test more appropriately by using the first and third measurement data, which represent the relatively narrow ranges of the first and second target objects.

[0154] In the present embodiment, the information processing device may include a first acquisition part and a linking part. Specifically, the first acquisition part acquires a first measurement data group and a second measurement data group, which is a collection of first measurement data representing a shape of a relatively wide range of a first target object and a collection of second measurement data representing a shape of a relatively wide range of the first target object and for which a relationship in terms of position and orientation between sensors at the time of acquisition of the first measurement data and the second measurement data with respect to the first measurement data is predefined. Then, the linking part links the second measurement data with a portion of the first target object corresponding to the second measurement data (i.e., represented by the data) based on the first measurement data group.

[0155] With this configuration, the information processing device can grasp the overall shape of the inspection target area of the first target object, for example, based on the second measurement data group. Furthermore, the information processing device can identify a portion of the first target object represented by the first measurement data representing a shape in a relatively narrow range, for example, based on the relationship between the positions and orientations of the sensors. Therefore, the information processing device can associate the second measurement data with the portion of the first target object according to the second measurement data. Therefore, the user can identify the portion of the first target object corresponding to the first measurement data representing the relatively narrow range of the first target object, and can then more appropriately perform the individual inspection using the first measurement data.

[0156] Although the embodiments have been described in detail, the present disclosure is not limited to such specific embodiments, and various modifications and changes may be made within the scope of the spirit described in the claims.

[0157] Finally, the present application claims priority to Japanese Patent Application No. 2022-156595, filed on September 29, 2022, the entire contents of which are incorporated herein by reference. DESCRIPTION OF REFERENCE SYMBOLS 100 sensor device 110 Sensor 120 sensors 200 test support device 300 terminal devices 2001 Data collection part 2002 Comparison target data generation part 2002A SfM processing part 2002B memory section 2002C Narrow angle data position estimation part 2002D Data set generation part 2002E memory part 2003 Comparative test support part 2003A Wide-angle data position estimation part 2003B memory part 2003C Narrow angle data position estimation part 2003D Comparison Target Data Search Part 2003E Display processing part 2003F Difference detection part D11 wide-angle data D12 narrow angle data D13 three-dimensional model D14 Wide-angle position information data D15 Sensor relative position information data D16 narrow angle position information data D21 wide-angle data D22 narrow angle data D23 wide-angle position information data D24 sensor relative position information data D25 Narrow angle position information data D26 Comparison target angle data DG11 wide-angle data group DG12 Narrow Angle Data Group DG14 Wide-angle position information data group DG16 narrow angle position information data group DS17 reference data set SYS test support system QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2016-99633

[0003] JP 2022-156595

[0157]

Claims

[1] Information processing device comprising: a first sensing part configured to acquire first measurement data and second measurement data, wherein the first measurement data represents a shape of a relatively narrow range of a first target object and the second measurement data represents a shape of a relatively wide range of the first target object, and for which a relationship in terms of position and orientation between sensors at a time of acquiring the first measurement data and the second measurement data with respect to the first measurement data is predefined; a second sensing part configured to acquire a third measurement data group and a fourth measurement data group, wherein the third measurement data group is a collection of third measurement data, the third measurement data representing a shape of a relatively narrow range of a second target object, the second target object being a comparison target of the first target object, the fourth measurement data group is a collection of fourth measurement data, the fourth measurement data representing a shape of a relatively wide range of the second target object, and for which a relationship in terms of position and orientation between the sensors at a time of acquiring the third measurement data and the fourth measurement data with respect to the third measurement data is predefined; and a linking part configured to link the first measurement data to the third measurement data based on the second measurement data and the fourth measurement data group, wherein the third measurement data represents a portion of the second target object that is the same portion as a portion of the first target object that corresponds to the first measurement data. [2] The information processing apparatus according to claim 1, wherein the first target object and the second target object are the same object at different times or different objects of the same configuration. [3] The information processing apparatus according to claim 2, further comprising: a first estimation part configured to estimate a position and an orientation of a sensor corresponding to the fourth measurement data on a per fourth measurement data basis based on the fourth measurement data group; a second estimation part configured to estimate a position and an orientation of a sensor corresponding to the third measurement data on a per-third measurement data basis based on estimation results of the first estimation part; a third estimation part configured to estimate a position and an orientation of a sensor corresponding to the second measurement data based on the fourth measurement data group and the estimation results of the first estimation part; and a fourth estimation part configured to estimate a position and an orientation of a sensor corresponding to the first measurement data based on estimation results of the third estimation part, wherein the linking part links the first measurement data to the third measurement data, the third measurement data representing the portion of the second target object that is the same portion as the portion of the first target object, based on estimation results of the position and orientation of the sensor corresponding to the first measurement data by the fourth estimation part and estimation results of the position and orientation of the sensor corresponding to the third measurement data on a per third measurement data basis by the second estimation part. [4] The information processing apparatus according to claim 3, wherein the first estimating part estimates the position and the orientation of the sensor corresponding to the fourth measurement data on a per fourth measurement data basis based on a relationship between the fourth measurement data each representing a same portion of the second target object included in the fourth measurement data group. [5] Information processing apparatus according to any one of claims 1 to 4, further comprising: a detection part configured to detect a difference between equal portions of the first target object and the second target object based on the first measurement data and the third measurement data linked by the linking part. [6] The information processing apparatus according to claim 5, wherein the detection part selects, from the same portions of the first target object and the second target object, portions between which a difference is detected according to at least one of a content of the difference or a degree of the difference. [7] The information processing apparatus according to claim 5, wherein the detection part labels the first measurement data and the third measurement data linked by the linking part to represent features, and detects a difference between the same portions of the first target object and the second target object based on a difference between the labels of the first measurement data and the second measurement data. [8] Information processing apparatus according to any one of claims 1 to 4, further comprising: a display part configured to display the first measurement data and the third measurement data linked by the linking part. [9] The information processing apparatus according to claim 8, wherein the display part displays the first measurement data and the third measurement data linked to each other by the linking part, emphasizing a difference in shape. [10] The information processing apparatus according to claim 8, wherein the display part displays an image representing an overall shape of an inspection target area of the first target object and the second target object. [11] The information processing apparatus according to claim 10, wherein the display part displays the first measurement data and the third measurement data linked by the linking part in association with a portion of the image. [12] The information processing apparatus according to claim 10, wherein the display part displays, in association with a portion of the image, information about a difference between like portions of the first target object and the second target object represented by the first measurement data and the third measurement data linked by the linking part. [13] Information processing device comprising: a first sensing part configured to acquire a first measurement data group and a second measurement data group, wherein the first measurement data group is a collection of first measurement data, the first measurement data representing a shape of a relatively narrow range of a first target object, and the second measurement data group is a collection of second measurement data, the second measurement data representing a shape of a relatively wide range of the first target object and for which a relationship in terms of position and orientation between sensors at a time of acquiring the first measurement data and the second measurement data with respect to the first measurement data is predefined; a storage part configured to store three-dimensional shape data of a second target object as a comparison target of the first target object and a third measurement data group, wherein the third measurement data group is a collection of third measurement data, the third measurement data representing a shape of the second target object and for which a correspondence relationship with a portion of the second target object or the three-dimensional shape data is predefined; and a linking part configured to link the first measurement data and the third measurement data based on the second measurement data group and the three-dimensional shape data of the second target object, wherein the first measurement data represents a portion of the first target object and the third measurement data represents a portion of the second target object that is a same portion as the portion of the first target object. [14] Information processing device comprising: a first sensing part configured to acquire a first measurement data group and a second measurement data group, wherein the first measurement data group is a collection of first measurement data, the first measurement data representing a shape of a relatively wide area of a first target object, the second measurement data group is a collection of second measurement data, the second measurement data representing a shape of a relatively wide area of the first target object, and for which a relationship in terms of position and orientation between sensors at a time of acquiring the first measurement data and the second measurement data with respect to the first measurement data is predefined; and a linking part configured to link the second measurement data to a portion of the first target object represented by the second measurement data based on the first measurement data group. [15] Program for causing an information processing device to execute: a first acquisition step of acquiring first measurement data and second measurement data, wherein the first measurement data represents a shape of a relatively narrow range of a first target object and the second measurement data represents a shape of a relatively wide range of the first target object and for which a relationship in terms of position and orientation between sensors at a time of acquiring the first measurement data and the second measurement data with respect to the first measurement data is predefined; a second acquisition step of acquiring a third measurement data group and a fourth measurement data group, wherein the third measurement data group is a collection of third measurement data, wherein the third measurement data represents a shape of a relatively narrow range of a second target object, wherein the second target object is a comparison target of the first target object, the fourth measurement data group is a collection of fourth measurement data, wherein the fourth measurement data represents a shape of a relatively wide range of the second target object and for which a relationship in terms of position and orientation between the sensors at a time of acquiring the third measurement data and the fourth measurement data with respect to the third measurement data is predefined; and a linking step of linking the first measurement data to the third measurement data based on the second measurement data and the fourth measurement data group, wherein the third measurement data represents a portion of the second target object that is the same portion as a portion of the first target object that corresponds to the first measurement data.

Images