Information processing method, information processing apparatus, and information processing system

Abstract

The camera generates a learning image by photographing an object point, and acquires a plurality of learning parameters related to the position and posture of the camera when the learning image is photographed (S1). A model is generated which takes the plurality of learning parameters as input and outputs a first displacement amount of an object block corresponding to the object point displayed in the learning image with respect to a reference position in the learning image (S2). The camera generates a measurement image by photographing the object point, and acquires a plurality of measurement parameters related to the position and posture of the camera when the measurement image is photographed (S3). A second displacement amount of the object block corresponding to the object point displayed in the measurement image with respect to the reference position in the measurement image is output by inputting the plurality of measurement parameters into the above model (S4). A displacement obtained by subtracting the second displacement amount from the displacement of the object block in the measurement image is output (S5, S6).

Description

Technical Field

[0001] This invention relates to information processing methods, information processing apparatus, and information processing systems. Background Technology

[0002] There is a technique that uses images obtained by photographing structures such as bridges to measure the displacement of those structures (see Patent Document 1).

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: International Publication No. 2019 / 097576 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] However, there is a problem that if the camera used for shooting vibrates, it will cause blurring or displacement in the captured image, which will hinder the measurement of the displacement of the structure.

[0008] Therefore, the present invention provides an information processing method, etc., for measuring the displacement of structures in a more appropriate manner.

[0009] Methods for solving problems

[0010] One aspect of the information processing method of the present invention is as follows: A plurality of first images generated by a camera capturing object points based on multiple positions and postures, and a plurality of first parameters related to the position and posture of the camera when each of the plurality of first images is captured; a generation model is generated that takes the plurality of first parameters as input and outputs a first displacement of the object block corresponding to the object point displayed in each of the plurality of first images, relative to a reference position in the first image; a plurality of second images generated by the camera capturing the object points, and a plurality of second parameters related to the position and posture of the camera when each of the plurality of second images is captured; a second displacement of the object block corresponding to the object point displayed in the second image, output by inputting the plurality of second parameters of each of the plurality of second images to the model, relative to a reference position in the second image; and a displacement obtained by subtracting the second displacement from the displacement of the object block in the plurality of second images.

[0011] Furthermore, these general or specific methods can be implemented by systems, devices, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, or by any combination of systems, devices, integrated circuits, computer programs, and recording media.

[0012] The effects of the invention

[0013] The information processing method of the present invention can more appropriately measure the displacement of structures. Attached Figure Description

[0014] Figure 1 This is a schematic diagram illustrating the measurement of displacement of a structure based on a processing device in an embodiment.

[0015] Figure 2 This is a block diagram illustrating the functional configuration of the processing device in the implementation method.

[0016] Figure 3 This is an explanatory diagram showing a first example of a captured image in an embodiment.

[0017] Figure 4 This is an explanatory diagram illustrating the method for generating the model in the implementation method.

[0018] Figure 5 This is an explanatory diagram showing a second example of a captured image in the embodiment.

[0019] Figure 6 This is a flowchart illustrating the process of positioning the target block in the implementation method.

[0020] Figure 7 This is a schematic diagram illustrating the measurement of displacement of a structure based on a processing device in a variation of embodiment 1.

[0021] Figure 8 This is a schematic diagram illustrating an example of a captured image in a variation of embodiment 1.

[0022] Figure 9 This is a schematic diagram illustrating a camera with a fixed sensor in a modified example 2 of the implementation method.

[0023] Figure 10 This is an explanatory diagram illustrating the method for generating the model in Variation 2 of the implementation method. Detailed Implementation

[0024] (Knowledge that forms the basis of this invention)

[0025] Regarding the technique for measuring the displacement of structures described in the "Background Art" section, the inventors have discovered the following problems.

[0026] There exists a technique for inspecting structures such as bridges, which uses high-precision cameras to capture images of the structure from a distance. Image-based displacement measurement offers the advantages of non-contact, simultaneous measurement of multiple points, contributing to labor-saving surveying operations for infrastructure maintenance and management, reducing the equipment and power required for surveying, and shortening the time required.

[0027] When a camera or its support (e.g., a tripod) is struck or propelled by an object or fluid (liquid or gas), the camera may vibrate or shift in position (hereinafter referred to as vibration, etc.). When such vibration occurs, the position of the subject shown in the image captured by the camera deviates from its position when the image was taken with the camera in the absence of vibration. If such an image is used to measure the displacement of a structure, the measurement result will be incorrect.

[0028] Thus, if the camera used for shooting vibrates, it will cause a positional shift in the captured image, which will hinder the measurement of the structure's displacement.

[0029] Therefore, the present invention provides an information processing method, etc., for measuring the displacement of structures in a more appropriate manner.

[0030] One aspect of the information processing method of the present invention is as follows: A plurality of first images generated by a camera capturing object points based on multiple positions and postures, and a plurality of first parameters related to the position and posture of the camera when each of the plurality of first images is captured; a generation model that takes the plurality of first parameters as input and outputs a first displacement of the object block corresponding to the object point displayed in each of the plurality of first images relative to a reference position in the first image; a plurality of second images generated by the camera capturing the object points, and a plurality of second parameters related to the position and posture of the camera when each of the plurality of second images is captured; a second displacement of the object block corresponding to the object point displayed in the second image relative to a reference position in the second image, output by inputting the plurality of second parameters of each of the plurality of second images to the model; and a displacement obtained by subtracting the second displacement from the displacement of the object block in the plurality of second images.

[0031] According to the above method, a model can be generated using the image actually captured by the camera and the parameters at the time of capture. Using the generated model, the position of the object block corresponding to the object point can be corrected and output. The object point is the point on which the displacement is measured. The above model outputs the displacement of the object block when the camera's position and posture are determined at the time of capture. The generation of the above model and the output of the displacement using the above model do not require complex geometric calculations considering the camera's position and posture; for example, they can be implemented using relatively simple algebraic calculations. This contributes to reducing computer resources and power consumption related to obtaining the displacement of the object block. Thus, according to the above information processing method, the displacement of structures can be measured more appropriately.

[0032] For example, each of the plurality of first images may display a plurality of reference points, and each of the plurality of first images may contain a plurality of reference blocks that correspond one-to-one with the plurality of reference points displayed in the first image. The plurality of first parameters may contain the displacement of the plurality of reference blocks in the first image. The second image may display the plurality of reference points, and the second image may contain the plurality of reference blocks corresponding to the plurality of reference points displayed in the second image. The plurality of second parameters may contain the displacement of the plurality of reference blocks in the second image.

[0033] According to the above method, the position of the reference block corresponding to the reference point displayed in the image along with the object point can be used as a parameter to correct the displacement of the object block. The reference point is the point that serves as a reference during displacement measurement. Therefore, in correcting the displacement of the object block, no information other than the image is required; that is, the amount of information needed for correction can be reduced. Thus, according to the above information processing method, the displacement of a structure can be measured more appropriately based on less information.

[0034] For example, the plurality of first parameters may include displacement data or angular velocity data of the camera when each of the plurality of first images is captured, obtained from the output value of the displacement sensor or the gyroscope sensor fixed to the camera, and the plurality of second parameters may include displacement data or angular velocity data of the camera when the second image is captured, obtained from the output value of the displacement sensor or the gyroscope sensor.

[0035] According to the above method, the displacement or angular velocity of the camera measured by the sensor can be used as a parameter to correct the displacement of the object block. In this case, no reference point is used in the correction of the object block's displacement; therefore, it has the advantage of being able to be performed even when a suitable reference point cannot be set in the image. Thus, according to the above information processing method, even when a suitable reference point cannot be set in the image, the displacement of the structure can be measured more appropriately.

[0036] For example, when acquiring the plurality of first images, the plurality of first images can be generated by sequentially photographing the object points when the position changes within a specified distance relative to the reference position of the camera, or when the posture changes within a specified angle relative to the reference posture of the camera.

[0037] According to the above method, a model can be generated using images captured sequentially by the camera as the camera's position and pose change, along with the parameters at the time of capture. Since the camera's position and pose are within specified limits relative to a reference position and reference pose, the offset of object points in the images can be minimized, and the accuracy of the displacement output by the model can be improved. Therefore, based on the above information processing method, the displacement of a structure can be measured with higher accuracy.

[0038] For example, the plurality of first images can also be generated by sequentially capturing the object points while the position and posture of the camera are vibrating.

[0039] According to the above method, a model can be generated using images sequentially captured by the camera while the camera's position and orientation are vibrating, along with the parameters taken during those captures. Since the camera's position and orientation are vibrating at this time, the offset of object points in the images can be minimized, improving reproducibility and the accuracy of the displacement output by the model. Therefore, based on the above information processing method, the displacement of structures can be measured with higher accuracy and greater appropriateness.

[0040] For example, the camera can be vibrated by a vibration device fixed to the camera, and the object point can be captured while the camera's position and posture are vibrating based on the vibration applied to the camera, thereby generating the plurality of first images.

[0041] According to the above method, the camera's position and orientation can be vibrated using a vibration device. Therefore, there is no need to use other means to vibrate the camera, making it easier and more effective to vibrate the camera and correct the displacement of the object block. Consequently, based on the above information processing method, the displacement of the structure can be measured more easily and appropriately.

[0042] For example, when the vibration device imparts vibration to the camera, it may impart a vibration comprising a component of movement in three mutually orthogonal directions and a component of rotation about three mutually orthogonal axes.

[0043] According to the above method, by applying vibration to the camera through the vibration initiation device, vibrations incorporating all possible translational and rotational components that can occur in three-dimensional space can be generated. Therefore, a model capable of outputting the displacement of the object block under various vibration conditions can be produced, enabling higher-precision correction of the object block's displacement. Consequently, based on the aforementioned information processing method, the displacement of a structure can be measured with higher accuracy and greater appropriateness.

[0044] For example, when obtaining the plurality of first parameters, instead of obtaining the plurality of first parameters when the camera is not subjected to vibration by the vibration initiation device, the plurality of first parameters when the camera is subjected to vibration by the vibration initiation device can be obtained.

[0045] According to the above method, by not acquiring parameters that contribute little to model generation and are used when the camera is vibrated without a vibration trigger, the amount of information acquired can be reduced, and the storage capacity required by the storage device can be reduced. Therefore, based on the above information processing method, displacement measurement of structures can be achieved using a smaller storage device.

[0046] For example, when the camera is vibrated by the vibration device, a vibration with a defined amplitude or a defined frequency is applied. When the plurality of first parameters are obtained, the vibration of the position and the vibration of the posture of the camera with the defined amplitude or the defined frequency are extracted and obtained.

[0047] Based on the above method, it is possible to generate a model that can output the displacement of an object block using vibrations of a defined amplitude or frequency. It is assumed that even if vibrations different from the defined amplitude or frequency occur in the structure or camera, a model that corrects for the displacement of the object block can be generated without being affected by these vibrations. Therefore, according to the above information processing method, even when the structure or camera is vibrating, the displacement of the structure can be measured more appropriately.

[0048] For example, each of the plurality of first images may display a plurality of reference point candidates, and each of the plurality of first images may contain a plurality of reference block candidates corresponding to the plurality of reference point candidates displayed in the first image. When generating the model, the plurality of first images containing a plurality of trial reference block groups selected from the plurality of reference block candidates based on a plurality of selection modes are used as the plurality of reference blocks to generate a plurality of trial models as the model. The trial model with the smaller error in the first displacement amount of the output of the plurality of trial models is generated as the model.

[0049] Based on the above method, more candidate reference points can be pre-set than actually used, and the model with smaller errors can be selected and used during model generation to correct the displacement of the object blocks. Therefore, according to the above information processing method, the displacement of structures can be measured with higher accuracy and more appropriateness.

[0050] For example, multiple third images can be captured simultaneously by a second camera (which changes position and posture as a first camera) and the first camera, each displaying multiple reference points. Each of the multiple third images contains multiple reference blocks that correspond one-to-one with the multiple reference points displayed in the third image. Multiple parameters include the positions of the multiple reference blocks in the third image. A second image displays the multiple reference points and contains multiple reference blocks corresponding to the multiple reference points displayed in the second image. Multiple second parameters include the positions of the multiple reference blocks in the second image.

[0051] According to the above method, multiple cameras whose position and pose change in an integrated manner can be used, and a model that can output the displacement of an object block can be generated using reference blocks contained in the images generated by these multiple cameras. The shooting parameters of the multiple cameras can be different. Therefore, by effectively using multiple cameras, reference points can be set at various locations, and accuracy can be improved. Thus, according to the above information processing method, the displacement of a structure can be measured with higher accuracy and more appropriateness.

[0052] Furthermore, one aspect of the information processing method of the present invention is as follows: obtaining multiple first images generated by a camera by capturing object points based on multiple positions and postures, and multiple first parameters related to the position and posture of the camera when each of the multiple first images is captured; generating a generation model that takes the multiple first parameters as input and outputs a first displacement of the object block corresponding to the object point displayed in each of the multiple first images relative to a reference position in the first image; obtaining multiple second images generated by the camera by capturing the object points, and multiple second parameters related to the position and posture of the camera when each of the multiple second images is captured; obtaining a second displacement of the object block corresponding to the object point displayed in the second image relative to a reference position in the second image, which is output by inputting the multiple second parameters into the model; and outputting the position obtained by subtracting the second displacement from the position of the object block in the second image.

[0053] According to the above method, a model can be generated using the image actually captured by the camera and the parameters at the time of capture. Using the generated model, the position of the object block corresponding to the object point can be corrected and output. The object point is the point on which the displacement is measured. The above model outputs the displacement of the object block when the camera's position and posture are determined at the time of capture. The generation of the above model and the output of the displacement using the above model do not require complex geometric calculations considering the camera's position and posture; for example, they can be implemented using relatively simple algebraic calculations. This contributes to reducing computer resources and power consumption related to obtaining the displacement of the object block. Thus, according to the above information processing method, the position of a structure can be measured more appropriately.

[0054] Furthermore, one aspect of the information processing apparatus of the present invention is an information processing apparatus comprising: an acquisition unit; a generation unit; and an output unit, wherein the acquisition unit acquires a plurality of first images generated by a camera by capturing object points based on a plurality of positions and postures, and a plurality of first parameters related to the position and posture of the camera when each of the plurality of first images is captured; the generation unit generates a model that takes the plurality of first parameters as input and outputs a first displacement amount of an object block corresponding to the object point displayed in each of the plurality of first images in the first image; the acquisition unit further acquires a second image generated by the camera by capturing the object points, and a plurality of second parameters related to the position and posture of the camera when the second image is captured; the output unit acquires a second displacement amount of the object block corresponding to the object point displayed in the second image, which is output by inputting the plurality of second parameters into the model, and outputs a displacement obtained by subtracting the second displacement amount from the displacement of the object block in the second image.

[0055] The same effect as the above information processing method can be achieved by following the above method.

[0056] Furthermore, one aspect of the information processing system of the present invention is an information processing system comprising: the aforementioned information processing apparatus; the camera, which generates the plurality of first images and the plurality of second images; and a vibration device fixed to the camera, which imparts vibration to the camera.

[0057] According to the above method, the camera is vibrated by a vibration device, thereby achieving the same effect as the information processing method described above.

[0058] In addition, these general or specific methods can be implemented by systems, devices, integrated circuits, computer programs or computer-readable recording media such as CD-ROMs, or by any combination of systems, devices, integrated circuits, computer programs and recording media.

[0059] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings.

[0060] Furthermore, the embodiments described below are all general or specific examples. The numerical values, shapes, materials, constituent elements, the arrangement and connection methods of the constituent elements, the steps, and the order of the steps shown in the following embodiments are examples and are not intended to limit the present invention. Moreover, constituent elements not described in the embodiments that show the highest-level concept in the following embodiments are described as arbitrary constituent elements.

[0061] (Implementation Method)

[0062] In this embodiment, an information processing device or the like that for more appropriately measuring the displacement of a structure will be described.

[0063] Figure 1 This is a schematic diagram illustrating the measurement of the displacement of the structure in this embodiment.

[0064] Figure 1 The image shows a bridge 1, a camera 21 that takes a picture of the bridge 1 from a distance, and a processing device 10. "Distance" refers to a location approximately 10 to 100 meters away from the bridge 1, but is not limited to this.

[0065] Bridge 1 is a bridge that is fixedly erected on the ground, and is an example of a structure. Bridge 1 has bridge legs extending from underground to the ground, and bridge trusses supported by the bridge legs. Bridge 1 is the object whose displacement is measured by the processing device 10 in this embodiment. In addition, the structure that is the object whose displacement is measured by the processing device 10 in this embodiment is not limited to bridge 1, but may also be a building, iron tower, statue, or other structure.

[0066] The parts of bridge 1 that are fixed relative to the ground (e.g., bridge piers) are assumed to be non-displaced relative to the ground, while the parts separated from the ground (e.g., bridge trusses) are assumed to be displaced relative to the ground. For example, due to the application of external forces to bridge 1, deformation (specifically, deflection or flexure) occurs in the steel bars or concrete constituting bridge 1, resulting in a relative displacement of the bridge trusses relative to the ground. For example, when a car or tram passes over the bridge trusses, the weight of the car or tram acts as an external force that causes the bridge trusses to displace in the downward direction. In addition, forces acting on the bridge trusses in various directions due to earthquakes or strong winds.

[0067] Here, the point in bridge 1 that is the object of displacement measurement is called the object point. Furthermore, the point in bridge 1 that is assumed to be non-displaced relative to the ground is called the reference point. Figure 1 In this example, one object point 31 and five reference points 33, 34, 35, 36, and 37 are defined. Additionally, multiple reference points need to be set; the more reference points, the higher the accuracy of the object point's displacement measurement. An arbitrary number of object points can be set. The displacement of each of the multiple object points is measured using multiple reference points.

[0068] Camera 21 is a photographing device that generates an image displaying bridge 1 by photographing bridge 1. Camera 21 is fixedly supported by tripod 23 on the ground. Tripod 23 is made of synthetic resin or metal, etc. The time interval for shooting by camera 21 is, for example, about 0.1 seconds or less. Camera 21 provides the generated image to processing device 10.

[0069] When camera 21 photographs bridge 1, if an external force is applied to camera 21 or tripod 23, vibrations will occur in camera 21 due to the bending and vibration of tripod 23. If camera 21 vibrates during shooting, bridge 1 will be displaced in the generated image, thus hindering accurate measurement of bridge 1's displacement. Therefore, it is necessary to suppress the influence of bridge 1's displacement in the image, based on vibrations generated by camera 21 during shooting, on the measurement results.

[0070] Vibration devices 25 and 26 are fixedly mounted on camera 21. Vibration devices 25 and 26 are devices that apply vibration to camera 21, respectively. Vibration devices 25 and 26 are, for example, vibration devices using solenoid coils.

[0071] The vibrations applied to the camera 21 by the vibration activation devices 25 and 26 can include components of movement in various directions or components of rotation about various axes. For example, the vibrations applied to the camera 21 by the vibration activation devices 25 and 26 may include components of movement in three mutually orthogonal directions and components of rotation about three mutually orthogonal axes. The vibration activation devices 25 and 26 can be used in the processing performed by the processing device 10 to suppress the effects of vibrations generated by the camera 21. Furthermore, while two examples of vibration activation devices have been provided, there may be one, or even three or more, vibration activation devices.

[0072] Furthermore, vibration starters 25 and 26 are not necessarily required. Any device that can vibrate the camera 21, such as a suitable object or fluid, can be used to make the camera 21 vibrate. Alternatively, the camera 21 can be made to vibrate by tapping it with a person's finger.

[0073] The processing device 10 is an information processing device for measuring the displacement of bridge 1. The processing device 10 acquires an image generated by camera 21 through shooting, and measures the displacement of bridge 1 based on the acquired image. At this time, the processing device 10 uses the displacement of the object point displayed in the image and the displacement of a reference point to perform correction processing to suppress the displacement of the object point in the image based on vibrations of camera 21 during shooting, etc. Thus, the processing device 10 measures the displacement of the object point, i.e., bridge 1, whose displacement based on vibrations of camera 21 during shooting, etc., has been suppressed.

[0074] The functions of the processing device 10 will now be described in more detail.

[0075] Figure 2 This is a block diagram illustrating the functional configuration of the processing device 10 in this embodiment.

[0076] like Figure 2As shown, the processing device 10 includes an acquisition unit 11, a generation unit 12, and an output unit 13. Each of the above-mentioned functional units can be implemented by the processor (e.g., CPU (Central Processing Unit), not shown) of the processing device 10 using memory (not shown) to execute a predetermined program.

[0077] The acquisition unit 11 is a functional unit that acquires an image generated by photographing the bridge 1, as well as multiple parameters related to the position and posture of the camera 21 when the image was captured. The acquisition unit 11 acquires the aforementioned image and multiple parameters during both the learning and measurement periods. Here, the learning period is the period for calculating the displacement of the object point used in the correction processing performed during the measurement period. Furthermore, the measurement period is the period for measuring the displacement of the object point and correcting the measured displacement using the displacement of the object point calculated during the learning period.

[0078] Furthermore, in the acquired image, the acquisition unit 11 determines the portion of the image containing the object point of a predetermined size as an object block, and the portion of the image containing the reference point of a predetermined size as a reference block. The process of determining the object block in the image is accomplished by determining the point containing the object point based on features surrounding the object point displayed in the captured image, and determining the portion containing that point of a predetermined size, which can be implemented using known techniques. The same applies to the reference point and the reference block.

[0079] Specifically, during the learning period, the acquisition unit 11 acquires multiple images (also called learning images or first images) generated by the camera 21 based on multiple positions and postures of the target point, and multiple parameters (also called learning parameters or first parameters) related to the position and posture of the camera 21 when each of the multiple learning images was captured. The multiple learning images are described using a subset of the multiple images captured by the camera 21 as an example, but the multiple learning images could also be all of the multiple images captured by the camera 21. The interval between the capture times of the multiple learning images is, for example, approximately 0.1 seconds to 0.01 seconds.

[0080] Here, multiple learning images each display multiple reference points. Each learning image contains multiple reference blocks that correspond one-to-one with the reference points displayed in that learning image. Furthermore, multiple learning parameters contain the positions of the multiple reference blocks in the learning image. The positions of the reference blocks in the learning image are, for example, coordinates in a two-dimensional coordinate system of x and y. The same applies below.

[0081] When camera 21 acquires multiple learning images, tripod 23 is fixed at the same position on the ground. The position and posture of camera 21 when no external force is applied to camera 21 and tripod 23 are respectively referred to as the reference position and reference posture of camera 21. Furthermore, multiple learning images are generated by sequentially photographing object points by camera 21 when its position changes within a predetermined distance relative to the reference position of camera 21, or when its posture changes within a predetermined angle relative to the reference posture of camera 21. The above description also includes the case where shooting is performed when the position changes within a predetermined distance relative to the reference position of camera 21 and when the posture changes within a predetermined angle relative to the reference posture of camera 21.

[0082] More specifically, when generating multiple learning images, multiple learning images are generated by sequentially capturing object points by the camera 21 while the position and pose of the camera 21 are vibrating.

[0083] As an example, the vibration of the position and orientation of camera 21 is applied by vibration-inducing devices 25 and 26. That is, when generating multiple learning images, camera 21 is vibrated by vibration-inducing devices 25 and 26 fixed to camera 21. Based on the vibration applied to camera 21, the object point is captured while the position and orientation of camera 21 are vibrating, thereby generating the image. Here, the case where vibration-inducing devices 25 and 26 are used to induce vibration in camera 21 is described as an example, but it is not limited to this. It may be only one of vibration-inducing devices 25 and 26, or if there are three or more vibration-inducing devices, it may be three or more vibration-inducing devices that induce vibration in camera 21. In addition, the vibration may be independent of the vibration-inducing devices.

[0084] Alternatively, the acquisition unit 11 may acquire multiple learning parameters when the camera 21 is vibrated by the vibration devices 25 and 26, but not when the camera 21 is not vibrated by the vibration devices 25 and 26. In other words, the acquisition unit 11 may acquire multiple learning parameters only when the camera 21 is vibrated by the vibration devices 25 and 26.

[0085] In addition, the acquisition unit 11 acquires multiple images (also called measurement images or second images) generated by the camera 21 through shooting the object point during the measurement, as well as multiple measurement parameters related to the position and posture of the camera 21 when each measurement image is shot.

[0086] Here, multiple measurement images each display multiple reference points. Each measurement image contains multiple reference blocks corresponding to the multiple reference points displayed in the measurement image. Furthermore, multiple measurement parameters contain the positions of the multiple reference blocks in the measurement image.

[0087] When the camera 21 captures measurement images, the tripod 23 is fixed in the same position as when capturing multiple learning images. Furthermore, the timing of the camera 21 capturing measurement images can be before, after, or between the capture of multiple learning images.

[0088] The generation unit 12 is a functional unit that generates the model 15. The model 15 takes multiple learning parameters as input and outputs the displacement (also called the first displacement) of the object block corresponding to the object point displayed in each of the multiple learning images relative to the reference position in the learning image.

[0089] Output unit 13 is a functional unit that corrects the displacement of the object block in the measurement image and outputs the corrected displacement. Specifically, output unit 13 obtains the displacement (also called the second displacement) of the object block corresponding to the object point displayed in the measurement image, caused by the vibration of the camera 21 in the measurement image, relative to a reference position, which is output by inputting multiple measurement parameters into the model 15. The second displacement is the displacement of the object point in the measurement image relative to the reference position, showing the displacement of the object block that is predicted to occur assuming that the camera 21 takes pictures based on the position and posture of the camera 21 when the measurement image is taken.

[0090] Furthermore, the output unit 13 outputs the displacement obtained by subtracting the second displacement from the displacement of the object block in the measurement image. Here, the operation of subtracting the second displacement from the displacement of the object block in the measurement image is equivalent to correction. By subtracting the second displacement from the displacement of the object block in the measurement image, the displacement in the measurement image corresponding to the actual displacement of the bridge 1 can be obtained. In addition, by performing known calculation processing (e.g., multiplying by a predetermined coefficient) on the displacement in the measurement image obtained as described above, the actual displacement of the bridge 1 can be calculated.

[0091] The following describes the detailed processing of each functional unit of the processing device 10.

[0092] Figure 3This is an explanatory diagram of image 40, which is the first example of an image captured in this embodiment. Image 40 is an example of an image generated by camera 21 during the learning period. When image 40 is captured, bridge 1 is considered to be stationary, and it is assumed that vibrations based on vibration devices 25 and 26 are applied to camera 21. The position of bridge 1 in this case is referred to as the "normal position of bridge 1". Similarly, the positions of the object point and the reference point of bridge 1 in this case are referred to as the normal position of the object point and the normal position of the reference point, respectively.

[0093] like Figure 3 As shown, Image 40 displays bridge 1, object point 31, and reference points 33-37. For example, let the right direction in Image 40 be the positive x-axis direction and the down direction be the positive y-axis direction, but it is not limited to these.

[0094] In image 40, object block 41 corresponds to object point 31. Furthermore, reference blocks 43–47 each correspond to reference points 33–37.

[0095] Furthermore, the location of object point 31 and reference points 33-37 is determined by an appropriate method. For example, object point 31 and reference points 33-37 can be set by prompting the user with image 40 and obtaining the user's acceptance instructions. Alternatively, object point 31 and reference points 33-37 can be set by inputting image 40 into a machine learning model that uses image as input to determine appropriate object points and reference points.

[0096] Figure 4 This is an explanatory diagram illustrating the method for generating model 15 in this embodiment.

[0097] Figure 4 This is a graph showing the time variation of the displacement of reference blocks 43-47 and object block 41 during the learning period. In each graph, the displacement is taken as the vertical axis and the time is taken as the horizontal axis. One scale unit on the vertical axis is, for example, about 1 mm (or pixel), and one scale unit on the horizontal axis is, for example, about 1 second, but it is not limited to these. Figure 4 The time variation of the displacement shown corresponds to the position of reference blocks 43-47 in all images generated by camera 21. Figure 4 The displacement vibrations of the reference blocks 43-47 and the object block 41 shown are due to the camera 21 vibrating during the shooting.

[0098] Figure 4 The displacement R1x indicates the displacement in the x direction of the reference block 43 in image 40 (in other words, the x component of the displacement). Figure 4 The displacement R1y indicates the displacement in the y direction of the reference block 43 in image 40 (in other words, the y component of the displacement).

[0099] same, Figure 4 The displacements R2x, R2y, R3x, R3y, R4x, R4y, R5x, and R5y respectively indicate the displacements of reference block 44 in the x and y directions, reference block 45 in the x and y directions, reference block 46 in the x and y directions, and reference block 47 in the x and y directions.

[0100] also, Figure 4 The displacements Tx and Ty represent the displacements of object block 41 in the x and y directions, respectively, in image 40.

[0101] First, the generation unit 12 generates a model 15 that takes displacements R1x to R5y as input and outputs displacement Tx. Model 15 is represented, for example, by a function fx with displacements R1x to R5y as independent variables and displacement Tx as dependent variable (see Equation 1 below). The function fx can also be expressed in any form of mathematical expression.

[0102] Tx=fx(R1x, R1y, R2x, R2y, R3x, R3y,

[0103] R4x, R4y, R5x, R5y)(Equation 1)

[0104] Generation unit 12 uses Figure 4 The displacements R1x to R5y shown are used to specifically determine the function fx, thereby generating model 15.

[0105] Taking the case where the function fx is represented by a linear combination of displacements R1x to R5y as independent variables as an example, the generation process of Model 15 will be explained.

[0106] In this case, the function fx behaves as follows (Equation 2).

[0107] Tx=p1×R1x+p2×R1y+p3×R2x+p4×R2y

[0108] +p5×R3x+p6×R3y+p7×R4x+p8×R4y

[0109] +p9×R5x+p10×R5y (Equation 2)

[0110] Here, p1, p2, ..., p10 are coefficients that do not depend on the displacements R1x to R5y, Tx, and Ty.

[0111] The acquisition unit 11 acquires images generated by the camera 21 with appropriate time intervals (e.g., about 0.01 seconds to 0.1 seconds) as multiple learning images. The generation unit 12 acquires the displacement amounts R1x to R5y of each of the multiple learning images acquired by the acquisition unit 11.

[0112] Next, the generation unit 12 uses the displacement values ​​R1x to R5y of each of the multiple learning images to determine the coefficients p1 to p10 using the least squares method. The more values ​​of the displacement values ​​R1x to R5y obtained, the more accurately the coefficients p1 to p10 can be determined.

[0113] In addition, as model 15, besides the function fx mentioned above, various information representing the relationship between displacements R1x to R5y and displacement Tx can also be used.

[0114] The generation unit 12 generates a model 15 that takes displacements R1x to R5y as input and outputs displacement Ty, similar to the generation process of model 15 described above. The model 15 that outputs displacement Ty is similar to the model 15 for displacement Tx; for example, it is represented by a function fy with displacements R1x to R5y as independent variables and displacement Ty as the dependent variable. The specific method for determining the function fy is the same as that for the function fx.

[0115] Thus, the generation unit 12 generates a model 15 that takes the displacements R1x to R5y of multiple reference blocks 43 to 47 in the image 40 as input and outputs the displacements Tx and Ty of the object block 41 in the image 40.

[0116] Figure 5 This is an explanatory diagram showing image 50, a second example of an image captured in this embodiment. Image 50 is an example of an image generated by camera 21 during measurement. Figure 5 When image 50 is captured, the object point 31 is displaced downward from its normal position due to the deflection of the bridge 1 caused by the external force pushing the bridge 1 downward due to the presence of a tram on the bridge 1. Therefore, in image 50, object block 41 is shown to have displaced in the positive y-axis direction. On the other hand, reference points 33-37 are shown to be at their normal positions (i.e., without displacement). Furthermore, the amount of downward displacement of the bridge 1 is, for example, about a few millimeters (or pixels), but is not limited to this.

[0117] like Figure 5 As shown, image 50 displays bridge 1, object point 31, and reference points 33-37. Furthermore, with... Figure 3 Similarly, object block 41 and reference blocks 43-47 are set.

[0118] The output unit 13 acquires the positions of the object block 41 and reference blocks 43-47 in the image 50 from the acquisition unit 11. Furthermore, the output unit 13 acquires the displacement amounts Tx and Ty of the object block 41, which are output by inputting the positions of the reference blocks 43-47 into the model 15 generated by the generation unit 12. At this time, the acquired displacement amounts Tx and Ty of the object block 41 indicate the displacement amounts of the object block 41 under the assumption that no external force is applied to the bridge 1.

[0119] In reality, as the tram travels on bridge 1, it exerts a force as an external force that pushes bridge 1 downward. Therefore, object block 41 is displaced in the positive y-axis direction compared to the position of object block 41 under the assumed case described above.

[0120] The output unit 13 outputs the position of the object block 41 obtained by subtracting the displacement of the object block 41 calculated using the model 15 from the position of the object block 41 obtained by the acquisition unit 11. The output value is the displacement in the image 50, which is equivalent to the downward displacement of the bridge 1 caused by the external force.

[0121] Figure 6 This is a flowchart illustrating the process of determining the position of the correction target block 41 in this embodiment.

[0122] In step S1, the acquisition unit 11 acquires multiple learning images and multiple learning parameters when each of the multiple learning images is captured during the learning period.

[0123] In step S2, the generation unit 12 uses the multiple learning images and multiple learning parameters obtained by the acquisition unit 11 in step S1 to generate a model 15 of the displacement amount (i.e., first displacement amount) of the object block 41 corresponding to the object point 31 displayed in each of the output learning images in the learning image.

[0124] In step S3, the acquisition unit 11 acquires multiple measurement images and multiple measurement parameters at the time each of the multiple measurement images is captured during the measurement.

[0125] In step S4, the output unit 13 uses the measurement image obtained in step S3 and multiple measurement parameters to obtain the displacement of the object block 41 caused by the vibration of the camera 21 (i.e., the second displacement).

[0126] In step S5, the output unit 13 uses the displacement of the object block 41 obtained in step S4 to correct the displacement of the object block 41 in the measurement image.

[0127] In step S6, the output unit 13 outputs the displacement of the object block 41 after correction in step S5.

[0128] Through the above series of processes, the processing device 10 is able to measure the displacement of the structure more appropriately.

[0129] Furthermore, the vibration imparted to the camera 21 by the vibration initiation devices 25 and 26 can also be a vibration with a defined amplitude or a defined frequency. In this case, when the acquisition unit 11 acquires multiple learning parameters, it can also extract and acquire the position vibration and posture vibration of the camera 21 with a defined amplitude or a defined frequency.

[0130] Thus, during the learning process, even if the bridge 1 or camera 21 generates vibrations with different amplitudes or frequencies than the specified values, the generation unit 12 can still generate the model 15 without being affected by the vibrations.

[0131] Here, any amplitude or frequency can be used to determine the amplitude or frequency; in particular, it can be a frequency different from the amplitude or frequency of the vibration that will occur on bridge 1. Thus, in the process of extracting vibrations with a defined amplitude or frequency, it has the advantage of being able to exclude vibrations naturally generated at camera 21 and to generate a more accurate model representing the relationship between the object block and the reference block. Furthermore, the amplitude or frequency of the vibration that will occur on bridge 1 can be calculated based on the size, weight, material, or strength of bridge 1, or the intensity of the wind blowing on bridge 1, etc.

[0132] Alternatively, the reference point can be determined by selecting an appropriate reference point candidate from a pre-determined list of reference point candidates. More specifically, multiple learning images may each display multiple reference point candidates, and each learning image may contain multiple reference block candidates corresponding to the multiple reference point candidates displayed in that learning image. In this case, when the generation unit 12 generates model 15, it uses multiple learning images containing multiple trial reference block groups selected from multiple reference block candidates based on multiple selection modes as multiple reference blocks to generate multiple trial models as model 15. Furthermore, the generation unit 12 generates model 15 from the multiple trial models whose output first displacement contains the smaller error.

[0133] (Modification 1 of the implementation method)

[0134] In this modified example, a configuration different from the above-described embodiment will be described for an information processing apparatus or the like that more appropriately measures the displacement of a structure. The information processing apparatus in this modified example uses images generated by multiple cameras to more appropriately measure the displacement of the structure.

[0135] Figure 7 This is a schematic diagram illustrating the measurement of displacement of the structure based on the processing device 10 in this modified example.

[0136] Figure 7 The image shows a bridge 1, cameras 21 and 22 that photograph the bridge 1 from a distance, and a processing device 10.

[0137] Bridge 1 is the same as in the above embodiment.

[0138] Cameras 21 and 22 are cameras equivalent to camera 21 in the above embodiment. Cameras 21 and 22 provide the generated images to the processing device 10.

[0139] Cameras 21 and 22 are supported together by tripod 23. Cameras 21 and 22 are fixed in a position that does not change relative to each other. Therefore, if an external force is applied to either camera 21 or 22 or to tripod 23, the position and orientation of cameras 21 and 22 will change as a single unit. Furthermore, cameras 21 and 22 take pictures simultaneously. The shooting parameters (lens, F-stop, focal length, and shutter speed, etc.) of cameras 21 and 22 can be different. Additionally, the pixel count of the images generated by cameras 21 and 22 can be different. Camera 21 is equivalent to a first camera, and camera 22 is equivalent to a second camera.

[0140] Vibration starters 25 and 26 are fixedly installed on cameras 21 and 22, respectively. Additionally, in... Figure 7 The diagram shows the case where vibration starters 25 and 26 are installed on camera 22, but vibration starters 25 and 26 can also be installed on camera 21, or vibration starter 25 can be installed on camera 21 and vibration starter 26 can be installed on camera 22. When there are three or more vibration starters, any number of vibration starters can be installed on camera 21 or 22.

[0141] In the images generated by cameras 21 and 22, an object point 31 and multiple reference points 33-37 are displayed. Here, the object point 31 and the multiple reference points 33-37 can be assigned to the images generated by camera 21 and camera 22 in any manner. Here, we will explain the case where the image generated by camera 21 displays the object point 31, and the image generated by camera 22 displays the multiple reference points 33-37. The image generated by camera 22 is equivalent to a third image. Multiple measurement parameters include the positions of multiple reference blocks in this third image.

[0142] However, more generally, one of cameras 21 and 22 can generate an image displaying the object point and a reference points, while the other camera 21 and 22 generates an image displaying (N-a) reference points. Here, N is the number of reference points, and a is an integer greater than 0 and less than N.

[0143] Furthermore, the example described is based on the scenario where cameras 21 and 22 are set to the same direction and photograph bridge 1 together. However, cameras 21 and 22 may not be set to the same direction. For example, camera 22 may be set to photograph in the opposite direction to that of camera 21.

[0144] Figure 8 These are schematic diagrams illustrating images 60A and 60B as examples of captured images in this modified example. Images 60A and 60B are examples of images generated by camera 21 during the learning process.

[0145] Figure 8 Image 60A shown in (a) and Figure 8 Image 60B shown in (b) is an example of images generated simultaneously by cameras 21 and 22. The scale of the subject displayed in images 60A and 60B differs due to differences in lenses or pixel counts, etc.

[0146] Image 60A shows object point 31 of bridge 1, and object block 41 corresponding to object point 31 is set.

[0147] Image 60B shows reference points 33A, 34A, 35A, 36A and 37A of bridge 1, and sets reference blocks 43A to 47A corresponding to each of reference points 33A to 37A.

[0148] The acquisition unit 11 acquires the positions of the target block 41 and the reference blocks 43A to 47A. The generation unit 12 uses the positions of the target block 41 and the reference blocks 43A to 47A acquired by the acquisition unit 11 to generate a model 15. Then, during the measurement, the acquisition unit 11 generates images by taking pictures using cameras 21 and 22. The output unit 13 uses the images generated by the acquisition unit 11 and, in the same manner as in the above embodiment, uses the model 15 to calculate the displacement of the target block 41, and outputs the value obtained by subtracting the displacement from the measured position of the target block 41.

[0149] in addition, Figure 8 The example given is a case where the magnification of image 60A (object point 31) is relatively small, while the magnification of images 60B (reference points 33A-37A) is relatively large. Setting the magnification to this level improves the accuracy of displacement measurement for object point 31. However, the magnification is not limited to a specific value. Figure 8 The example shown is an example of this. Specifically, it could be that the magnification of the image 60A capturing the object point 31 is relatively high, while the magnification of the images 60B capturing the reference points 33A to 37A is relatively low. By setting such a magnification, the accuracy of the displacement measurement of the reference points 33A to 37A is improved, and as a result, the measurement error of the object point 31 can be reduced.

[0150] Furthermore, the magnification of the image 60A capturing the object point 31 and the magnification of the images 60B capturing the reference points 33A to 37A can also be set together with the magnification of the object point 31. Figure 8 The image shown is identical to image 60B. Thus, the advantages of improved measurement accuracy and reduced measurement error, as described above, can be achieved.

[0151] (Modification 2 of the implementation method)

[0152] In this modified example, a configuration different from the above-described embodiment and the above-described modified example will be described for an information processing device or the like that which is more suitable for measuring the displacement of a structure.

[0153] The information processing device of this modified example uses the output value of a sensor fixed to the camera as multiple parameters to more appropriately measure the displacement of the structure.

[0154] Figure 9 This is a schematic diagram showing the camera 21 with sensors 71 and 72 fixed in this modified example.

[0155] like Figure 9 As shown, sensors 71 and 72 are fixedly mounted on camera 21.

[0156] Here, we will explain using the case where sensor 71 is a displacement sensor and sensor 72 is a gyroscope sensor as an example. That is, sensor 71, for example, senses the amount of displacement in three mutually orthogonal axes and outputs the sensed displacement. Sensor 72, for example, senses the angular velocity about three mutually orthogonal axes and outputs the sensed angular velocity. In addition, the type, number, or position of the sensors provided on camera 21 are not limited to the above.

[0157] When the position or orientation of the camera 21 changes, the displacement and angular velocity output by the sensors 71 and 72 change accordingly.

[0158] The acquisition unit 11 uses the output values ​​from sensors 71 and 72, namely, displacement and angular velocity, as parameters. That is, the acquisition unit 11 uses the displacement and angular velocity acquired during the learning period as learning parameters. Furthermore, the acquisition unit 11 uses the displacement and angular velocity acquired during the measurement period as measurement parameters.

[0159] That is, the multiple learning parameters include displacement data or angular velocity data of the camera 21 obtained from the output values ​​of the displacement sensor or gyroscope sensor fixed to the camera 21 when each of the multiple learning images is captured. In addition, the multiple measurement parameters include displacement data or angular velocity data of the camera 21 obtained from the output values ​​of the displacement sensor or gyroscope sensor when the measurement image is captured.

[0160] Figure 10 This is an explanatory diagram illustrating the method for generating the model in this variation.

[0161] Figure 10 The time-varying output values ​​of sensors 71 and 72 during the learning period are shown. Figure 10 In this diagram, the output value is taken as the vertical axis and time as the horizontal axis. One division on the vertical axis is approximately 1 mm for displacement sensor 71 and approximately 1 deg / s for gyroscope sensor 72. One division on the horizontal axis is approximately 1 second, but it is not limited to these values.

[0162] Figure 10 The displacements D1, D2, and D3 shown indicate the displacements in the directions of each of the three axes output by sensor 71. Figure 10 The angular velocities G1, G2, and G3 represent the angular velocities of each of the three axes output by sensor 72.

[0163] Similar to the embodiment described above, generation unit 12 generates a model that takes displacements D1, D2, and D3 and angular velocities G1, G2, and G3 as inputs and outputs the displacement Tx in the x-direction of the object block. The model is represented, for example, by the function gx (see Equation 3).

[0164] Tx=gx(D1, D2, D3, G1, G2, G3) (Formula 3)

[0165] Furthermore, when the function gx is represented, for example, by a linear combination of the displacements D1, D2, and D3 and the time integrals D4, D5, and D6 of the angular velocities G1, G2, and G3 (refer to Equation 4), the above model is generated by using the least squares method to determine the coefficients q1, etc. Tx=q1×D1+q2×D2+q3×D3+q4×D4+q5×D5+q6×D6 (Equation 4)

[0166] If the method shown in this variation is used, the position of the object block can be corrected based on the position and pose changes of the camera 21 without using a reference block.

[0167] (Modification 3 of the implementation method)

[0168] In this modified example, a processing device for more appropriately measuring the position of a structure will be described.

[0169] The processing apparatus of this modified example performs a process similar to that described in the above embodiment for measuring the displacement of the structure, thereby more appropriately measuring the position of the structure.

[0170] The processing apparatus in this modified example differs from the processing apparatus in the above-described embodiments in the processing of the output unit 13. The processing of the acquisition unit 11 and the generation unit 12 is the same as that of the processing apparatus 10 in the above-described embodiments, and therefore, the description is omitted.

[0171] In this modified example, the output unit 13 of the processing device obtains the second displacement of the object block corresponding to the object point displayed in the measurement image, which is output by inputting multiple measurement parameters to the model 15, and outputs the position obtained by subtracting the second displacement from the position of the object block in the measurement image.

[0172] In this way, the processing device in this modified example is able to measure the position of the structure more appropriately.

[0173] Furthermore, in the above-described embodiments and variations, each component can be implemented using dedicated hardware or by executing software programs suitable for each component. Alternatively, each structural component can be implemented by a program execution unit such as a CPU or processor reading and executing software programs recorded on a recording medium such as a hard disk or semiconductor memory. Here, the software for the information processing apparatus, etc., implementing the above-described embodiments and variations is the program described below.

[0174] That is, the program, computer, and camera generate multiple first images of an object point based on multiple positions and postures. The position and posture of the camera are obtained for each of the multiple first images at the time of capture. These multiple first parameters are used as inputs. A generation model is generated by outputting a first displacement of the reference position in the first image corresponding to the object point. The camera then generates multiple second images of the object point. The position and posture of the camera are obtained for each of the multiple second images at the time of capture. The model inputs and outputs the multiple second parameters for each of the multiple second images, and the second displacement of the reference position in the second image corresponding to the object point is obtained. The program executes a displacement output method based on the displacement of the object block in the multiple second images and the difference between the second displacement.

[0175] The above description describes one or more embodiments of an information processing device, but the present invention is not limited to these embodiments. Various modifications or combinations of elements from different embodiments that can be conceived by those skilled in the art without departing from the spirit of the present invention may also be included within the scope of one or more embodiments.

[0176] Industrial availability

[0177] This invention relates to a measuring device for measuring the displacement of structures.

[0178] Explanation of reference numerals in the attached figures

[0179] 1 Bridge

[0180] 10 processing devices

[0181] 11 Acquired Department

[0182] 12 Generation Department

[0183] 13 Output Section

[0184] 15 models

[0185] 21 and 22 cameras

[0186] 23 Tripod

[0187] 25, 26 Vibration starting devices

[0188] 31 object points

[0189] Reference points 33, 33A, 34, 34A, 35, 35A, 36, 36A, 37, 37A

[0190] Images 40, 50, 60A, and 60B

[0191] 41 object block

[0192] Reference blocks 43, 43A, 44, 44A, 45, 45A, 46, 46A, 47, 47A

[0193] Sensors 71 and 72

[0194] Displacement values ​​of D1, D2, D3, R1x, R1y, R2x, R2y, R3x, R3y, R4x, R4y, R5x, R5y, Tx, Ty

[0195] Angular velocities G1, G2, G3

Claims

1. An information processing method, wherein, Obtain multiple first images generated by the camera by capturing object points based on multiple positions and poses, and multiple first parameters related to the position and pose of the camera when each of the multiple first images was captured. A model is generated that takes the plurality of first parameters as input and outputs the first displacement of the object block corresponding to the object point displayed in each of the plurality of first images relative to the reference position in the first image. Obtain multiple second images generated by the camera by capturing the object point, and multiple second parameters related to the position and pose of the camera when each of the multiple second images was captured. Obtain a second displacement amount, which is the object block corresponding to the object point displayed in the second image and output relative to a reference position in the second image, by inputting the plurality of second parameters about each of the plurality of second images to the model. The output is the displacement obtained by subtracting the second displacement from the displacement of the object block in the plurality of second images.

2. The information processing method according to claim 1, wherein, Each of the multiple first images displays multiple reference points. Each of the plurality of first images contains a plurality of reference blocks that correspond one-to-one with the plurality of reference points displayed in the first image. The plurality of first parameters include the displacements of the plurality of reference blocks in the first image. The second image shows the plurality of reference points. The second image includes multiple reference blocks corresponding to the multiple reference points displayed in the second image. The plurality of second parameters include the displacement of the plurality of reference blocks in the second image.

3. The information processing method according to claim 1, wherein, The plurality of first parameters include displacement data or angular velocity data of the camera obtained from the output values ​​of a displacement sensor or gyroscope sensor fixed to the camera at the time when each of the plurality of first images was captured. The plurality of second parameters include displacement data or angular velocity data of the camera at the time the second image was captured, obtained from the output value of the displacement sensor or the gyroscope sensor.

4. The information processing method according to any one of claims 1 to 3, wherein, When acquiring the plurality of first images, the plurality of first images are generated by sequentially photographing the object points when the position changes within a specified distance relative to the reference position of the camera, or when the posture changes within a specified angle relative to the reference posture of the camera.

5. The information processing method according to claim 4, wherein, When generating the plurality of first images, the plurality of first images are generated by sequentially capturing the object points while the position and posture of the camera are vibrating.

6. The information processing method according to claim 5, wherein, The camera is vibrated by a vibration device fixed to the camera, and the object point is captured while the camera's position and orientation are vibrating based on the vibration applied to the camera, thereby generating the plurality of first images.

7. The information processing method according to claim 6, wherein, When the vibration device imparts vibration to the camera, it imparts a vibration that includes a component of movement in three mutually orthogonal directions and a component of rotation about three mutually orthogonal axes.

8. The information processing method according to claim 6, wherein, When obtaining the plurality of first parameters Instead of obtaining the plurality of first parameters when the camera is not subjected to vibration by the vibration initiation device, the plurality of first parameters are obtained when the camera is subjected to vibration by the vibration initiation device.

9. The information processing method according to claim 6, wherein, When the camera is vibrated by the vibration initiation device, a vibration with a defined amplitude or a defined frequency is imparted. When obtaining the plurality of first parameters Extract and obtain the position vibration and posture vibration of the camera with a determined amplitude or a determined frequency.

10. The information processing method according to claim 2, wherein, Each of the multiple first images displays multiple candidate reference points. Each of the plurality of first images contains a plurality of reference block candidates corresponding to the plurality of reference point candidates displayed in that first image. When generating the model Using the plurality of first images comprising a plurality of trial reference block groups selected from the plurality of reference block candidates based on a plurality of selection modes as the plurality of reference blocks, a plurality of trial models are generated as the model. The trial model with the smaller error in the first displacement of the output from the multiple generated trial models is generated as the model.

11. The information processing method according to claim 1, wherein, Multiple third images, each displaying multiple reference points, are captured simultaneously by a second camera whose position and posture change integrally with the first camera (which is also a first camera). Each of the plurality of third images contains a plurality of reference blocks that correspond one-to-one with the plurality of reference points displayed in the third image. The parameters include the positions of the reference blocks in the third image. The second image shows the plurality of reference points. The second image includes multiple reference blocks corresponding to the multiple reference points displayed in the second image. The plurality of second parameters include the positions of the plurality of reference blocks in the second image.

12. An information processing method, wherein, Obtain multiple first images generated by the camera by capturing object points based on multiple positions and poses, and multiple first parameters related to the position and pose of the camera when each of the multiple first images was captured. A model is generated that takes the plurality of first parameters as input and outputs the first displacement of the object block corresponding to the object point displayed in each of the plurality of first images relative to the reference position in the first image. Obtain multiple second images generated by the camera by capturing the object point, and multiple second parameters related to the position and pose of the camera when each of the multiple second images was captured. Obtain the second displacement of the object block corresponding to the object point displayed in the second image, relative to a reference position in the second image, which is output by inputting the plurality of second parameters into the model. The output is the position obtained by subtracting the second displacement from the position of the object block in the second image.

13. An information processing apparatus, wherein, have: Acquisition Department; Generation Department; as well as Output section The acquisition unit acquires multiple first images generated by the camera based on multiple positions and postures of the object point, and multiple first parameters related to the position and posture of the camera when each of the multiple first images was captured. The generation unit generates a model that takes the plurality of first parameters as input and outputs a model of the first displacement of the object block corresponding to the object point displayed in each of the plurality of first images in the first image. The acquisition unit further acquires a second image generated by the camera by capturing the object point, and multiple second parameters related to the position and pose of the camera when the second image was captured. The output unit obtains the second displacement of the object block corresponding to the object point displayed in the second image, which is output by inputting the plurality of second parameters to the model, and outputs the displacement obtained by subtracting the second displacement from the displacement of the object block in the second image.

14. An information processing system, wherein, Include: The information processing apparatus according to claim 13; The camera generates the plurality of first images and the plurality of second images; and A vibration-inducing device, fixed to the camera, imparts vibration to the camera.

Images