Information processing method, information processing device, and information processing system

Abstract

This method involves: (S1) obtaining learning images generated by imaging a target point using a camera, and a plurality of learning parameters related to the orientation and position of the camera when the learning images were captured; (S2) generating a model for outputting a first displacement amount, relative to a reference position in a learning image, of a target block which corresponds to the target point depicted in the learning image with the plurality of learning parameters as inputs; (S3) obtaining measurement images generated by imaging the target point using a camera, and a plurality of measurement parameters related to the orientation and position of the camera when the measurement images were captured; (S4) obtaining a second displacement amount, relative to the reference position in a measurement image, of the target block which corresponds to the target point depicted in the measurement image, said second displacement amount being outputted as a result of inputting the plurality of measurement parameters into the model; and (S5, S6) outputting the displacement obtained by subtracting the second displacement amount from the displacement of the target block in the measurement image.

Description

[Technical field]

[0001] The present invention relates to an information processing method, an information processing device, and an information processing system. [Background technology]

[0002] There is a technique for measuring the displacement of a structure such as a bridge by using an image of the structure (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 097576 Summary of the Invention [Problem to be solved by the invention]

[0004] However, if the camera used for capturing images vibrates, the captured image may be blurred or displaced, which may interfere with the measurement of the displacement of the structure.

[0005] Therefore, the present invention provides an information processing method and the like that allows for more appropriately measuring the displacement of a structure. [Means for solving the problem]

[0006] An information processing method according to one embodiment of the present invention is an information processing method which acquires a plurality of first images generated by a camera photographing a target point at a plurality of positions and orientations and a plurality of first parameters related to the position and orientation of the camera when each of the plurality of first images was photographed, generates a model using the plurality of first parameters as input and outputs a first displacement amount of a target block corresponding to the target point reflected in each of the plurality of first images from a reference position in the first image, acquires a plurality of second images generated by the camera photographing the target point and a plurality of second parameters related to the position and orientation of the camera when each of the plurality of second images was photographed, acquires a second displacement amount of a target block corresponding to the target point reflected in the second image from a reference position in the second image, which is output by inputting the plurality of second parameters for each of the plurality of second images into the model, and outputs a displacement obtained by subtracting the second displacement amount from the displacement of the target block in the plurality of second images.

[0007] These comprehensive or specific aspects may be realized as a system, an apparatus, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, an integrated circuit, a computer program, and a recording medium. Effect of the Invention

[0008] The information processing method of the present invention makes it possible to more appropriately measure the displacement of a structure. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram showing how a displacement of a structure is measured by a processing device according to an embodiment. [Diagram 2] FIG. 2 is a block diagram illustrating a functional configuration of the processing device according to the embodiment. [Diagram 3] FIG. 3 is an explanatory diagram showing a first example of a captured image in the embodiment. [Figure 4] FIG. 4 is an explanatory diagram showing a method of generating a model in the embodiment. [Diagram 5] FIG. 5 is an explanatory diagram showing a second example of a captured image in the embodiment. [Figure 6] FIG. 6 is a flow diagram showing a process for correcting the position of a target block in the embodiment. [Figure 7] FIG. 7 is a schematic diagram showing how the displacement of a structure is measured by the processing device in the first modification of the embodiment. [Figure 8] FIG. 8 is a schematic diagram showing an example of a captured image in the first modification of the embodiment. [Figure 9] FIG. 9 is a schematic diagram showing a camera to which a sensor is fixed in the second modification of the embodiment. [Figure 10] FIG. 10 is an explanatory diagram showing a method of generating a model in the second modification of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] (Findings on which the present invention is based) The present inventors have found that the following problems occur with the technology for measuring the displacement of a structure described in the "Background Art" section.

[0011] There is a technology for inspecting structures such as bridges that uses images of the structure taken from a distance with a high-precision camera. Displacement measurement using images has the advantage of being non-contact and capable of measuring multiple points simultaneously, and can contribute to reducing the labor required for measurement work in infrastructure maintenance and management, reducing the equipment and electricity required for measurement work, and shortening the time required.

[0012] When a camera or a support (e.g., a tripod) supporting the camera is hit or pushed by an object or fluid (liquid or gas), the camera may vibrate or shift in position (hereinafter, referred to as vibration, etc.). The position of a subject shown in an image taken by a camera when such vibration, etc. occurs is shifted from the position when the image is taken by the camera without vibration, etc. If such an image is used to measure the displacement of a structure, the measurement result will not be correct.

[0013] As described above, when the camera used for photographing vibrates, a position shift occurs in the photographed image, which may cause a problem in that it may be difficult to measure the displacement of a structure.

[0014] Therefore, the present invention provides an information processing method and the like that allows for more appropriately measuring the displacement of a structure.

[0015] An information processing method according to one embodiment of the present invention is an information processing method which acquires a plurality of first images generated by a camera photographing a target point at a plurality of positions and orientations and a plurality of first parameters related to the position and orientation of the camera when each of the plurality of first images was photographed, generates a model using the plurality of first parameters as input and outputs a first displacement amount of a target block corresponding to the target point reflected in each of the plurality of first images from a reference position in the first image, acquires a plurality of second images generated by the camera photographing the target point and a plurality of second parameters related to the position and orientation of the camera when each of the plurality of second images was photographed, acquires a second displacement amount of a target block corresponding to the target point reflected in the second image from a reference position in the second image, which is output by inputting the plurality of second parameters for each of the plurality of second images into the model, and outputs a displacement obtained by subtracting the second displacement amount from the displacement of the target block in the plurality of second images.

[0016] According to the above aspect, a model is generated using an image actually captured by a camera and parameters at the time of capturing the image, and the generated model is used to correct and output the position of a target block corresponding to a target point. The target point is a point that is the target of displacement measurement. The above model is a model that outputs the amount of displacement of a target block when the camera captures the image at the specified position and orientation when the position and orientation of the camera at the time of capturing the image are specified. The generation process of the model and the output process of the amount of displacement using the model do not require complex geometric calculations that take into account the position and orientation of the camera, and can be realized, for example, by relatively simple algebraic calculations. This can contribute to reducing computer resources and power consumption related to obtaining the amount of displacement of a target block. In this way, according to the above information processing method, the displacement of a structure can be measured more appropriately.

[0017] For example, each of the plurality of first images may include a plurality of reference points, each of the plurality of first images may include a plurality of reference blocks that correspond one-to-one to the plurality of reference points in the first image, the plurality of first parameters may include displacements of the plurality of reference blocks in the first image, the second image may include a plurality of reference points, each of the plurality of first images may include a plurality of reference blocks that correspond to the plurality of reference points in the second image, and the plurality of second parameters may include displacements of the plurality of reference blocks in the second image.

[0018] According to the above aspect, the displacement amount of the target block can be corrected by using as a parameter the position of the reference block corresponding to the reference point shown in the image together with the target point. The reference point is a point that serves as a reference when measuring the displacement. Therefore, there is no need to obtain information other than the image to correct the displacement amount of the target block, which means that the information required for correction can be reduced. Therefore, according to the above information processing method, the displacement of the structure can be measured more appropriately based on less information.

[0019] For example, the plurality of first parameters may include displacement data or angular velocity data of the camera at the time when each of the plurality of first images was captured, obtained from output values ​​of a displacement sensor or a gyro sensor fixed to the camera, and the plurality of second parameters may include displacement data or angular velocity data of the camera at the time when the second image was captured, obtained from output values ​​of the displacement sensor or the gyro sensor.

[0020] According to the above aspect, the displacement amount of the target block can be corrected by using the displacement amount or angular velocity of the camera measured by the sensor as a parameter. In this case, since no reference point is used for correcting the displacement amount of the target block, there is an advantage that it can be executed even when an appropriate reference point cannot be set on the image. Therefore, according to the above information processing method, even if an appropriate reference point cannot be set on the image, it is possible to more appropriately measure the displacement of the structure.

[0021] For example, when acquiring the multiple first images, the multiple first images may be acquired by sequentially photographing the target points when the position of the camera changes within a predetermined distance from a reference position, or when the attitude of the camera changes within a predetermined angle from a reference attitude of the camera, thereby generating the multiple first images.

[0022] According to the above aspect, a model can be generated using images captured sequentially by the camera while the position and orientation of the camera are changing, and the parameters at the time of capturing the images. At this time, since the position and orientation of the camera are within a predetermined range from the reference position and reference orientation, respectively, the deviation of the position of the target point in the image can be kept relatively small, and the accuracy of the displacement amount output by the model can be improved. Therefore, according to the above information processing method, the displacement of the structure can be measured more appropriately with higher accuracy.

[0023] For example, when generating the multiple first images, the multiple first images may be generated by sequentially photographing the target points while the position and attitude of the camera are vibrating.

[0024] According to the above aspect, a model can be generated using images sequentially captured by the camera while the position and orientation of the camera are vibrating, and the parameters at the time of capturing the images. At this time, since the position and orientation of the camera are vibrating, the positional deviation of the target point in the image can be kept relatively small, and reproducibility can be improved, thereby improving the accuracy of the displacement amount output by the model. Therefore, according to the above information processing method, the displacement of the structure can be measured more appropriately with higher accuracy.

[0025] For example, the multiple first images may be generated by photographing the target point while a vibration device fixed to the camera applies vibration to the camera and the position and attitude of the camera are vibrating based on the vibration applied to the camera.

[0026] According to the above aspect, the vibration of the position and the attitude of the camera can be imparted by the vibration device. Therefore, since there is no need to vibrate the camera by other means, the camera can be vibrated more easily and effectively, and the displacement amount of the target block can be corrected. Therefore, according to the above information processing method, the displacement of the structure can be measured more easily and appropriately.

[0027] For example, when the vibration device applies vibration to the camera, the vibration may include movement components in three mutually orthogonal directions and rotation components around three mutually orthogonal axes.

[0028] According to the above aspect, since the vibration including all the components of movement and rotation that can occur in three-dimensional space is applied to the camera by the vibration device, it is possible to generate a model that can output the displacement amount of the target block when various vibrations occur, and it is possible to correct the displacement amount of the target block with higher accuracy. Therefore, according to the above information processing method, it is possible to more appropriately measure the displacement of the structure with higher accuracy.

[0029] For example, when acquiring the multiple first parameters, it is possible to acquire the multiple first parameters when the camera is being vibrated by the vibration device, rather than acquiring the multiple first parameters when the camera is not being vibrated by the vibration device.

[0030] According to the above aspect, by not acquiring parameters when the camera is not vibrated by the vibration device, which parameters have a relatively small contribution to the generation of the model, the amount of information to be acquired can be reduced, and the required storage capacity of the storage device can be reduced. Therefore, according to the above information processing method, the measurement of the displacement of a structure can be realized using a smaller storage device.

[0031] For example, when the vibration device is used to apply vibration to the camera, vibrations having a specific amplitude or specific frequency may be applied, and when acquiring the multiple first parameters, the position vibrations and posture vibrations of the camera having the specific amplitude or specific frequency may be extracted and acquired from the position vibrations and posture vibrations of the camera.

[0032] According to the above aspect, a model capable of outputting the displacement amount of a target block is generated using vibrations of a specific amplitude or a specific frequency. Even if vibrations different from the specific amplitude or specific frequency occur in a structure or a camera, a model that corrects the displacement amount of a target block can be generated without being affected by the vibrations. Therefore, according to the above information processing method, even if the structure or the camera is vibrating, the displacement of the structure can be measured more appropriately.

[0033] For example, each of the plurality of first images may include a plurality of reference point candidates, and each of the plurality of first images may include a plurality of reference block candidates corresponding to the plurality of reference point candidates included in the first image. When generating the model, a plurality of trial models may be generated as the model using the plurality of first images including, as the plurality of reference blocks, a plurality of trial reference block groups selected from the plurality of reference block candidates using a plurality of selection patterns, and a trial model having a smaller error contained in the first displacement amount to be output may be generated as the model from among the plurality of generated trial models.

[0034] According to the above aspect, it is possible to set more reference point candidates than the reference points actually used, and to select and use a model with a smaller error when generating a model to correct the displacement amount of the target block. Therefore, according to the above information processing method, it is possible to more appropriately measure the displacement of a structure with higher accuracy.

[0035] For example, a second camera, which changes its position and attitude as a unit with the first camera, captures a plurality of third images simultaneously with the capture of a plurality of first images by the first camera, and each of the third images captures a plurality of reference points, and each of the third images includes a plurality of reference blocks that correspond one-to-one to the reference points captured in the third image, first The parameters may include positions of the plurality of reference blocks in the third image, the second image may include the plurality of reference points, the second image may include a plurality of reference blocks corresponding to the plurality of reference points included in the second image, and the plurality of second parameters may include positions of the plurality of reference blocks in the second image.

[0036] According to the above aspect, a model can be generated that can output the displacement amount of a target block by using a plurality of cameras whose positions and orientations change as a unit, and using reference blocks contained in images generated by the plurality of cameras. The shooting parameters of the plurality of cameras may be different. Therefore, by effectively using the plurality of cameras, reference points can be set at various positions, and accuracy can be improved. Therefore, according to the above information processing method, the displacement of a structure can be measured more appropriately with higher accuracy.

[0037] Moreover, an information processing method according to one embodiment of the present invention is an information processing method which acquires a plurality of first images generated by a camera photographing a target point at a plurality of positions and orientations and a plurality of first parameters related to the position and orientation of the camera when each of the plurality of first images was photographed, generates a model using the plurality of first parameters as input and outputs a first displacement amount of a target block corresponding to the target point reflected in each of the plurality of first images from a reference position in the first image, acquires a plurality of second images generated by the camera photographing the target point and a plurality of second parameters related to the position and orientation of the camera when each of the plurality of second images was photographed, acquires a second displacement amount of a target block corresponding to the target point reflected in the second images from a reference position in the second image, which is output by inputting the plurality of second parameters into the model, and outputs a position obtained by subtracting the second displacement amount from the position of the target block in the second image.

[0038] According to the above aspect, a model is generated using an image actually captured by a camera and parameters at the time of capturing the image, and the generated model can be used to correct and output the position of a target block corresponding to a target point. The target point is a point that is the subject of displacement measurement. The above model is a model that outputs the amount of displacement of a target block when the camera captures an image at the specified position and orientation when the position and orientation of the camera at the time of capturing the image are specified. The generation process of the model and the output process of the amount of displacement using the model do not require complex geometric calculations that take into account the position and orientation of the camera, and can be realized, for example, by relatively simple algebraic calculations. This can contribute to reducing computer resources and power consumption related to obtaining the amount of displacement of a target block. In this way, according to the above information processing method, the position of a structure can be measured more appropriately.

[0039] Moreover, an information processing device according to one aspect of the present invention includes an acquisition unit, a generation unit, and an output unit, wherein the acquisition unit acquires a plurality of first images generated by a camera photographing a target point at a plurality of positions and orientations, and a plurality of first parameters related to the positions and orientations of the camera when each of the plurality of first images was photographed, and the generation unit uses the plurality of first parameters as input to generate a target block corresponding to the target point captured in each of the plurality of first images, From the reference position A model that outputs a first displacement amount is generated, and the acquisition unit further generates a model that outputs a first displacement amount by using the camera to capture the target point. Multiple A second image; Multiple Second image each and a plurality of second parameters related to the position and orientation of the camera when the image was captured, Regarding each of the plurality of second images The plurality of second parameters are input into the model to be output. The The object block corresponding to the object point shown in the second image is From the reference position A second displacement amount is obtained. Multiple The information processing device outputs a displacement obtained by subtracting the second displacement amount from the displacement of the target block in the second image.

[0040] According to the above aspect, the same effects as those of the above information processing method are achieved.

[0041] Furthermore, an information processing system according to one aspect of the present invention is an information processing system including the above-mentioned information processing device, the camera that generates the multiple first images and the multiple second images, and a vibration device fixed to the camera that applies vibration to the camera.

[0042] According to the above aspect, by applying vibration to the camera using the vibration device, the same effect as the above information processing method can be achieved.

[0043] These comprehensive or specific aspects may be realized as a system, an apparatus, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, an integrated circuit, a computer program, or a recording medium.

[0044] Hereinafter, the embodiment will be specifically described with reference to the drawings.

[0045] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component arrangement and connection forms, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in an independent claim showing a top concept are described as optional components.

[0046] (Embodiment) In this embodiment, an information processing device and the like that can more appropriately measure the displacement of a structure will be described.

[0047] FIG. 1 is a schematic diagram showing how the displacement of a structure is measured in this embodiment.

[0048] 1 shows a bridge 1, a camera 21 that captures an image of the bridge 1 from a distance, and a processing device 10. Note that "distant" refers to a point that is, for example, several tens to several hundreds of meters away from the bridge 1, but is not limited to this.

[0049] The bridge 1 is a bridge fixed to the ground and is an example of a structure. The bridge 1 includes piers extending from underground to above ground, and bridge girders supported by the piers. The bridge 1 is an object whose displacement is measured by the processing device 10 in this embodiment. Note that the object whose displacement is measured by the processing device 10 in this embodiment is not limited to the bridge 1, and may be a building, a steel tower, a statue, a tower, or another such structure.

[0050] It is assumed that the parts of the bridge 1 that are fixed to the ground (for example, piers) do not displace relative to the ground, but the parts that are separated from the ground (for example, bridge girders) displace relative to the ground. The amount of displacement of the bridge girders relative to the ground occurs, for example, when an external force is applied to the bridge 1, causing deformation (specifically, bending or flexure) in the steel frame or concrete that constitutes the bridge 1. For example, when a car or train passes over the bridge girder, the weight of the car or train creates an external force that displaces the bridge girder downward. In addition, earthquakes or strong winds create forces that displace the bridge girder in various directions.

[0051] Here, a point on the bridge 1 that is the subject of displacement measurement is called a target point. Also, a point on the bridge 1 that is not expected to displace relative to the ground is called a reference point. In FIG. 1, one target point 31 and five reference points 33, 34, 35, 36, and 37 are set. Note that multiple reference points need to be set, and the greater the number of reference points, the higher the accuracy of measuring the displacement of the target point. Any number of target points greater than or equal to one may be set. The displacement of each of the one or more target points is measured using the multiple reference points.

[0052] The camera 21 is an imaging device that captures an image of the bridge 1, thereby generating an image showing the bridge 1. The camera 21 is supported by a tripod 23 that is fixedly placed on the ground. The material of the tripod 23 may be synthetic resin, metal, or the like. The time interval between images captured by the camera 21 is, for example, approximately 0.1 seconds or less. The camera 21 provides the generated image to the processing device 10.

[0053] If an external force is applied to the camera 21 or the tripod 23 when the camera 21 captures an image of the bridge 1, the camera 21 may vibrate due to bending and vibration of the tripod 23. If the camera 21 vibrates during shooting, a displacement occurs in the bridge 1 shown in the generated image, which prevents accurate measurement of the displacement of the bridge 1. Therefore, it is necessary to suppress the effect on the measurement result of the displacement of the bridge 1 in the image, which is due to the vibration generated in the camera 21 during shooting.

[0054] Vibration devices 25 and 26 are fixedly installed on the camera 21. The vibration devices 25 and 26 are devices that apply vibration to the camera 21. The vibration devices 25 and 26 are, for example, vibration devices that use solenoid coils.

[0055] The vibrations applied to the camera 21 by the vibration devices 25 and 26 may include components of movement in various directions or components of rotation around various axes. The vibrations applied to the camera 21 by the vibration devices 25 and 26 may include, for example, components of movement in three mutually orthogonal directions and components of rotation around three mutually orthogonal axes. The vibration devices 25 and 26 may be used in a process executed by the processing device 10 for suppressing the effects of vibrations and the like generated in the camera 21. Note that, although an example will be described in which two vibration devices are installed, the number of vibration devices installed may be one or three or more.

[0056] It should be noted that the vibration devices 25 and 26 are not necessarily required. Instead of the vibration devices 25 and 26, any device that can cause vibrations in the camera 21 may be used, and vibrations may be caused in the camera 21 by colliding an appropriate object or fluid against the camera 21. Vibrations may also be caused in the camera 21 by lightly tapping the camera 21 with a human finger.

[0057] The processing device 10 is an information processing device that measures the displacement of the bridge 1. The processing device 10 acquires an image generated by imaging with the camera 21, and measures the displacement of the bridge 1 based on the acquired image. At this time, the processing device 10 performs a correction process that suppresses the displacement of the target point in the image based on the vibration of the camera 21 at the time of shooting, etc., by using the displacement amount of the target point and the reference point shown in the image. In this way, the processing device 10 measures the target point with the displacement based on the vibration of the camera 21 at the time of shooting, etc., suppressed, that is, the displacement of the bridge 1.

[0058] The functionality of the processing device 10 will be explained in more detail hereinafter.

[0059] FIG. 2 is a block diagram showing a functional configuration of the processing device 10 in this embodiment.

[0060] 2, the processing device 10 includes an acquisition unit 11, a generation unit 12, and an output unit 13. Each of the above functional units can be realized by a processor (e.g., a CPU (Central Processing Unit), not shown) included in the processing device 10 executing a predetermined program using a memory (not shown).

[0061] The acquisition unit 11 is a functional unit that acquires an image generated by photographing the bridge 1 and a plurality of parameters related to the position and attitude of the camera 21 when the image was captured. The acquisition unit 11 acquires the image and the plurality of parameters during a learning period and a measurement period. Here, the learning period is a period for calculating the amount of displacement of the target point used in the correction process performed during the measurement period. Moreover, the measurement period is a period for measuring the displacement of the target point and correcting the measured displacement using the amount of displacement of the target point calculated during the learning period.

[0062] The acquisition unit 11 also identifies a portion of a predetermined size in which the target point is captured in the acquired image as a target block, and identifies a portion of a predetermined size in which the reference point is captured as a reference block. The process of identifying the target block in the image is performed by identifying a point in which the target point is captured based on the characteristics of the surroundings of the target point captured in the captured image, and identifying a portion of a predetermined size that includes the point, and can be realized by well-known techniques. The same applies to the reference point and the reference block.

[0063] Specifically, during the learning period, acquisition unit 11 acquires a plurality of images (also referred to as learning images or first images) generated by camera 21 photographing a target point at a plurality of positions and attitudes, and a plurality of parameters (also referred to as learning parameters or first parameters) related to the position and attitude of camera 21 when each of the plurality of learning images was photographed. Although a case will be described in which the plurality of learning images are a portion of the plurality of images photographed by camera 21 as an example, the plurality of learning images may be all of the plurality of images photographed by camera 21. The time interval at which the plurality of learning images are photographed is, for example, about 0.1 to 0.01 seconds.

[0064] Here, each of the multiple learning images has multiple reference points. Each of the multiple learning images includes multiple reference blocks that correspond one-to-one to the multiple reference points in the learning image. The multiple learning parameters include positions of the multiple reference blocks in the learning image. The positions of the reference blocks in the learning image are, for example, coordinates in a two-dimensional coordinate system of x and y. The same applies hereinafter.

[0065] When the camera 21 acquires a plurality of learning images, the tripod 23 is fixed at the same position on the ground. The position and attitude of the camera 21 in a state where no external force is applied to the camera 21 and the tripod 23 are also referred to as the reference position and the reference attitude of the camera 21, respectively. The camera 21 acquires a plurality of learning images by sequentially capturing target points when, for example, the position of the camera 21 changes within a predetermined distance from the reference position, or when the attitude of the camera 21 changes within a predetermined angle from the reference attitude, thereby generating a plurality of learning images. The above expression also includes capturing images when the position of the camera 21 changes within a predetermined distance from the reference position and the attitude of the camera 21 changes within a predetermined angle from the reference attitude.

[0066] More specifically, when generating a plurality of learning images, camera 21 generates a plurality of learning images by sequentially capturing images of target points while the position and attitude of camera 21 are vibrating.

[0067] The vibration of the position and attitude of the camera 21 is, for example, vibration applied by the vibration applying devices 25 and 26. That is, when the camera 21 generates a plurality of learning images, the vibration applying devices 25 and 26 fixed to the camera 21 apply vibration to the camera 21, and the target point is photographed while the position and attitude of the camera 21 are vibrating based on the vibration applied to the camera 21. Here, a case where the vibration applying devices that apply vibration to the camera 21 are the vibration applying devices 25 and 26 will be described as an example, but the present invention is not limited to this, and only one of the vibration applying devices 25 and 26 may be used, or when three or more vibration applying devices are provided, three or more vibration applying devices may apply vibration to the camera 21. Vibration may be caused by a device other than the vibration applying device.

[0068] The acquisition unit 11 may acquire a plurality of learning parameters when the camera 21 is being vibrated by the vibration devices 25 and 26, rather than acquiring a plurality of learning parameters when the camera 21 is not being vibrated by the vibration devices 25 and 26. In other words, the acquisition unit 11 may acquire a plurality of learning parameters only when the camera 21 is being vibrated by the vibration devices 25 and 26.

[0069] In addition, during the measurement period, the acquisition unit 11 acquires multiple images (also referred to as measurement images or second images) generated by the camera 21 photographing the target point, and multiple measurement parameters related to the position and orientation of the camera 21 when each measurement image was photographed.

[0070] Here, a plurality of reference points are captured in each of the plurality of measurement images, the measurement images include a plurality of reference blocks corresponding to the plurality of reference points captured in the measurement images, and the plurality of measurement parameters include positions of the plurality of reference blocks in the measurement images.

[0071] When camera 21 captures the measurement image, tripod 23 is fixed in the same position as when the multiple learning images were captured. Note that the timing when camera 21 captures the measurement image may be before capturing the multiple learning images, after capturing the multiple learning images, or between capturing the multiple learning images.

[0072] The generation unit 12 is a functional unit that generates a model 15. The model 15 receives a plurality of learning parameters as input, and outputs a displacement amount (also referred to as a first displacement amount) of a target block corresponding to a target point shown in each of a plurality of learning images from a reference position in the learning image.

[0073] The output unit 13 is a functional unit that corrects the displacement of the target block in the measurement image and outputs the corrected displacement. Specifically, the output unit 13 acquires the displacement amount (also called the second displacement amount) of the target block corresponding to the target point shown in the measurement image, which is output by inputting a plurality of measurement parameters to the model 15, from the reference position caused by the vibration of the camera 21 in the measurement image. The second displacement amount is the displacement amount of the target point in the measurement image from the reference position, and indicates the displacement amount that is estimated to be the displacement of the target block if the camera 21 were to capture the measurement image with the position and orientation of the camera 21 at the time the measurement image was captured.

[0074] Moreover, the output unit 13 outputs a displacement obtained by subtracting the second displacement amount from the displacement of the target block in the measurement image. Here, the operation of subtracting the second displacement amount from the displacement of the target block in the measurement image corresponds to correction. By subtracting the second displacement amount from the displacement of the target block in the measurement image, a displacement amount in the measurement image that corresponds to the actual displacement amount of the bridge 1 is obtained. Note that the actual displacement amount of the bridge 1 can be calculated by performing a well-known arithmetic process (e.g., multiplying by a predetermined coefficient) on the displacement amount in the measurement image obtained as described above.

[0075] The processing of each functional unit included in the processing device 10 will be described in detail below.

[0076] 3 is an explanatory diagram showing image 40, which is a first example of a captured image in this embodiment. Image 40 is an example of an image generated by camera 21 during a learning period. When image 40 is captured, bridge 1 is considered to be stationary, and vibration is applied to camera 21 by vibration devices 25 and 26, etc. The position of bridge 1 in this case is also referred to as the "normal position of bridge 1." Similarly, the positions of the target point and reference point of bridge 1 in this case are also referred to as the normal position of the target point and the normal position of the reference point, respectively.

[0077] 3, image 40 shows bridge 1, target point 31, and reference points 33 to 37. For example, the right direction in image 40 is the positive x-axis direction, and the downward direction is the positive y-axis direction, but is not limited to this.

[0078] In an image 40, a target block 41 corresponds to the target point 31. Furthermore, reference blocks 43 to 47 correspond to the reference points 33 to 37, respectively.

[0079] Where the target point 31 and the reference points 33 to 37 are set is determined by an appropriate method. For example, the target point 31 and the reference points 33 to 37 may be set by presenting the image 40 to a user and acquiring an instruction accepted from the user. Alternatively, the target point 31 and the reference points 33 to 37 may be set using the target point 31 and the reference points 33 to 37 obtained by inputting the image 40 into a machine learning model that determines appropriate target points and reference points using the image as input.

[0080] FIG. 4 is an explanatory diagram showing a method for generating the model 15 in this embodiment.

[0081] FIG. 4 shows a graph illustrating the change over time in the amount of displacement of each of the reference blocks 43-47 and the target block 41 during the learning period. In each graph, the vertical axis represents the amount of displacement, and the horizontal axis represents time. One division on the vertical axis is, for example, about 1 mm (or pixel), and one division on the horizontal axis is, for example, about 1 second, but is not limited thereto. The change over time in the amount of displacement shown in FIG. 4 corresponds to the positions of the reference blocks 43-47 in all the images generated by the camera 21. The reason why the amount of displacement of the reference blocks 43-47 and the target block 41 shown in FIG. 4 fluctuates is because the camera 21 vibrates when the images are taken.

[0082] 4 indicates the amount of displacement in the x direction (in other words, the x component of the displacement) of the reference block 43 in the image 40. The amount of displacement R1y in Fig. 4 indicates the amount of displacement in the y direction (in other words, the y component of the displacement) of the reference block 43 in the image 40.

[0083] Similarly, the displacement amounts R2x, R2y, R3x, R3y, R4x, R4y, R5x and R5y in Figure 4 respectively indicate the displacement amounts in the x and y directions of reference block 44, the displacement amounts in the x and y directions of reference block 45, the displacement amounts in the x and y directions of reference block 46, and the displacement amounts in the x and y directions of reference block 47.

[0084] Moreover, displacement amounts Tx and Ty in FIG. 4 indicate the displacement amounts of a target block 41 in an image 40 in the x direction and y direction, respectively.

[0085] First, the generation unit 12 generates a model 15 that receives the displacement amounts R1x to R5y as input and outputs a displacement amount Tx. The model 15 is expressed, for example, by a function fx that has the displacement amounts R1x to R5y as independent variables and the displacement amount Tx as a dependent variable (see Equation 1 below). The function fx may be expressed by any form of mathematical expression.

[0086] Tx=fx(R1x,R1y,R2x,R2y,R3x,R3y, R4x,R4y,R5x,R5y) (Formula 1)

[0087] The generating unit 12 generates the model 15 by specifically determining the function fx using the displacement amounts R1x to R5y shown in FIG.

[0088] The process of generating the model 15 will be described taking as an example a case where the function fx is expressed as a linear combination of the displacement amounts R1x to R5y, which are the independent variables.

[0089] In this case, the function fx is expressed as shown below (Equation 2).

[0090] Tx= p1×R1x+p2×R1y+p3×R2x+p4×R2y +p5×R3x+p6×R3y+p7×R4x+p8×R4y +p9×R5x+p10×R5y (Formula 2)

[0091] Here, p1, p2, . . . , p10 are coefficients that are independent of the displacements R1x to R5y, Tx, and Ty.

[0092] The acquisition unit 11 acquires, as a plurality of learning images, images having an appropriate time interval (for example, about 0.01 to 0.1 seconds) from among the images generated by the camera 21. The generation unit 12 acquires the amounts of displacement R1x to R5y in each of the plurality of learning images acquired by the acquisition unit 11.

[0093] Next, the generating unit 12 determines the coefficients p1 to p10 by the least squares method using the amounts of displacement R1x to R5y in each of the multiple learning images. The greater the number of values ​​of the amounts of displacement R1x to R5y that are obtained, the more accurately the coefficients p1 to p10 can be determined.

[0094] As the model 15, in addition to the above function fx, various information indicating the relationship between the displacement amounts R1x to R5y and the displacement amount Tx can be used.

[0095] The generating unit 12 generates the model 15 which receives the displacements R1x to R5y as input and outputs the displacement Ty in the same manner as the generating process of the model 15 described above. The model 15 which outputs the displacement Ty is expressed by, for example, a function fy which has the displacements R1x to R5y as independent variables and the displacement Ty as a dependent variable, in the same manner as the case of the displacement Tx. A specific method of determining the function fy is the same as the case of the function fx.

[0096] In this manner, the generating unit 12 generates the model 15 that receives the displacement amounts R1x to R5y of the multiple reference blocks 43 to 47 in the image 40 as input and outputs the displacement amounts Tx and Ty of the target block 41 in the image 40.

[0097] FIG. 5 is an explanatory diagram showing an image 50 which is a second example of a captured image in this embodiment. The image 50 is an example of an image generated by the camera 21 during a measurement period. When the image 50 in FIG. 5 is captured, the target point 31 is displaced downward from its normal position due to the bending of the bridge 1 caused by an external force that pushes the bridge 1 downward due to the presence of a train on the bridge 1. As a result, the target block 41 is displaced in the positive direction of the y axis in the image 50. Meanwhile, the reference points 33 to 37 are each located at their normal positions (i.e., not displaced). The amount of downward displacement of the bridge 1 is, for example, about several mm (or pixels), but is not limited thereto.

[0098] 5, an image 50 shows a bridge 1, a target point 31, and reference points 33 to 37. Also, similar to the case of FIG. 3, a target block 41 and reference blocks 43 to 47 are set.

[0099] The output unit 13 acquires the positions of the target block 41 and the reference blocks 43 to 47 in the image 50 from the acquisition unit 11. The output unit 13 also acquires the displacement amounts Tx and Ty of the target block 41, which are output by inputting the positions of the reference blocks 43 to 47 into the model 15 generated by the generation unit 12. The acquired displacement amounts Tx and Ty of the target block 41 at this time indicate the displacement amounts of the target block 41 when it is assumed that no external force is applied to the bridge 1.

[0100] In reality, a train is running across bridge 1, which creates an external force pushing bridge 1 downward, so that target block 41 is displaced in the positive y-axis direction from the position of target block 41 in the above hypothetical case.

[0101] The output unit 13 outputs a value obtained by subtracting the amount of displacement of the target block 41 calculated using the model 15 from the position of the target block 41 acquired from the acquisition unit 11 as the position of the target block 41. The output value is the amount of displacement in the image 50, which corresponds to the amount of downward displacement of the bridge 1 due to an external force.

[0102] FIG. 6 is a flow diagram showing a process for correcting the position of the target block 41 in this embodiment.

[0103] In step S1, the acquisition unit 11 acquires, during a learning period, a plurality of learning images and a plurality of learning parameters at the time when each of the plurality of learning images was captured.

[0104] In step S2, the generation unit 12 uses the multiple learning images and multiple learning parameters acquired by the acquisition unit 11 in step S1 to generate a model 15 that outputs the displacement amount (i.e., the first displacement amount) in the learning image of the target block 41 corresponding to the target point 31 shown in each of the learning images.

[0105] In step S3, the acquisition unit 11 acquires a plurality of measurement images and a plurality of measurement parameters when each of the plurality of measurement images is captured during the measurement period.

[0106] In step S4, the output unit 13 acquires the amount of displacement of the target block 41 caused by the vibration of the camera 21 (that is, the second amount of displacement) using the measurement image acquired in step S3 and the plurality of measurement parameters.

[0107] In step S5, the output unit 13 corrects the displacement of the target block 41 in the measurement image using the amount of displacement of the target block 41 acquired in step S4.

[0108] In step S6, the output unit 13 outputs the displacement of the target block 41 after the correction in step S5.

[0109] Through the above series of processes, the processing device 10 can more appropriately measure the displacement of the structure.

[0110] The vibrations applied to the camera 21 by the vibration devices 25 and 26 may be vibrations having a specific amplitude or a specific frequency. In this case, when the acquisition unit 11 acquires a plurality of learning parameters, the acquisition unit 11 may extract and acquire the position vibrations and posture vibrations of a specific amplitude or a specific frequency from the position vibrations and posture vibrations of the camera 21.

[0111] By doing this, even if vibrations of a specific amplitude or frequency other than the specific amplitude or frequency occur in the bridge 1 or the camera 21 during the learning period, the generation unit 12 can generate the model 15 without being affected by the vibrations.

[0112] Here, the specific amplitude or specific frequency can be any amplitude or frequency, and may in particular be a frequency different from the amplitude or frequency of vibration that may occur in the bridge 1. In this way, in the process of extracting vibrations of a specific amplitude or frequency, vibrations that naturally occur in the camera 21 can be excluded, which has the advantage of enabling a more accurate model showing the relationship between the target block and the reference block to be generated. The amplitude or frequency of vibration that may occur in the bridge 1 can be calculated from the dimensions, weight, material, or strength of the bridge 1, or the strength of the wind blowing on the bridge 1, etc.

[0113] The reference point may be determined by selecting an appropriate reference point candidate from among predefined reference point candidates. More specifically, each of the multiple learning images may include multiple reference point candidates, and each of the multiple learning images may include multiple reference block candidates corresponding to the multiple reference point candidates included in the learning image. In this case, when the generating unit 12 generates the model 15, multiple trial models as the model 15 are generated using multiple learning images including multiple trial reference block groups selected from the multiple reference block candidates in multiple selection patterns as multiple reference blocks. Then, the generating unit 12 generates, as the model 15, a trial model having a smaller error included in the first displacement amount to be output from among the multiple generated trial models.

[0114] (First Modification of the Embodiment) In this modification, an information processing device that more appropriately measures the displacement of a structure will be described, which has a different configuration from the above embodiment. The information processing device of this modification more appropriately measures the displacement of a structure by using images generated by multiple cameras.

[0115] FIG. 7 is a schematic diagram showing how the displacement of a structure is measured by the processing device 10 in this modified example.

[0116] FIG. 7 shows a bridge 1, cameras 21 and 22 for photographing the bridge 1 from a distance, and a processing device 10.

[0117] The bridge 1 is the same as that in the above embodiment.

[0118] The cameras 21 and 22 correspond to the camera 21 in the above embodiment. The cameras 21 and 22 provide the generated images to the processing device 10.

[0119] Both cameras 21 and 22 are supported by tripod 23. Cameras 21 and 22 are fixed so that their relative positions do not change. Therefore, when an external force is applied to camera 21 or 22 or tripod 23, cameras 21 and 22 change their position and attitude as a whole. Furthermore, cameras 21 and 22 capture images simultaneously. The imaging parameters (lens, F-number, focal length, shutter speed, etc.) of cameras 21 and 22 may be different. Furthermore, the number of pixels of images generated by cameras 21 and 22 may be different. Camera 21 corresponds to a first camera, and camera 22 corresponds to a second camera.

[0120] Vibration devices 25 and 26 are fixedly installed on cameras 21 and 22. Although Fig. 7 shows a state in which vibration devices 25 and 26 are installed on camera 22, vibration devices 25 and 26 may be installed on camera 21, or vibration device 25 may be installed on camera 21 and vibration device 26 may be installed on camera 22. When there are three or more vibration devices, any number of vibration devices may be installed on camera 21 or 22.

[0121] The images captured by cameras 21 and 22 capture target point 31 and multiple reference points 33-37. Here, target point 31 and multiple reference points 33-37 may be allocated in any manner to the images captured by cameras 21 and 22. Here, an example will be described in which target point 31 is captured in the image captured by camera 21, and multiple reference points 33-37 are captured in the image captured by camera 22. The image captured by camera 22 corresponds to a third image. The multiple measurement parameters include positions of multiple reference blocks in the third image.

[0122] However, more generally, an image generated by one of the cameras 21 and 22 may include a target point and a reference points, and an image generated by the other of the cameras 21 and 22 may include (Na) reference points, where N is the number of reference points and a is an integer between 0 and N.

[0123] Although the case where cameras 21 and 22 are installed facing the same direction and both photograph bridge 1 will be described as an example, the orientations of cameras 21 and 22 do not have to be the same. For example, camera 22 may be installed facing in a direction opposite to the direction in which camera 21 photographs.

[0124] 8 is a schematic diagram showing images 60A and 60B which are examples of captured images in this modified example. Images 60A and 60B are examples of images generated by camera 21 during the learning period.

[0125] An image 60A shown in Fig. 8(a) and an image 60B shown in Fig. 8(b) are examples of images simultaneously generated by the cameras 21 and 22. The scale of the subject shown in the images 60A and 60B is different due to differences in the lenses or the number of pixels, etc.

[0126] Image 60A shows target point 31 on bridge 1, and target block 41 corresponding to target point 31 is set.

[0127] The image 60B shows reference points 33A, 34A, 35A, 36A, and 37A of the bridge 1, and reference blocks 43A to 47A corresponding to the reference points 33A to 37A, respectively, are set.

[0128] The acquisition unit 11 acquires the positions of the target block 41 and the reference blocks 43A to 47A. The generation unit 12 generates a model 15 using the positions of the target block 41 and the reference blocks 43A to 47A acquired by the acquisition unit 11. Thereafter, the acquisition unit 11 generates images by photographing with the cameras 21 and 22 during the measurement period. The output unit 13 uses the images generated by the acquisition unit 11 to calculate the amount of displacement of the target block 41 using the model 15 in the same manner as in the above embodiment, and outputs a value obtained by subtracting the amount of displacement from the measured position of the target block 41.

[0129] In FIG. 8, a case has been described as an example in which the magnification ratio of the image 60A in which the target point 31 is captured is relatively small, and the magnification ratio of the image 60B in which the reference points 33A to 37A are captured is relatively large. By using such a magnification ratio, the measurement accuracy of the displacement of the target point 31 can be improved. However, the magnitude of the magnification ratio is not limited to the example shown in FIG. 8. Specifically, there may be a case in which the magnification ratio of the image 60A in which the target point 31 is captured is relatively large, and the magnification ratio of the image 60B in which the reference points 33A to 37A are captured is relatively small. By using such a magnification ratio, the measurement accuracy of the displacement of the reference points 33A to 37A can be improved, and as a result, the measurement error of the target point 31 can be reduced.

[0130] Furthermore, the magnification ratio of image 60A in which target point 31 is captured and the magnification ratio of image 60B in which reference points 33A to 37A are captured may both be set to approximately the same as that of image 60B shown in Fig. 8. In this way, the advantages of improved measurement accuracy and reduced measurement error as described above can be obtained.

[0131] (Modification 2 of the embodiment) In this modification, an information processing device that more appropriately measures the displacement of a structure will be described with a different configuration from the above embodiment and the modification.

[0132] The information processing device of this modified example more appropriately measures the displacement of a structure by using output values ​​of sensors fixed to a camera as a plurality of parameters.

[0133] FIG. 9 is a schematic diagram showing camera 21 to which sensors 71 and 72 are fixed in this modified example.

[0134] As shown in FIG. 9, the camera 21 has sensors 71 and 72 fixedly installed thereon.

[0135] Here, an example will be described in which sensor 71 is a displacement sensor and sensor 72 is a gyro sensor. That is, sensor 71 senses, for example, the amount of displacement in three mutually orthogonal axial directions and outputs the sensed amount of displacement. Sensor 72 senses, for example, angular velocity around three mutually orthogonal axes and outputs the sensed angular velocity. Note that the type, number, or position of sensors installed in camera 21 is not limited to the above.

[0136] When the position or attitude of the camera 21 changes, the displacement amount and angular velocity output by the sensors 71 and 72 change accordingly.

[0137] The acquisition unit 11 uses the output values ​​output by the sensors 71 and 72, that is, the displacement amount and the angular velocity, as parameters. That is, the acquisition unit 11 uses the displacement amount and the angular velocity acquired during the learning period as learning parameters. Also, the acquisition unit 11 uses the displacement amount and the angular velocity acquired during the measurement period as measurement parameters.

[0138] That is, the multiple learning parameters include displacement data or angular velocity data of the camera 21 when each of the multiple learning images was captured, which is obtained from the output value of a displacement sensor or gyro sensor fixed to the camera 21. Also, the multiple measurement parameters include displacement data or angular velocity data of the camera 21 when the measurement image was captured, which is obtained from the output value of a displacement sensor or gyro sensor.

[0139] FIG. 10 is an explanatory diagram showing a method of generating a model in this modified example.

[0140] Fig. 10 shows the time change in the output values ​​of sensors 71 and 72 during the learning period. In Fig. 10, the output value is plotted on the vertical axis, and time is plotted on the horizontal axis. One division on the vertical axis is, for example, about 1 mm for sensor 71, which is a displacement sensor, and about 1 deg / s for sensor 72, which is a gyro sensor, and one division on the horizontal axis is, for example, about 1 second, but is not limited to this.

[0141] Displacement amounts D1, D2, and D3 shown in Fig. 10 indicate the displacement amounts in the directions of the three axes output by sensor 71. Angular velocities G1, G2, and G3 in Fig. 10 indicate the angular velocities about the three axes output by sensor 72.

[0142] As in the above embodiment, the generator 12 receives the displacements D1, D2, and D3 and the angular velocities G1, G2, and G3, and generates a model that outputs the displacement Tx of the target block in the x direction. The model is expressed by, for example, a function gx (see Equation 3).

[0143] Tx=gx(D1,D2,D3,G1,G2,G3) (Formula 3)

[0144] When the function gx is expressed, for example, as a linear combination of the displacements D1, D2, and D3, and the time integral values ​​D4, D5, and D6 of the angular velocities G1, G2, and G3, respectively (see Equation 4), the above model is generated by determining each coefficient q1, etc. by the least squares method.

[0145] Tx=q1×D1+q2×D2+q3×D3+q4×D4+q5×D5+q6×D6 (Formula 4)

[0146] By using the method shown in this modification, it is possible to correct the position of the target block based on changes in the position and orientation of the camera 21 without using a reference block.

[0147] (Third Modification of the Embodiment) In this modification, a processing device that more appropriately measures the position of a structure will be described.

[0148] The processing device of this modification performs processing similar to the processing for measuring the displacement of a structure in the above embodiment, thereby more appropriately measuring the position of the structure.

[0149] The processing device of this modification differs from the processing device in the above embodiment in the processing of the output unit 13. The processing of the acquisition unit 11 and the generation unit 12 is similar to that of the processing device 10 in the above embodiment, and therefore description thereof will be omitted.

[0150] The output section 13 of the processing device in this modified example acquires a second displacement amount in the measurement image of a target block corresponding to a target point shown in the measurement image, which is output by inputting multiple measurement parameters into the model 15, and outputs the position of the target block in the measurement image minus the second displacement amount.

[0151] In this way, the processing device in this modified example can more appropriately measure the position of the structure.

[0152] In the above embodiment and the above modification, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software for realizing the information processing device of the above embodiment and the above modification is a program such as the following.

[0153] In other words, this program causes a computer to execute an information processing method of acquiring a plurality of first images generated by a camera photographing a target point at a plurality of positions and attitudes and a plurality of first parameters related to the position and attitude of the camera when each of the plurality of first images was photographed, generating a model using the plurality of first parameters as input to output a first displacement amount of a target block corresponding to the target point reflected in each of the plurality of first images from a reference position in the first image, acquiring a plurality of second images generated by the camera photographing the target point and a plurality of second parameters related to the position and attitude of the camera when each of the plurality of second images was photographed, acquiring a second displacement amount of a target block corresponding to the target point reflected in the second image from a reference position in the second image, which is output by inputting the plurality of second parameters for each of the plurality of second images into the model, and outputting a displacement obtained by subtracting the second displacement amount from the displacement of the target block in the plurality of second images.

[0154] Although the information processing device according to one or more aspects has been described based on the embodiment, the present invention is not limited to this embodiment. As long as it does not deviate from the spirit of the present invention, various modifications conceived by a person skilled in the art to this embodiment and forms constructed by combining components in different embodiments may also be included within the scope of one or more aspects. [Industrial Applicability]

[0155] The present invention can be used in a measurement device that measures the displacement of a structure. [Explanation of symbols]

[0156] 1 Bridges 10 Processing equipment 11 Acquisition Department 12 Generation part 13 Output section 15 Models 21, 22 Camera 23 Tripod 25, 26 Vibration device 31 Target points 33, 33A, 34, 34A, 35, 35A, 36, 36A, 37, 37A reference points 40, 50, 60A, 60B Images 41 Target Block 43, 43A, 44, 44A, 45, 45A, 46, 46A, 47, 47A Reference Blocks 71, 72 Sensors D1, D2, D3, R1x, R1y, R2x, R2y, R3x, R3y, R4x, R4y, R5x, R5y, Tx, Ty displacement amount G1, G2, G3 Angular velocity

Claims

1. Acquiring a plurality of first images generated by a camera photographing a target point at a plurality of positions and orientations, and a plurality of first parameters related to the positions and orientations of the camera when each of the plurality of first images was photographed; generating a model that uses the plurality of first parameters as input and outputs a first displacement amount from a reference position in each of the plurality of first images of a target block corresponding to the target point shown in each of the plurality of first images; Acquire a plurality of second images generated by the camera photographing the target point and a plurality of second parameters related to the position and orientation of the camera when each of the plurality of second images was photographed; acquiring a second displacement amount from a reference position in the second image of a target block corresponding to the target point shown in the second image, the second displacement amount being output by inputting the second parameters for each of the second images into the model; and outputting a displacement obtained by subtracting the second displacement amount from the displacement of the target block in the plurality of second images. Information processing methods.

2. A plurality of reference points are captured in each of the plurality of first images, Each of the plurality of first images includes a plurality of reference blocks that correspond one-to-one to the plurality of reference points depicted in the first image, the first parameters include displacements of the reference blocks in the first image; The second image includes the plurality of reference points, The second image includes a plurality of reference blocks corresponding to the plurality of reference points captured in the second image, The second parameters include displacements of the reference blocks in the second image. The information processing method according to claim 1 .

3. the plurality of first parameters include displacement data or angular velocity data of the camera when each of the plurality of first images is captured, the displacement data or angular velocity data being obtained from an output value of a displacement sensor or a gyro sensor fixed to the camera, The second parameters include displacement data or angular velocity data of the camera when the second image was captured, the displacement data or angular velocity data being obtained from an output value of the displacement sensor or the gyro sensor.

3. The information processing method according to claim 1.

4. When acquiring the plurality of first images, when the position of the camera changes within a predetermined distance from a reference position, or when the attitude of the camera changes within a predetermined angle from a reference attitude, the plurality of first images are generated by sequentially photographing the target points, thereby acquiring the plurality of first images. The information processing method according to any one of claims 1 to 3.

5. When generating the plurality of first images, the plurality of first images are generated by sequentially photographing the target points while the position and orientation of the camera are vibrating. The information processing method according to claim 4.

6. The plurality of first images are a vibration device fixed to the camera applies vibration to the camera; The camera is generated by photographing the target point while the position and orientation of the camera are vibrating based on the vibration being applied to the camera. The information processing method according to claim 5.

7. When the vibration device applies vibration to the camera, the vibration includes components of movement in three mutually orthogonal directions and components of rotation about three mutually orthogonal axes. The information processing method according to claim 6.

8. When obtaining the plurality of first parameters, The plurality of first parameters are not acquired when the vibration is not applied to the camera by the vibration applying device, The plurality of first parameters are obtained when the camera is being vibrated by the vibration device.

8. The information processing method according to claim 6 or 7.

9. When the vibration is applied to the camera by the vibration applying device, vibration having a specific amplitude or a specific frequency is applied, When obtaining the plurality of first parameters, Among the vibrations of the position and the posture of the camera, the vibrations of the position and the posture of the camera having the specific amplitude or the specific frequency are extracted and acquired. The information processing method according to any one of claims 6 to 8.

10. A plurality of reference point candidates are captured in each of the plurality of first images, Each of the plurality of first images includes a plurality of reference block candidates corresponding to the plurality of reference point candidates appearing in the first image, When generating the model, generating a plurality of trial models as the model using the plurality of first images including, as the plurality of reference blocks, a plurality of trial reference block groups selected from the plurality of reference block candidates in a plurality of selection patterns; Among the plurality of trial models thus generated, a trial model having a smaller error included in the first displacement amount to be output is generated as the model. The information processing method according to claim 2 .

11. a second camera, which changes position and attitude integrally with the first camera, captures a plurality of third images simultaneously with the capture of a plurality of first images by the first camera, each of which includes a plurality of reference points; Each of the plurality of third images includes a plurality of reference blocks that correspond one-to-one to the plurality of reference points shown in the third image, the first parameters include positions of the reference blocks in the third image; The second image includes the plurality of reference points, The second image includes a plurality of reference blocks corresponding to the plurality of reference points captured in the second image, The second parameters include positions of the reference blocks in the second image. The information processing method according to claim 1 .

12. Acquiring a plurality of first images generated by a camera photographing a target point at a plurality of positions and orientations, and a plurality of first parameters related to the positions and orientations of the camera when each of the plurality of first images was photographed; generating a model that uses the plurality of first parameters as input and outputs a first displacement amount from a reference position in each of the plurality of first images of a target block corresponding to the target point shown in each of the plurality of first images; Acquire a plurality of second images generated by the camera photographing the target point and a plurality of second parameters related to the position and orientation of the camera when each of the plurality of second images was photographed; acquiring a second displacement amount from a reference position in the second image of a target block corresponding to the target point captured in the second image, the second displacement amount being output by inputting the plurality of second parameters into the model; A position obtained by subtracting the second displacement amount from the position of the target block in the second image is output. Information processing methods.

13. An acquisition unit; A generation unit; An output unit, The acquisition unit acquires a plurality of first images generated by a camera photographing a target point at a plurality of positions and orientations, and a plurality of first parameters related to the positions and orientations of the camera when each of the plurality of first images was photographed; the generating unit generates a model that uses the plurality of first parameters as input and outputs a first displacement amount from a reference position in each of the plurality of first images of a target block corresponding to the target point captured in each of the plurality of first images; The acquisition unit further acquires a plurality of second images generated by the camera by photographing the target point, and a plurality of second parameters related to a position and an attitude of the camera when each of the plurality of second images was photographed; The output unit acquires a second displacement amount from a reference position in the second image of a target block corresponding to the target point shown in the second image, the second displacement amount being output by inputting the second parameters for each of the second images into the model, and outputs a displacement obtained by subtracting the second displacement amount from the displacement of the target block in the second images. Information processing device.

14. An information processing device according to claim 13; the camera generating the first images and the second images; a vibration device fixed to the camera and applying vibration to the camera; Information processing system.

Images