Apparatus, method and program for processing imaging data

Abstract

To allow a user to easily identify a range of imaging data for which accuracy of measurement result is likely to be insufficient, out of accumulated pieces of imaging data, and efficiently delete imaging data in a range of deletion.SOLUTION: A processing apparatus is configured to display an imaging data adjuster including an image obtained by visualizing accuracy of measurement result based on imaging data on each imaging point, and delete, in accordance with user operation to designate a range on the imaging data adjuster, imaging data included in the designated range. Especially, the apparatus is configured to display, on the imaging data adjuster 202, a map image 211 which visualizes the accuracy of measurement result along a trajectory, and delete, in accordance with the user operation to designate the range on the map image, imaging data included in the designated range. The apparatus is configured to display, on the imaging data adjuster 203, a timeline image 221 obtained by visualizing the accuracy of measurement result along a time axis, and delete, in accordance with the user operation to designate a range on the timeline image, imaging data included in the designated range.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] The present invention relates to a photography data processing device, photography data processing method, and photography data processing program that processes photography data from each point obtained by sequentially photographing a measurement target location with a camera by moving the device body in order to perform 3D measurement processing to generate 3D spatial information of the measurement target location. [Background technology]

[0002] There is known a 3D measurement technique that generates 3D spatial information (map data) about a measurement target location based on images of the measurement target location. In recent years, the SLAM (Simultaneous Localization And Mapping) method has been attracting attention for this 3D measurement. In the SLAM method, a photographing device is held by a moving object, and based on images of each point in the measurement target location, 3D spatial information and the position information of the object can be generated as a measurement result. In particular, when a user performs a photographing task of photographing each point in the measurement target location while moving within the measurement target location using a photographing device that can be carried by hand, the user can easily perform 3D measurement by acquiring photographed images of each point in the measurement target location.

[0003] On the other hand, in 3D measurement using SLAM, the accumulation of errors is unavoidable because relative self-localization is repeated. Such accumulation of errors in self-localization causes a significant decrease in the accuracy of the 3D measurement results.

[0004] As a method for improving the accuracy degradation caused by such accumulated errors, a technology has been known that, when an error occurs during the creation of three-dimensional spatial information (map data), allows the creation of the three-dimensional spatial information to be restarted from the middle without restarting from the beginning (see Patent Document 1). Also, this technology detects an event (lost) that causes an error and presents the user with the period during which the event occurred, allowing the user to grasp the position where the creation of the three-dimensional spatial information should be restarted.

[0005] Furthermore, a loop closing technique is known as a method for improving accuracy degradation due to accumulated errors (see Non-Patent Document 1). With this technique, when it is detected that the vehicle's trajectory forms a loop, i.e., the vehicle has returned to a location where it was previously photographed, the self-location estimation result (past location) acquired during previous photography at that location is regarded as the correct location, and the location on the trajectory from the current location to the past location is corrected, thereby eliminating the accumulated errors. With this technique, it is possible to eliminate the trouble of restarting photography halfway through, as with the technique disclosed in Patent Document 1. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 6639734 [Non-patent literature]

[0007] [Non-Patent Document 1] SSII2015 Tutorial: Incremental 3D reconstruction from video images using feature point tracking and its applications (lecture given at the Image Sensing Symposium on June 10, 2015) Summary of the Invention [Problem to be solved by the invention]

[0008] In three-dimensional measurement using SLAM, the loop-closing technique disclosed in Patent Document 2 is effective in improving accuracy degradation caused by accumulated errors.

[0009] On the other hand, in the loop closing process, the process of correcting the position of each shooting point is executed in sequence, starting from the shooting point where the loop is closed, going back in time from the starting point, i.e., in the opposite direction to the moving direction from the starting point to the shooting start point. At this time, the corrected position contains an error, and this position correction error accumulates as the shooting points are traced back. Therefore, the accuracy of the position correction gradually decreases. In other words, the accuracy of the point cloud data generated from the shooting data at each shooting point gradually decreases as the distance from the starting point of the loop closing process increases.

[0010] That is, in the section before the start point of the loop closing process, i.e., the section where the loop closing process has been performed, the accuracy of the point cloud data generated from the photographing data gradually decreases as the distance from the start point of the loop closing process increases due to the accumulation of position correction errors in the loop closing process. On the other hand, in the section after the start point of the loop closing process, i.e., the section where the loop closing process has not been performed, the accuracy of the point cloud data generated from the photographing data gradually decreases as the distance from the start point of the loop closing process increases due to the accumulation of self-position estimation errors in the 3D measurement.

[0011] Even if the loop closing process is performed in this way, there is a high possibility that high-precision point cloud data cannot be generated from the photographing data at a point far from the starting point of the loop closing process. On the other hand, if low-precision point cloud data is included in the point cloud data generated from the photographing data at each photographing point, the accuracy of the point cloud data as a whole will be low, and problems such as double images will occur.

[0012] For this reason, in order to generate highly accurate point cloud data as a measurement result, it is desirable to delete in advance the photographic data that causes the accuracy of the point cloud data to decrease. However, the conventional technology does not disclose any mechanism for efficiently deleting the range of photographic data that is likely to cause the measurement result to be insufficient in accuracy from among the accumulated photographic data.

[0013] Therefore, the main object of the present invention is to provide a photography data processing device, a photography data processing method, and a photography data processing program that enable a user to easily identify, from among the accumulated photography data, a range of photography data in which the accuracy of the measurement results is likely to be insufficient, and that can efficiently delete the photography data in the range that should be deleted. [Means for solving the problem]

[0014] The photographing data processing device of the present invention is a photographing data processing device in which a processor processes photographing data of each point obtained by sequentially photographing the measurement target location with a camera by moving the device body in order to perform 3D measurement processing that generates 3D spatial information of the measurement target location, and further comprises a memory unit that accumulates processing information regarding the photographing data and the implementation status of loop closing processing in the 3D measurement processing, and a display that displays a screen for checking the photographing data and the implementation status of the loop closing processing, and in order to adjust the photographing data accumulated in the memory unit, the processor displays on the display screen a photographing data adjustment unit including an image that visualizes the accuracy of the measurement results based on the photographing data for each photographing location, and deletes the photographing data included in a specified range in response to a user's operation to specify a range in the photographing data adjustment unit.

[0015] In addition, the shooting data processing method of the present invention is a method for processing shooting data in which a processor processes shooting data obtained by sequentially photographing the measurement target location with a camera by moving the device body in order to perform 3D measurement processing that generates 3D spatial information of the measurement target location, and in order to adjust the shooting data stored in the memory unit, a shooting data adjustment unit including an image visualizing the accuracy of the measurement results based on the shooting data for each shooting point is displayed on the display screen, and the shooting data included in a specified range is deleted in response to a user's operation to specify a range in the shooting data adjustment unit.

[0016] In addition, the shooting data processing program of the present invention is a shooting data processing program that causes a processor to process shooting data from each point acquired by sequentially photographing the measurement target location with a camera by moving the device body in order to perform 3D measurement processing that generates 3D spatial information of the measurement target location, and is configured to adjust the shooting data stored in the memory unit by displaying on the display screen a shooting data adjustment unit including an image that visualizes the accuracy of the measurement results based on the shooting data for each shooting point, and to delete the shooting data included in a specified range in response to a user's operation to specify a range in the shooting data adjustment unit. Effect of the Invention

[0017] According to the present invention, a user can check an image in which the accuracy of the measurement result is visualized and specify the range of the photographed data to be deleted. This allows the user to easily identify the range of the photographed data in which the accuracy of the measurement result is likely to be insufficient among the accumulated photographed data, and efficiently delete the photographed data in the range to be deleted. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is an explanatory diagram showing a situation of a photographing operation using the photographing device according to the present embodiment; [Diagram 2] Plan view showing the measurement location [Diagram 3] FIG. 1 is an explanatory diagram showing an overhead point cloud trajectory image according to a first example of an overhead map image; [Figure 4] FIG. 1 is an explanatory diagram showing an overhead trajectory image according to a first example of an overhead map image; [Diagram 5] FIG. 11 is an explanatory diagram showing an overhead trajectory image according to a second example of an overhead map image; [Figure 6] FIG. 11 is an explanatory diagram showing an overhead trajectory image according to a third example of an overhead map image; [Figure 7] FIG. 11 is an explanatory diagram showing an overhead trajectory image according to a fourth example of an overhead map image; [Figure 8] FIG. 13 is an explanatory diagram showing an overhead trajectory image according to a fifth example of an overhead map image; [Figure 9]FIG. 13 is an explanatory diagram showing an example of a timeline image. [Figure 10] FIG. 13 is an explanatory diagram showing an example of a timeline image. [Figure 11] FIG. 1 is a block diagram showing a schematic configuration of an imaging device. [Figure 12] FIG. 1 is a flow diagram showing a procedure of a section determination process performed by a processor. [Figure 13] FIG. 13 is an explanatory diagram showing the shooting screen displayed on the display. [Figure 14] FIG. 13 is an explanatory diagram showing a shooting data adjustment screen displayed on a display. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] The first invention made to solve the above problems is a photographing data processing device in which a processor processes photographing data of each point obtained by sequentially photographing the measurement target location with a camera by moving the device body in order to perform 3D measurement processing to generate 3D spatial information of the measurement target location, and further comprises a memory unit that accumulates the photographing data and processing information related to the implementation status of a loop closing process in the 3D measurement processing, and a display that displays a screen for checking the photographing data and the implementation status of the loop closing process, and in order to adjust the photographing data accumulated in the memory unit, the processor displays on the display screen a photographing data adjustment unit including an image that visualizes the accuracy of the measurement results based on the photographing data for each photographing location, and deletes the photographing data included in a specified range in response to a user's operation to specify a range in the photographing data adjustment unit.

[0020] This allows the user to check an image in which the accuracy of the measurement results is visualized and specify the range of photographed data to be deleted. This allows the user to easily identify the range of photographed data in which the accuracy of the measurement results is likely to be insufficient among the accumulated photographed data, and efficiently delete the photographed data in the range to be deleted.

[0021] In addition, in a second invention, the processor is configured to display the shooting data adjustment unit including a map image in which the accuracy of measurement results based on shooting data for each shooting location is visualized along a trajectory representing the transition of shooting locations, and to delete the shooting data included in a specified range in response to a user's operation to specify a range on the map image.

[0022] According to this, the user can appropriately specify the range of the photographic data to be deleted. In this case, the map image may be set to sections divided according to the accuracy of the measurement results based on the photographic data for each photographing point.

[0023] In addition, in a third invention, the processor is configured to display on a screen the shooting data adjustment unit including a timeline image in which the accuracy of measurement results based on shooting data for each shooting location is visualized along a time axis, and to delete the shooting data included in a specified range in response to a user's operation to specify a range on the timeline image.

[0024] According to this, the user can appropriately specify the range of the photographic data to be deleted. In this case, in the timeline image, sections may be set according to the accuracy of the measurement results based on the photographic data for each photographing point.

[0025] In addition, a fourth invention is configured such that the processor displays, in response to a user's operation to specify a shooting timing, an image captured by the camera at a specified shooting timing on a screen of the display.

[0026] This allows the user to visually check the captured image and thereby confirm the shooting conditions at the shooting timing (shooting time, shooting position) specified by the user.

[0027] In addition, in a fifth invention, the processor is configured to determine, based on the processing information, sections recommended for deletion where the accuracy of the measurement results based on the photographing data for each photographing location is expected to be insufficient, and to visualize the sections recommended for deletion and display them on the display screen.

[0028] This allows the user to easily identify the range of shooting data where the measurement results are likely to be insufficient in accuracy.

[0029] In addition, in a sixth invention, the processor is configured to display an operation unit on the display screen through which a user can instruct bulk deletion, and to delete the photographic data included in the recommended deletion section in accordance with user operation of the operation unit.

[0030] According to this, the imaging data included in the recommended deletion section can be easily deleted. In particular, when there are multiple recommended deletion sections, the imaging data included in the multiple recommended deletion sections can be deleted collectively.

[0031] In addition, in a seventh invention, the processor is configured to display an input unit on the display screen into which a user can input a threshold value that serves as a criterion for determining the recommended deletion section, and to set the input threshold value in accordance with the user's input into the input unit.

[0032] According to this, the user can arbitrarily specify the threshold value that is the criterion for determining whether a section is recommended for deletion. Note that two input units may be provided so that the first threshold value that is the criterion for determining a section for which the loop closing process has not been performed and the second threshold value that is the criterion for determining a section for which the loop closing process has been performed can be set to different values.

[0033] In addition, the eighth invention is a method for processing image data in which a processor processes image data obtained by sequentially photographing the measurement target location with a camera by moving the device body in order to perform 3D measurement processing to generate 3D spatial information of the measurement target location, and in order to adjust the image data stored in the memory unit, a image data adjustment unit including an image visualizing the accuracy of the measurement results based on the image data for each shooting point is displayed on the display screen, and the image data included in a specified range is deleted in response to a user's operation to specify a range in the image data adjustment unit.

[0034] According to this, as in the first invention, the user can easily identify the range of accumulated shooting data in which the measurement results are likely to have insufficient accuracy, and can efficiently delete the range of shooting data that should be deleted.

[0035] In addition, a ninth invention is a photography data processing program that causes a processor to process photography data obtained by sequentially photographing the measurement target location with a camera by moving the device body in order to perform 3D measurement processing to generate 3D spatial information of the measurement target location, and in order to adjust the photography data stored in the memory unit, a photography data adjustment unit including an image visualizing the accuracy of the measurement results based on the photography data for each photography point is displayed on the display screen, and the photography data included in a specified range is deleted in response to a user's operation to specify a range in the photography data adjustment unit.

[0036] According to this, as in the first invention, the user can easily identify the range of accumulated shooting data in which the measurement results are likely to have insufficient accuracy, and can efficiently delete the range of shooting data that should be deleted.

[0037] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0038] Fig. 1 is an explanatory diagram showing a situation in which a user performs a photographing operation using an image capturing device 1 according to this embodiment. Fig. 2 is a plan view showing a measurement target location.

[0039] The photographing device 1 (photographed data processing device) includes a device body 11 and a sensor unit 12. The sensor unit 12 includes a visible camera 21 (photographing unit). The visible camera 21 is a monocular camera that detects visible light to photograph a subject, and outputs a photographed image, for example, a color image in the RGB format. The photographing device 1 can be configured as a tablet terminal or a notebook PC.

[0040] A user (worker) walks through a location to be measured while holding the device body 11 of the photographing device 1. At this time, the photographing device 1 photographs the location to be measured with the visible camera 21, and photographic data is sequentially acquired for each point in the location to be measured. The photographic data includes data such as the photographed image, the sensor detection result, the photographing time, and features extracted from the photographed image. Note that, although an example has been shown here in which the photographic data is acquired by the user holding the device body 11 walking through the location to be measured, photographic data of the location to be measured may also be acquired using a self-propelled robot or the like equipped with the photographing device 1.

[0041] In this embodiment, the photographing device 1 is a three-dimensional measuring device. That is, in the photographing device 1, three-dimensional measurement processing is performed based on photographing data of each point sequentially acquired by the visible camera 21, and three-dimensional spatial information on the measurement target location is generated. In the three-dimensional measurement processing, point cloud data (environment map) is generated as three-dimensional spatial information on the measurement target location using the SLAM method. At this time, self-position estimation processing is performed in conjunction with the generation of the point cloud data, and the self-position for each time, i.e., the position of the photographing point, is estimated.

[0042] Furthermore, when the image capturing device 1 detects that its own trajectory forms a loop, i.e., that its own device has returned to a location where it previously captured an image, it performs a loop closing process. In the loop closing process, for a previously captured location, the self-location estimation result (past location) acquired during the previous image capturing is regarded as the correct location, and the location of each image capturing location on the trajectory from the current location to the past location is corrected.

[0043] In the example shown in Fig. 2, shooting starts from the shooting start point, goes around a part of the area of ​​the measurement target location, and returns to the shooting start point. Furthermore, shooting continues and another area of ​​the measurement target location is shot. In this case, a loop closing process is performed when returning to the shooting start point. This improves the accuracy of the point cloud data and the self-location estimation result for each shooting point included in the loop.

[0044] Next, the map image will be described. First, a first example of the map image will be described. Fig. 3 is an explanatory diagram showing an overhead point cloud trajectory image according to the first example of the map image. Fig. 4 is an explanatory diagram showing an overhead trajectory image according to the first example of the map image.

[0045] In the loop closing process, the shooting point where the loop is closed is set as the starting point, and the process of correcting the position of each shooting point is executed sequentially from the starting point back in time, that is, in the opposite direction to the moving direction from the starting point to the shooting start point. At this time, the corrected position contains an error, and this position correction error accumulates as the shooting points are traced back. Therefore, the accuracy of the position correction gradually decreases. In other words, the accuracy of the point cloud data generated from the shooting data at each shooting point gradually decreases as the distance from the starting point of the loop closing process increases.

[0046] In this way, in the section prior to the start point of the loop closing process, i.e., the section where the loop closing process has already been performed, errors in the position correction in the loop closing process accumulate, and the accuracy of the point cloud data generated from the shooting data gradually decreases as you move away from the start point of the loop closing process.

[0047] On the other hand, in the section after the start point of the loop closing process, i.e., the section where the loop closing process has not been performed, errors in self-position estimation in 3D measurement accumulate, and the accuracy of the point cloud data generated from the shooting data gradually decreases as you move away from the start point of the loop closing process.

[0048] Even if the loop closing process is performed in this way, there is a high possibility that high-precision point cloud data cannot be generated from the photographing data at a point far from the starting point of the loop closing process. On the other hand, if low-precision point cloud data is included in the point cloud data generated from the photographing data at each photographing point, the accuracy of the point cloud data as a whole will be low, and problems such as double images will occur.

[0049] Therefore, in this embodiment, in order to generate highly accurate point cloud data as a measurement result, the photographing data that causes a decrease in the accuracy of the point cloud data is deleted in advance. In particular, in this embodiment, the accuracy of the point cloud data generated from the photographing data at each photographing point is visualized and presented to the user so that the user can visually confirm it.

[0050] In this embodiment, the distance between the starting point of the loop closing process and the target shooting point is calculated as an index for evaluating the accuracy of point cloud data generated from the shooting data of each shooting point.

[0051] The amount of separation may be the time between the shooting time at the start point of the loop closing process and the shooting time at the target shooting location. The amount of separation may also be the distance (distance on the trajectory) between the start point of the loop closing process and the target shooting location. The amount of separation may also be the number of times (number of frames) that images are shot between the start point of the loop closing process and the target shooting location. The amount of separation may also be the number of key frames of the SLAM method shot between the start point of the loop closing process and the target shooting location.

[0052] In this embodiment, based on the amount of deviation for each shooting location, it is determined whether each shooting location (shooting time) falls into a high-precision section, an acceptable accuracy section, or a section recommended for deletion (section determination process). Specifically, the amount of deviation for each shooting location is compared with a predetermined determination criterion to determine whether each shooting location falls into a high-precision section, an acceptable accuracy section, or a section recommended for deletion.

[0053] At this time, it is determined whether the loop closing process has been performed at each shooting location, that is, whether each shooting location is included in a section where the loop closing process has not been performed or a section where the loop closing process has been performed. If the target shooting location is included in a section where the loop closing process has not been performed, a determination criterion (first threshold value) for the section where the loop closing process has not been performed is used. If the target shooting location is included in a section where the loop closing process has been performed, a determination criterion (second threshold value) for the section where the loop closing process has been performed is used.

[0054] Here, the high-precision section is a section where the loop closing process has already been performed and is not far from the starting point of the loop closing process, so it is assumed that high accuracy can be ensured for the point cloud data generated from the shooting data.The acceptable accuracy section is a section where the loop closing process has not been performed, but is not far from the starting point of the loop closing process, so it is assumed that accuracy within an acceptable range can be ensured for the point cloud data generated from the shooting data at the shooting points included in the section.The recommended deletion section is a section where the accuracy of the point cloud data generated from the shooting data at the shooting points included in the section is assumed to be insufficient, because it is far from the starting point of the loop closing process.

[0055] In this manner, in this embodiment, the amount of deviation that is an index of the accuracy of the point cloud data (measurement results) generated from the photographing data at each photographing point is calculated. Then, based on the amount of deviation for each photographing point, the accuracy of the point cloud data generated from the photographing data at each photographing point is visualized and presented to the user along a trajectory that represents the transition of the photographing points from the photographing start point to the photographing end point.

[0056] In this embodiment, the section determination process determines whether each shooting point corresponds to a high-precision section, an acceptable accuracy section, or a section recommended for deletion, and based on this determination result, the high-precision section, the acceptable accuracy section, and the section recommended for deletion are set on the trajectory. That is, the trajectory is divided into the high-precision section, the acceptable accuracy section, and the section recommended for deletion. Then, the high-precision section, the acceptable accuracy section, and the section recommended for deletion set on the trajectory are visualized by using different colors.

[0057] In the example shown in FIG. 3, the high-precision section, the allowable accuracy section, and the section recommended for deletion are visualized by color coding on the bird's-eye view point cloud trajectory image. In the example shown in FIG. 4, the high-precision section, the allowable accuracy section, and the section recommended for deletion are visualized by color coding on the bird's-eye view trajectory image. Here, the bird's-eye view point cloud trajectory image is a combination of the bird's-eye view point cloud image and the bird's-eye view trajectory image. The bird's-eye view point cloud image is an image (rendering) of each point of the point cloud data as viewed from a viewpoint in the sky. The bird's-eye view trajectory image is a self-location estimation result based on the shooting data of each time, that is, a line representing a trajectory connecting the shooting points of each time, drawn as viewed from a viewpoint in the sky.

[0058] In the overhead trajectory image, the trajectory high-precision section, the acceptable accuracy section, and the section recommended for deletion set on the trajectory are depicted in the colors assigned to them. For example, the high-precision section is depicted in light blue, the acceptable accuracy section is depicted in green, and the section recommended for deletion is depicted in red. This allows the user to determine the accuracy of the point cloud data generated from the photographed data at each photographing point.

[0059] In the first example shown in FIG. 3 and FIG. 4, after a section where the trajectory forms a loop (loop section), there is a section where the trajectory does not form a loop. In the loop section, a loop closing process is performed, and position correction is performed throughout. For this reason, the loop section is set as a high-precision section. On the other hand, in a section where the trajectory does not form a loop, the loop closing process is not performed. In this section, the accuracy is highest at the shooting point that is the starting point of the loop closing process, and the accuracy decreases as the distance from the starting point of the loop closing process increases. For this reason, a section close to the starting point of the loop closing process is set as an acceptable accuracy section, and a section away from the starting point of the loop closing process is set as a section recommended for deletion.

[0060] In this embodiment, each section (high accuracy section, acceptable accuracy section, and section recommended for deletion) classified according to the accuracy of point cloud data (measurement results) generated from the photographing data of each photographing location is visualized by color coding, but each section may be visualized by changing a display mode other than color. For example, the display mode may be changed for pattern and animation elements. Here, pattern elements include line types, designs, etc. Also, animation elements include blinking display, etc.

[0061] In addition, in this embodiment, the accuracy of the point cloud data generated from the shooting data at each shooting location is visualized in stages by dividing it into multiple sections, but the accuracy of the point cloud data generated from the shooting data at each shooting location may be expressed in a seamless manner by using a drawing method such as gradually changing the gradation of the trajectory image depending on the accuracy of the point cloud data generated from the shooting data at each shooting location.

[0062] Next, a second example of the map image will be described below. Fig. 5 is an explanatory diagram showing an overhead trajectory image according to the second example of the map image.

[0063] In the second example shown in Fig. 5, there are two sections (loop sections) where the trajectory forms a loop, and the loop closing process is performed twice. The sections before the starting point of the last loop closing process, i.e., the two loop sections and the sections between them, are set as high-precision sections. In the sections after the starting point of the last loop closing process, i.e., the sections where the loop closing process has not been performed, the sections close to the starting point of the loop closing process are set as acceptable accuracy sections, and the sections away from the starting point of the loop closing process are set as recommended deletion sections, as in the first example (see Figs. 3 and 4).

[0064] Next, a third example of the map image will be described below. Fig. 6 is an explanatory diagram showing an overhead view trajectory image according to the third example of the map image.

[0065] In the third example shown in FIG. 6, contrary to the first example (see FIG. 3 and FIG. 4), a section where the trajectory does not form a loop exists before a section where the trajectory forms a loop (loop section). The loop section is set as a high-precision section. On the other hand, since the loop closing process is performed on a section from the shooting point where the loop is closed to the shooting start point as the starting point, the loop closing process is also performed on a section where the trajectory does not form a loop. Therefore, among the sections where the trajectory does not form a loop, the section close to the loop section is set as a high-precision section. On the other hand, among the sections where the trajectory does not form a loop, the section close to the shooting start point is set as a recommended deletion section because it is far from the starting point of the loop closing process.

[0066] In this embodiment, the section where the loop closing process has been performed is divided into a high-precision section and a section recommended for deletion, but an acceptable accuracy section may be set between the high-precision section and the section recommended for deletion depending on the amount of distance from the starting point of the loop closing process.

[0067] Next, a fourth example of the map image will be described below. Fig. 7 is an explanatory diagram showing an overhead trajectory image according to the fourth example of the map image.

[0068] In the fourth example shown in Fig. 7, all sections from the shooting start point to the shooting end point are included in the section where the trajectory forms a loop (loop section). Therefore, the loop closing process is performed on all sections from the shooting start point to the shooting end point. On the other hand, in this example, since the loop is large, the section close to the shooting start point is set as the section recommended for deletion because it is far from the starting point of the loop closing process. The other sections are set as high-precision sections.

[0069] Next, a fifth example of the map image will be described below. Fig. 8 is an explanatory diagram showing an overhead view trajectory image according to the fifth example of the map image.

[0070] In the fifth example shown in FIG. 8, like the first example (see FIG. 3 and FIG. 4), there is a section where the trajectory does not form a loop after the section where the trajectory forms a loop (loop section). Furthermore, in this example, like the third example (see FIG. 6), there is a section where the trajectory does not form a loop before the loop section. In this example, the loop closing process is performed in a section from the shooting point where the loop is closed to the shooting start point. In the section where the loop closing process has been performed, the section close to the shooting start point is set as a section recommended for deletion because it is far from the starting point of the loop closing process, and the other sections are set as high-precision sections. On the other hand, in the section where the loop closing process has not been performed, the section close to the shooting end point is set as a section recommended for deletion because it is far from the starting point of the loop closing process, and the other sections are set as allowable accuracy sections.

[0071] Next, the timeline image will be described below. Figures 9 and 10 are explanatory diagrams showing examples of the timeline image.

[0072] In this embodiment, a timeline image is generated and presented to a user. In the timeline image, the accuracy of point cloud data (measurement results) generated from shooting data for each shooting time is visualized along the time axis. Specifically, in the timeline image, high-precision sections, acceptable accuracy sections, and sections recommended for deletion set on the time axis are drawn in different colors. For example, high-precision sections are drawn in light blue, acceptable accuracy sections are drawn in green, and sections recommended for deletion are drawn in red. Note that in the timeline image, it is preferable to use the same colors as in the map image (see FIGS. 3 to 8).

[0073] Here, the timeline image shown in FIG. 9(A) is for the first example (see FIG. 3 and FIG. 4). The timeline image shown in FIG. 9(B) is for the second example (see FIG. 5). The timeline image shown in FIG. 10(A) is for the third example (see FIG. 6). The timeline image shown in FIG. 10(B) is for the fourth example (see FIG. 7). The timeline image shown in FIG. 10(C) is for the fifth example (see FIG. 8).

[0074] Next, a description will be given of a schematic configuration of the photographing device 1. Fig. 11 is a block diagram showing a schematic configuration of the photographing device 1.

[0075] In addition to the sensor unit 12, the photographing device 1 includes a display 13 (display unit), an input device 14, a memory 15 (storage unit), and a processor 16 (CPU).

[0076] In addition to the visible camera 21, the sensor unit 12 includes a depth camera 22 and an IMU 23 (Inertial Measurement Unit). The depth camera 22 is a stereo camera that detects infrared light to capture an image of a subject, and outputs depth information (distance image) as a detection result. Based on the detection result of the depth camera 22, the distance to the subject can be measured. Note that the depth camera 22 may be a sensor capable of acquiring depth information of other methods, such as LiDAR, in addition to the stereo camera. The IMU 23 detects three-dimensional angular velocity and acceleration. Based on the detection result of the IMU 23, the movement amount and rotation amount of the imaging device 1 can be measured. Note that the visible camera 21, the depth camera 22, and the IMU 23 may not be integrated as a sensor unit. Also, the depth camera 22 and the IMU 23 may be omitted, and only the visible camera 21 may be provided.

[0077] The display 13 presents various information related to the photographing operation to the user, and displays a photographing screen 101 (see FIG. 13) and the like. The input device 14 is used by the user to perform input operations. The input device 14 may be a keyboard, a mouse, a touch pad, a touch panel, or the like. When the photographing apparatus 1 is configured as a tablet terminal, a touch panel display in which a touch panel as the input device 14 and a display panel as the display 13 are integrated is provided.

[0078] The memory 15 stores programs executed by the processor 16, etc. The memory 15 also stores imaging data for each imaging point. The imaging data includes data such as images captured by the visible camera 21, the detection results of the depth camera 22 and the IMU 23, the imaging time, and features extracted from the captured images. The memory 15 also stores processing information related to the implementation status of the loop closing process. The processing information includes information such as whether or not the loop closing process has been implemented for each imaging point. The memory 15 also stores measurement results generated by the processor 16.

[0079] The processor 16 performs various processes by executing the programs stored in the memory 15. In this embodiment, the processor 16 performs a detection information acquisition process, a point cloud generation process, a section determination process, a first accuracy visualization process, a second accuracy visualization process, a screen control process, and a shooting data deletion process.

[0080] In the detection information acquisition process, the processor 16 acquires an image captured by the visible camera 21. The processor 16 also acquires the detection results of the depth camera 22 and the IMU 23.

[0081] In the point cloud generation process (three-dimensional measurement process), the processor 16 generates point cloud data (environment map) as three-dimensional spatial information related to the measurement target location using the SLAM method based on the captured images of each point sequentially acquired by the visible camera 21. In the point cloud generation process, self-position estimation is performed in addition to the generation of the point cloud data, and the self-position for each time, i.e., the position of the photographing point, is acquired.

[0082] The point cloud generation process also includes a loop detection process. In the loop detection process, the processor 16 detects that the trajectory of the own aircraft forms a loop, that is, that the own aircraft has returned to a location where the own aircraft had previously captured an image. In this case, the detection of the location where the own aircraft had previously captured an image is not performed by comparing the image with a preset specified captured image, but rather, if the current captured image is similar to any of the past captured images, it is determined that the own aircraft has returned to a location where the own aircraft had previously captured an image. Note that the similarity of the captured images may be determined based on a feature amount extracted from the captured image.

[0083] The point cloud generation process also includes a loop closing process. When a loop is detected in the loop detection process, the loop closing process is performed. In the loop closing process, the processor 16 regards the self-position estimation result (past position) acquired during the previous shooting as the correct position for the previously shot point, and corrects the position of each shooting point on the trajectory from the current position to the past position.

[0084] In the section determination process, the processor 16 calculates the distance from the starting point of the loop closing process to the target shooting point as an index of accuracy of the point cloud data (measurement results) generated from the shooting data of each shooting point, and compares the distance for each shooting point with a predetermined determination criterion to determine whether each shooting point falls into a high-precision section, an acceptable accuracy section, or a section recommended for deletion (see FIG. 12).

[0085] In the first accuracy visualization process, the processor 16 generates a map image (see Figs. 3 to 8). In the map image, the accuracy of point cloud data generated from the photographing data for each photographing location is visualized along a trajectory. Specifically, in the map image, a high accuracy section, an acceptable accuracy section, and a section recommended for deletion set on the trajectory are depicted in different colors. The map image may be an overhead point cloud trajectory image (see Fig. 3) or an overhead trajectory image (see Figs. 4 to 8).

[0086] In the second accuracy visualization process, the processor 16 generates a timeline image (see Figs. 9 and 10). In the timeline image, the accuracy of point cloud data generated from the shooting data for each shooting location (shooting time) is visualized along the time axis. Specifically, in the timeline image, the high accuracy section, the allowable accuracy section, and the section recommended for deletion set within the shooting work period from the shooting start time (shooting time at the shooting start location) to the shooting end time (shooting time at the shooting end location) set on the time axis are depicted in different colors.

[0087] In the screen control process, the processor 16 controls the screen to be displayed on the display 13. In this embodiment, a shooting screen 101 (see FIG. 13) and a shooting data adjustment screen 201 (see FIG. 14) are generated and displayed on the display 13.

[0088] In the shooting data deletion process, the processor 16 deletes the shooting data stored in the memory 15 that is included in the section specified by the user or the section recommended for deletion, in accordance with the user's instruction operation on the shooting data adjustment screen 201 (see Figure 14).

[0089] In this embodiment, the image capturing device 1 performs the three-dimensional measurement process (point cloud generation process). However, the three-dimensional measurement process may be performed by a server device (not shown) that can communicate with the image capturing device 1.

[0090] Next, a description will be given of the section determination process performed by the processor 16. Fig. 12 is a flow chart showing the procedure of the section determination process.

[0091] In the section determination process, the distance from the starting point of the loop closing process to the target shooting point is calculated as an index of accuracy of the point cloud data (measurement result) generated from the shooting data of each shooting point, and the distance for each shooting point is compared with a predetermined determination criterion to determine whether each shooting point corresponds to a high-precision section, an acceptable accuracy section, or a section recommended for deletion. At this time, if the target shooting point is included in a section where the loop closing process has not been performed, a determination criterion (first threshold value) for the section where the loop closing process has not been performed is used. On the other hand, if the target shooting point is included in a section where the loop closing process has been performed, a determination criterion (second threshold value) for the section where the loop closing process has been performed is used.

[0092] Specifically, first, the processor 16 calculates the amount of separation between the imaging location to be determined (ST101). The amount of separation may be the time between the imaging time at the starting point of the loop closing process and the imaging time at the target imaging location. The amount of separation may also be the distance (distance on the trajectory) between the starting point of the loop closing process and the target imaging location. The amount of separation may also be the number of times (number of frames) images are captured between the starting point of the loop closing process and the target imaging location. The amount of separation may also be the number of key frames of the SLAM method captured between the starting point of the loop closing process and the target imaging location.

[0093] Next, the processor 16 determines whether or not the shooting location to be determined is included in a section in which the loop closing process has already been performed (ST102).

[0094] Here, if the shooting location to be determined is not included in the section where the loop closing process has been performed, that is, if the shooting location to be determined is included in the section where the loop closing process has not been performed (No in ST102), then the processor 16 determines whether the amount of deviation is equal to or greater than a first threshold (ST103). If the amount of deviation is less than the first threshold (No in ST103), the processor 16 sets the shooting location to be determined to the allowable accuracy section (ST105). On the other hand, if the amount of deviation is equal to or greater than the first threshold (Yes in ST103), the processor 16 sets the shooting location to be determined to the deletion recommended section (ST106).

[0095] If the shooting location to be determined is included in a section where the loop closing process has been performed (Yes in ST102), the processor 16 then determines whether the amount of separation is equal to or greater than a second threshold (ST104). If the amount of separation is less than the second threshold (No in ST104), the processor 16 sets the shooting location to be determined as a high-precision section (ST107). On the other hand, if the amount of separation is equal to or greater than the second threshold (Yes in ST104), the processor 16 sets the shooting location to be determined as a deletion recommended section (ST106).

[0096] Such section determination processing is performed for each shooting location, and high-precision sections, acceptable accuracy sections, and sections recommended for deletion are set on the trajectory, and high-precision sections, acceptable accuracy sections, and sections recommended for deletion are also set within the shooting work period from the shooting start time to the shooting end time.

[0097] Next, a description will be given of the shooting screen 101 displayed on the display 13. FIG.

[0098] A main window 102 (main image display frame) and a sub-window 103 (sub-image display frame) are provided on the shooting screen 101. The display magnifications of the main window 102 and the sub-window 103 are different, so that the image is displayed enlarged in the main window 102 and reduced in the sub-window 103.

[0099] Further, the main window 102 and the sub window 103 each display a captured image 121 and a map image 122. The captured image 121 is a current captured image, that is, an image captured in real time and output from the visible camera 21. The map image 122 is an overhead point cloud image, that is, an image (rendering) of each point of the point cloud data as viewed from a viewpoint in the sky. Note that, instead of the overhead point cloud image, an overhead point cloud trajectory image (see FIG. 3) or an overhead trajectory image (see FIGS. 4 to 8) may be displayed on the map image 122.

[0100] The shooting screen 101 is also provided with a "Start shooting" button 105 and a "Recording confirmation" button 106. When the user operates the "Start shooting" button 105, shooting by the visible camera 21 starts, and the captured image is stored in the memory 15. When the user operates the "Recording confirmation" button 106, the screen transitions to a shooting status confirmation screen (see FIG. 14). When the shooting work is completed, the user can check the shooting status by displaying the shooting status confirmation screen. Furthermore, the user can interrupt the shooting work at a necessary timing during the shooting work and display the shooting status confirmation screen to check the shooting status up to the present. In this case, the shooting work can be resumed by returning to the shooting screen 101.

[0101] Furthermore, the photographing screen 101 is provided with a "screen switching" button 107 and a check box 108 related to displaying the sub-window 103. When the user operates the "screen switching" button 107, a state is switched between one in which the photographed image 121 is enlarged and displayed in the main window 102 and the map image 122 is reduced and displayed in the sub-window 103 (the state shown in FIG. 13) and one in which the map image 122 is enlarged and displayed in the main window 102 and the photographed image 121 is reduced and displayed in the sub-window 103. When the user checks the check box 108, the state transitions to one in which the sub-window 103 is not displayed.

[0102] Next, a description will be given of the shooting data adjustment screen 201 displayed on the display 13. Fig. 14 is an explanatory diagram showing the shooting data adjustment screen 201.

[0103] The shooting data adjustment screen 201 is provided with a first shooting data adjustment section 202 and a second shooting data adjustment section 203.

[0104] In the first photographing data adjustment unit 202, a map image 211 is displayed. The map image 211 is generated by the first accuracy visualization process. In the map image 211, the accuracy of point cloud data (measurement results) generated from photographing data for each photographing point is visualized along a trajectory that represents the transition of the photographing points. Specifically, in the map image 211, a high accuracy section, an allowable accuracy section, and a section recommended for deletion set on the trajectory are drawn in different colors. For example, the high accuracy section is drawn in light blue, the allowable accuracy section is drawn in green, and the section recommended for deletion is drawn in red.

[0105] In the example shown in Figure 14, an overhead point cloud trajectory image (see Figure 3) is displayed as map image 211, but an overhead trajectory image (see Figures 4 to 8) in which the overhead point cloud image has been removed from the overhead point cloud trajectory image may also be displayed as map image 211.

[0106] In addition, the first photography data adjustment unit 202 displays a start point mark 212 and an end point mark 213 for specifying photography locations that are the start point and end point of a range in which photography data is deleted. Specifically, when the user performs an operation to select the photography location marks that are the start point and end point of a range in which photography data is deleted from among the photography location marks drawn on the overhead trajectory image included in the map image 211, the start point mark 212 and the end point mark 213 are displayed on the selected photography location mark.

[0107] In addition, in the first photographing data adjustment unit 202, when the user performs an operation to select a recommended deletion section on the trajectory in the map image 211, for example, a click operation on the recommended deletion section, the recommended deletion section is designated as a range from which photographing data is to be deleted. At this time, the start mark 212 and the end mark 213 move to the positions of the start and end points of the recommended deletion section. Note that the user may operate the ranges of the recommended deletion section and the acceptable accuracy section in the map image 211 to designate both the recommended deletion section and the acceptable accuracy section as ranges from which photographing data is to be deleted.

[0108] In the second photographing data adjustment unit 203, a timeline image 221 (see Figs. 9 and 10) is displayed. The timeline image 221 is generated by the second accuracy visualization process. In the timeline image 221, the accuracy of point cloud data (measurement results) generated from photographing data for each photographing time is visualized along the time axis. Specifically, in the timeline image 221, a high accuracy section, an acceptable accuracy section, and a section recommended for deletion set on the time axis are drawn in different colors. For example, a high accuracy section is drawn in light blue, an acceptable accuracy section is drawn in green, and a section recommended for deletion is drawn in red.

[0109] It is preferable that the color coding of the high-precision section, the allowable-precision section, and the section recommended for deletion is common to the first and second shooting data adjustment units 202, 203. Furthermore, as shown in Fig. 9 and Fig. 10, characters or figures indicating a section where the loop closing process has been performed, a starting point of the loop closing process, a section where the trajectory forms a loop (loop section), etc. may be added to the timeline image 221.

[0110] Also, the second photographing data adjustment unit 203 displays a start mark 222 and an end mark 223 for specifying the photographing times that are the start and end points of the range in which photographing data is to be deleted. The user can adjust the photographing times that are the start and end points of the range in which photographing data is to be deleted by performing an operation (e.g., a slide operation) to move the start mark 222 and the end mark 223.

[0111] In addition, in the second shooting data adjustment unit 203, when the user performs an operation to select a section recommended for deletion in the timeline image 221, for example, a click operation on the section recommended for deletion, the section recommended for deletion is designated as a range in which shooting data is to be deleted. At this time, the start mark 222 and the end mark 223 move to the positions of the start and end points of the section recommended for deletion. Note that the user may operate the ranges of the section recommended for deletion and the section with acceptable accuracy in the timeline image 221 to designate both the section recommended for deletion and the section with acceptable accuracy as a range in which shooting data is to be deleted.

[0112] The operation of designating the range of the photographed data to be deleted may be performed by either the first or second photographed data adjustment unit 202, 203. When the operation of designating the range of the photographed data to be deleted is performed by either the first or second photographed data adjustment unit 202, 203, the designation content is reflected in the other. For example, when the user designates the range of the photographed data to be deleted by operating the start mark 212 and the end mark 213 in the first photographed data adjustment unit 202, the start mark 222 and the end mark 223 are displayed in the second photographed data adjustment unit 203 at the position of the photographed time corresponding to the range designated by the first photographed data adjustment unit 202.

[0113] The photographed image display section 204 is also provided on the photographed data adjustment screen 201. The photographed image display section 204 displays the photographed image at the photographed time and photographed location designated by the user in the first and second photographed data adjustment sections 202 and 203. Specifically, in the first photographed data adjustment section 202, when the user performs an operation (e.g., a tap operation) to designate an arbitrary position on the map image 211, the photographed image at the photographed location corresponding to the designated position is displayed. In addition, in the second photographed data adjustment section 203, when the user performs an operation (e.g., a tap operation) to designate an arbitrary position on the timeline image 221, the photographed image at the photographed time corresponding to the designated position is displayed.

[0114] The shooting data adjustment screen 201 is also provided with a "delete" button 205 and a "delete all" button 206. When the user operates the "delete" button 205, the shooting data included in the range designated by the user in the first and second shooting data adjustment units 202 and 203 is deleted. When the user operates the "delete all" button 206, the shooting data included in the deletion recommended section is deleted.

[0115] The shooting data adjustment screen 201 is also provided with a threshold setting section 207 for setting a threshold used in the section determination process. The threshold setting section 207 is provided with a first threshold input section 208 and a second threshold input section 209. The first threshold input section 208 allows the user to input a first threshold. The second threshold input section 209 allows the user to input a second threshold.

[0116] The first and second thresholds are used as criteria for determining whether or not a target shooting location corresponds to a recommended deletion section, i.e., a section in which the accuracy of point cloud data generated from shooting data is assumed to be lower than an acceptable range, in the section determination process. In particular, the first threshold is used as a determination criterion for a section in which loop closing process has not been performed. The second threshold is used as a determination criterion for a section in which loop closing process has been performed.

[0117] In the example shown in FIG. 14, the first and second thresholds are set for the amount of separation regarding time. The amount of separation regarding time represents the time between the shooting time at the starting point of the loop closing process and the shooting time at the target shooting location. In a section where the loop closing process has not been performed, if the amount of separation is equal to or greater than the first threshold, the target shooting location is determined to correspond to the section recommended for deletion. In a section where the loop closing process has been performed, if the amount of separation is equal to or greater than the second threshold, the target shooting location is determined to correspond to the section recommended for deletion.

[0118] In this manner, in this embodiment, the first threshold value, which is a judgment criterion for sections where loop closing processing has not been performed, and the second threshold value, which is a judgment criterion for sections where loop closing processing has been performed, can be set to different values.

[0119] The amount of separation may be the distance (distance on the trajectory) between the starting point of the loop closing process and the target shooting point. The amount of separation may also be the number of times (number of frames) images are taken between the starting point of the loop closing process and the target shooting point. The amount of separation may also be the number of key frames of the SLAM method taken between the starting point of the loop closing process and the target shooting point. The type of separation (time, distance, number of frames, number of key frames) may be changeable on a setting screen (not shown) or the shooting data adjustment screen 201.

[0120] In addition, the first and second threshold input sections 208 and 209 may display first and second thresholds (initial values) that are set in advance, and the user may change the first and second thresholds as necessary. Although a threshold can be input into both the first threshold input section 208 and the second threshold input section 209, the other threshold may be automatically set in conjunction with the input of either one. In this case, it is preferable to set the first threshold to be greater than the second threshold using a predetermined calculation formula, regardless of which is input.

[0121] 14, one recommended deletion section is set in the first and second shooting data adjustment units 202 and 203, and one range in which shooting data is to be deleted is set by user specification, but multiple ranges in which shooting data is to be deleted may be set. For example, as in the example shown in FIG. 8, two recommended deletion sections, one on the shooting start point side and one on the shooting end point side, may be set, and two ranges in which shooting data is to be deleted by user specification.

[0122] In addition, in the example shown in FIG. 14, both the first and second shooting data adjustment units 202, 203 are provided on the shooting data adjustment screen 201, but only one of them may be provided on the shooting data adjustment screen 201.

[0123] In addition, in this embodiment, the recommended deletion section where the measurement result is expected to be insufficient in accuracy is visualized and presented to the user, and the shooting data included in the recommended deletion section or the section specified by the user is deleted by the user performing an operation to instruct the deletion, but the process of visualizing the recommended deletion section and presenting it to the user may be omitted, and the shooting data included in the recommended deletion section may be automatically deleted without the user's operation. In this case, for example, the user may set a first threshold value and a second threshold value in advance, and a macro command may be executed to automatically delete only the recommended deletion section when shooting is finished.

[0124] As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. In addition, it is also possible to combine the components described in the above embodiments to create new embodiments. [Industrial Applicability]

[0125] The imaging data processing device, imaging data processing method, and imaging data processing program of the present invention have the effect of allowing a user to easily identify, from the accumulated imaging data, the range of imaging data where the measurement results are likely to be insufficient in accuracy, and to efficiently delete the imaging data in the range that should be deleted.They are useful as imaging data processing device, imaging data processing method, and imaging data processing program that process imaging data from each point obtained by sequentially photographing the measurement target location with a camera as the user holding the device body moves, in order to perform 3D measurement processing that generates 3D spatial information of the measurement target location. [Explanation of symbols]

[0126] 1: Shooting device (photographic data processing device) 11: Device body 12: Sensor section 13: Display 14: Input devices 15: Memory 16: Processor 21: Visible camera 22: Depth camera 101: Shooting screen 201: Shooting data adjustment screen 202: First imaging data adjustment unit 203: Second imaging data adjustment unit 204: Photographed image display section 205: "Delete" button 206: "Bulk delete" button 207: Threshold setting unit 208: First threshold input section 209: Second threshold input section 211: Bird's-eye view map image 212: Start mark 213: End mark 221: Timeline image 222:Start mark 223: End mark

Claims

1. A photographing data processing device that processes photographing data of each point acquired by sequentially photographing a measurement target location with a camera while moving a device body in order to generate point cloud data as a three-dimensional measurement result of the measurement target location, a storage unit that stores the photographed data and processing information relating to the implementation status of the loop closing process in the three-dimensional measurement process; a display that displays a screen for confirming the accuracy of the point cloud data and the implementation status of the loop closing process; The processor: In order to adjust the photography data stored in the storage unit, a photography data adjustment unit including an image in which the accuracy of the point cloud data based on the photography data for each photography point is visualized is displayed on the screen of the display; 10. A photographic data processing device, comprising: a photographic data adjustment unit that, in response to a user's operation to specify a range, deletes photographic data included in a specified range.

2. The processor: displaying the photographing data adjustment unit including a map image in which the accuracy of the point cloud data is visualized along a trajectory representing a transition of photographing points; 2. The imaging data processing device according to claim 1, wherein, in response to a user's operation to designate a range on the map image, imaging data included in the designated range is deleted.

3. The processor: displaying on a screen the photographing data adjustment unit including a timeline image in which the accuracy of the point cloud data is visualized along a time axis; The photographic data processing device according to claim 1 , wherein, in response to a user's operation to specify a range on the timeline image, photographic data included in the specified range is deleted.

4. The processor:

2. The photographic data processing device according to claim 1, wherein, in response to a user's operation to specify a photographic timing, an image photographed by the camera at a specified photographic timing is displayed on the screen of the display.

5. The processor: determining a recommended deletion section for which the accuracy of the point cloud data is expected to be insufficient based on the processing information; 2. The imaging data processing device according to claim 1, wherein the recommended deletion section is visualized and displayed on the screen of the display.

6. The processor: an operation section for a user to instruct batch deletion is displayed on the screen of the display; The imaging data processing device according to claim 5 , wherein the imaging data included in the recommended deletion section is deleted in response to a user operation on the operation unit.

7. The processor: an input unit that allows a user to input a threshold value that serves as a criterion for determining the recommended deletion section is displayed on the screen of the display; 6. The imaging data processing device according to claim 5, wherein the threshold value is set in response to a user's input via the input unit.

8. A method for processing photographed data, in which a processor processes photographed data of each point acquired by sequentially photographing a measurement target location with a camera while moving a device body, in order to generate point cloud data as a three-dimensional measurement result of the measurement target location, In order to adjust the photography data stored in the storage unit, a photography data adjustment unit including an image in which the accuracy of the point cloud data based on the photography data for each photography point is visualized is displayed on a display screen; A method for processing photographic data, comprising the steps of: deleting photographic data included in a designated range in response to a user's operation to designate the range in the photographic data adjustment unit;

9. A photography data processing program that causes a processor to process photography data of each point acquired by sequentially photographing a measurement target location with a camera while moving the device body in order to generate point cloud data as a three-dimensional measurement result of the measurement target location, In order to adjust the photography data stored in the storage unit, a photography data adjustment unit including an image in which the accuracy of the point cloud data based on the photography data for each photography point is visualized is displayed on a display screen; 11. A computer program product for processing photographic data, comprising: a photographic data adjustment unit that, in response to a user's operation to designate a range, deletes photographic data included in a designated range.

Images