Image acquisition system and image acquisition method

Abstract

To photograph a photographic object by properly putting it in a photographic image.SOLUTION: An image acquisition system comprises: a photographing unit which photographs a photographic object to acquire a photographic image; a display unit which displays the photographic image acquired by the photographing unit; and a control unit which controls the photographing unit and the display unit. The control unit executes: recognizing a space reference point associated with the photographic object; displaying on the display unit a first photographic marker which is at a fixed positional relation with the photographic object, on the basis of the recognized space reference point; displaying on the display unit a second photographic marker which is at a fixed positional relation with the photographing unit; and recording the photographic image of the photographic object, when it is detected that the first photographic marker and the second photographic marker overlap.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present disclosure relates to an image acquisition system and an image acquisition method.

Background Art

[0002] Conventionally, an information processing apparatus including a photographing unit and a display unit has been known (see, for example, Patent Document 1). The information processing apparatus specifies the position and orientation of the photographing unit in space with reference to an AR marker, and superimposes and displays an object on a display unit that displays a photographed image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The photographed image may be used for, for example, image determination. The image used for image determination needs to contain the object in the image so that the object is at a predetermined photographing position. However, when photographing an image with reference to an AR marker as in Patent Document 1, the angle, size, position, etc. of the object vary, and it is difficult to acquire an image optimal for image determination.

[0005] Therefore, an object of the present disclosure is to provide an image acquisition system and an image acquisition method capable of appropriately capturing a photographing object in a photographed image.

Means for Solving the Problems

[0006] The image acquisition system of the present disclosure includes a photographing unit that photographs a subject to acquire a photographed image, a display unit that displays the photographed image acquired by the photographing unit, and a control unit that controls the photographing unit and the display unit. The control unit executes steps of recognizing a spatial reference point associated with the subject, displaying, on the display unit, a first photographing marker whose positional relationship with the subject is fixed based on the recognized spatial reference point, displaying, on the display unit, a second photographing marker whose positional relationship with the photographing unit is fixed, and recording the photographed image of the subject when it is detected that the first photographing marker and the second photographing marker overlap.

[0007] Further, the image acquisition method of the present disclosure is an image acquisition method executed by an image acquisition system that photographs a subject to acquire a photographed image. The image acquisition system executes steps of recognizing a spatial reference point associated with the subject, displaying, on a display unit that displays the photographed image, a first photographing marker whose positional relationship with the subject is fixed based on the recognized spatial reference point, displaying, on the display unit, a second photographing marker whose positional relationship with a photographing unit that photographs the photographed image is fixed, and recording the photographed image of the subject when it is detected that the first photographing marker and the second photographing marker overlap.

Advantages of the Invention

[0008] According to the present disclosure, the subject can be appropriately framed in the photographed image for photographing.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Mode for Carrying Out the Invention

[0010] Hereinafter, embodiments according to the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited by this embodiment. Further, the components in the following embodiments include those that can be replaced and are easy for those skilled in the art, or those that are substantially the same. Furthermore, the components described below can be combined as appropriate, and when there are a plurality of embodiments, the embodiments can also be combined.

[0011] [This Embodiment] The image acquisition system 10 according to the present embodiment is used, for example, as a system for recording the work content of aircraft maintenance work in images. Further, in the image acquisition system 10, it is determined whether the captured image of the work location is appropriate or not by image determination. Note that the image acquisition system 10 may be used in a system for conducting education and training on maintenance work, or may be used in a system for supporting the implementation of maintenance work. The image acquisition system 10 will be described with reference to FIG. 1.

[0012] (Image Acquisition System) FIG. 1 is a diagram of the image acquisition system according to the present embodiment. As shown in FIG. 1, the image acquisition system 10 includes a control unit 31, a storage unit 32, a display unit 33, an input unit 34, and a photographing unit 35. The image acquisition system 10 is a wearable terminal worn by an operator. Further, an operator terminal 39 is connected to the image acquisition system 10 via a communication network 37. Note that although not shown in the figure, the operator terminal 39 is, for example, a stationary terminal, and like the image acquisition system 10, has a display unit and an input unit, and can input information for recording work.

[0013] The control unit 31 includes, for example, an integrated circuit such as a CPU (Central Processing Unit). The control unit 31 executes data processing for recording an image of the maintenance work. The storage unit 32 is an arbitrary storage device such as a semiconductor storage device and a magnetic storage device. The photographed images taken by the photographing unit 35, information generated by various processes, etc. are stored in this storage unit 32. The display unit 33 is a display device such as a transmissive liquid crystal display, for example. The input unit 34 is an input device such as a keyboard and a mouse, for example. The photographing unit 35 is a camera, for example, and photographs a photographing object to obtain a photographed image.

[0014] The image acquisition system 10 executes a process of requesting the acquisition of a photographed image or executes a display process for the photographed image in order for the worker of the maintenance work to record an image of the work location of the maintenance work.

[0015] Next, with reference to FIGS. 2 and 3, the photographed image 50 displayed on the display unit 33 will be described. When acquiring the photographed image 50, the control unit 31 displays the photographed image 50 photographed by the photographing unit 35 on the display unit 33 and also displays the first photographing marker 51 and the second photographing marker 52.

[0016] Here, a spatial reference point 53 in a three-dimensional space is provided at the work location that is the photographing object. The spatial reference point 53 is, for example, a QR code (registered trademark) attached to the photographing object. Note that the spatial reference point 53 may be, for example, a feature point of the photographing object as the spatial reference point, and may be recognizable by any method.

[0017] The first imaging marker 51 is an object of the three-dimensional model displayed on the display unit 33. The first imaging marker 51 is a marker whose positional relationship with the object to be imaged is fixed with respect to the spatial reference point 53. Four first imaging markers 51 are provided and arranged in a rectangular grid pattern. The first imaging marker 51 has a wireframe display form and is spherical in shape. The wireframe is composed of wires along the three-dimensional directions of the vertical direction, the left-right direction, and the depth direction with the object to be imaged as the front. Therefore, by visually recognizing the first imaging marker 51, the operator can easily grasp the attitude angles of the object to be imaged in the roll direction, the pitch direction, and the yaw direction.

[0018] The second imaging marker 52 is an object of the three-dimensional model displayed on the display unit 33, similar to the first imaging marker 51. The second imaging marker 52 is a marker whose positional relationship with the imaging unit 35 is fixed with respect to the spatial reference point 53. Note that the positional relationship between the spatial reference point 53 and the imaging unit 35 is obtained by position detection. In the present embodiment, the positional relationship between the second imaging marker 52 and the imaging unit 35 is fixed with respect to the spatial reference point 53, but the positional relationship between the second imaging marker 52 and the imaging unit 35 may be fixed with respect to the imaging unit 35. Four second imaging markers 52 are provided, similar to the first imaging markers 51, arranged in a rectangular grid pattern, and have a spherical shape with a wireframe display form.

[0019] Here, one of the display ranges of the first imaging marker 51 and the second imaging marker 52 is large and the other is small. For example, the size of the display range of the first imaging marker 51 is smaller than that of the second imaging marker 52. Also, the first imaging marker 51 and the second imaging marker 52 have different colors. As shown in FIG. 3, when the first imaging marker 51 and the second imaging marker 52 are in a superimposed state, the colors change to the same color.

[0020] When the control unit 31 detects that the first imaging marker 51 and the second imaging marker 52 are in a superimposed state, it records the captured image 50 of the object to be imaged. At this time, when the first imaging marker 51 and the second imaging marker 52 are superimposed, the control unit 31 displays the time until the recording of the captured image 50 of the object to be imaged is completed. Specifically, as shown in FIG. 3, the control unit 31 displays a progress bar 54 on the display unit 33. The progress bar 54 represents the time until recording completion (imaging completion) as the progress rate of the imaging task. Note that the display form of the time until recording completion is not limited to the progress bar 54, and any display form may be applied.

[0021] In addition, the first imaging marker 51 includes position information indicating the first rotational position in the captured image 50. Since four first imaging markers 51 are arranged, for example, 0° of position information is assigned to the first imaging marker 51 located in the upper right shown in FIG. 2. Similarly, 90° of position information is assigned to the first imaging marker 51 located in the lower right, 180° of position information is assigned to the first imaging marker 51 located in the lower left, and 270° of position information is assigned to the first imaging marker 51 located in the upper left.

[0022] Similar to the first imaging marker 51, the second imaging marker 52 includes position information indicating the second rotational position in the captured image 50. Since four second imaging markers 52 are arranged, for example, 0° of position information is assigned to the first imaging marker 51 located in the upper right shown in FIG. 2. Similarly, 90° of position information is assigned to the first imaging marker 51 located in the lower right, 180° of position information is assigned to the first imaging marker 51 located in the lower left, and 270° of position information is assigned to the first imaging marker 51 located in the upper left.

[0023] After the recording of the captured image 50 of the object to be imaged is completed, when the first rotational position and the second rotational position are different, the control unit 31 rotates the captured image 50 so that the first rotational position and the second rotational position match, and records the rotated captured image 50.

[0024] (Image acquisition method) Next, with reference to FIG. 4, the image acquisition method according to the present embodiment will be described. FIG. 4 is a flowchart relating to the image acquisition method according to the present embodiment. When the control unit 31 of the image acquisition system 10 records an image with the work location of the maintenance work as the imaging object, first, the control unit 31 recognizes the spatial reference point 53 of the imaging object imaged by the imaging unit 35 (step S1).

[0025] After executing step S1, the control unit 31 calculates the imaging location of the imaging object in the space of the three-dimensional model based on the recognized spatial reference point (step S2). That is, in step S2, the control unit 31 performs superposition of the real space and the virtual space based on the spatial reference point.

[0026] After executing step S2, the control unit 31 superimposes and displays the first imaging marker 51 on the captured image 50 captured by the imaging unit 35 and displayed on the display unit 33 based on the calculated imaging location (step S3). Note that after executing step S3, the control unit 31 may perform notification to the effect that recording of the captured image 50 of the imaging object is to be executed.

[0027] After executing step S3, the control unit 31 superimposes and displays the second imaging marker 52 fixed to the imaging unit 35 on the captured image 50 (step S4). After that, the control unit 31 detects that the first imaging marker 51 and the second imaging marker 52 are superimposed and executes recording of the captured image 50 (step S5). In step S5, the control unit 31 detects the superposition of the first imaging marker 51 and the second imaging marker 52 based on the distance between the center position of the first imaging marker 51 and the center position of the second imaging marker 52. Specifically, the control unit 31 determines that they are superimposed if the distance between the center position of the first imaging marker 51 and the center position of the second imaging marker 52 is shorter than a preset distance, and determines that they are not superimposed if the distance is equal to or greater than the preset distance. Also, in step S5, when the first imaging marker 51 and the second imaging marker 52 are superimposed, the control unit 31 displays the progress bar 54 shown in FIG. 3 and displays the time until recording of the captured image 50 is completed.

[0028] In addition, after the recording of the captured image 50 of the object to be captured is completed, the control unit 31 may output a capture completion sound from a speaker (not shown).

[0029] After executing step S6, the control unit 31 determines the quality of the work location by image determination based on the acquired captured image 50 (step S6), and ends the process related to the image acquisition method.

[0030] As described above, the image acquisition system 10 and the image acquisition method described in this embodiment are understood as follows, for example.

[0031] The image acquisition system 10 according to the first aspect includes a photographing unit 35 that photographs an object to be photographed and acquires a photographed image, a display unit 33 that displays the photographed image acquired by the photographing unit 35, and a control unit 31 that controls the photographing unit 35 and the display unit 33. The control unit 31 executes step S1 of recognizing a spatial reference point 53 associated with the object to be photographed, step S3 of displaying a first photographing marker 51 having a fixed positional relationship with the object to be photographed on the display unit 33 based on the recognized spatial reference point 53, step S4 of displaying a second photographing marker 52 having a fixed positional relationship with the photographing unit 35 on the display unit 33, and step S5 of recording the photographed image 50 of the object to be photographed when it is detected that the first photographing marker 51 and the second photographing marker 52 overlap.

[0032] According to this configuration, by superimposing the first photographing marker 51 and the second photographing marker 52 and recording the photographed image 50, the positional relationship between the photographing unit 35 and the object to be photographed can be photographed in a state where the positional relationship is suitable for image determination. Therefore, the object to be photographed can be appropriately captured in the photographed image 50 and photographed.

[0033] As a second aspect, in the image acquisition system 10 according to the first aspect, each of the first photographing marker 51 and the second photographing marker 52 is arranged in a square lattice pattern of four.

[0034] According to this configuration, it is possible to appropriately perform alignment between the first imaging marker 51 and the second imaging marker 52.

[0035] As a third aspect, in the image acquisition system 10 according to the first or second aspect, each of the first imaging marker 51 and the second imaging marker 52 has a wireframe display form and a spherical shape.

[0036] According to this configuration, even when the first imaging marker 51 and the second imaging marker 52 overlap, it is possible to perform appropriate alignment while ensuring the visibility of the first imaging marker 51 and the second imaging marker 52.

[0037] As a fourth aspect, in the image acquisition system 10 according to any one of the first to third aspects, one of the first imaging marker 51 and the second imaging marker 52 has a large display range and the other has a small display range.

[0038] According to this configuration, it is possible to make it easy to align the positions of the first imaging marker 51 and the second imaging marker 52.

[0039] As a fifth aspect, in the image acquisition system 10 according to any one of the first to fourth aspects, the first imaging marker 51 and the second imaging marker 52 have different colors, and when the first imaging marker 51 and the second imaging marker 52 overlap, the control unit 31 changes the colors of the first imaging marker 51 and the second imaging marker 52 to the same color.

[0040] According to this configuration, it becomes possible to easily grasp that the first imaging marker 51 and the second imaging marker 52 overlap.

[0041] As a sixth aspect, in the image acquisition system 10 according to any one of the first to fifth aspects, when the first imaging marker 51 and the second imaging marker 52 overlap, the control unit 31 displays the time until the recording of the captured image 50 of the object to be imaged is completed.

[0042] According to this configuration, it is possible to prompt the operator to keep the position of the imaging unit 35 with respect to the object to be imaged stationary until the recording is completed.

[0043] As a seventh aspect, in the image acquisition system 10 according to any one of the first to sixth aspects, the image acquisition system 10 further includes a speaker, and after the recording of the captured image of the object to be imaged is completed, the control unit 31 causes the speaker to output a shooting completion sound.

[0044] According to this configuration, it is possible to prompt the operator to keep the position of the imaging unit 35 with respect to the object to be imaged stationary until the recording is completed.

[0045] As an eighth aspect, in the image acquisition system 10 according to any one of the first to seventh aspects, the control unit 31 detects the center position of the first imaging marker 51 and the center position of the second imaging marker 52, and based on the distance between the center positions, detects the overlap of the first imaging marker 51 and the second imaging marker 52.

[0046] According to this configuration, the overlap between the first imaging marker 51 and the second imaging marker 52 can be easily detected.

[0047] As a ninth aspect, in the image acquisition system 10 according to any one of the first to eighth aspects, the first imaging marker 51 includes position information indicating a first rotation position of the captured image 50, the second imaging marker 52 includes position information indicating a second rotation position of the captured image 50, and after the recording of the captured image 50 of the object to be imaged is completed, when the first rotation position and the second rotation position are different, the control unit 31 rotates the captured image 50 so that the first rotation position and the second rotation position coincide.

[0048] According to this configuration, even when the captured image 50 is recorded in a rotated state, it can be recorded in a state where the rotation state is corrected.

[0049] The image acquisition method according to the tenth aspect is an image acquisition method executed by an image acquisition system 10 that captures an object to be imaged and acquires a captured image, the image acquisition system 10 including a step S1 of recognizing a spatial reference point 53 associated with the object to be imaged, a step S3 of displaying a first imaging marker 51 having a fixed positional relationship with the object to be imaged on a display unit 33 that displays the captured image 50 based on the recognized spatial reference point 53, a step S4 of displaying a second imaging marker 52 having a fixed positional relationship with an imaging unit 35 that captures the captured image 50 on the display unit 33, and a step S5 of recording the captured image 50 of the object to be imaged when it is detected that the first imaging marker 51 and the second imaging marker 52 overlap.

[0050] According to this configuration, by superimposing the first imaging marker 51 and the second imaging marker 52 and recording the captured image 50, the positional relationship between the imaging unit 35 and the object to be imaged can be set to a positional relationship suitable for image determination for imaging. Therefore, the object to be imaged can be appropriately framed in the captured image 50 for imaging.

Description of Reference Numerals

[0051] 10 Image acquisition system 31 Control unit 32 Memory unit 33 Display unit 34 Input unit 35 Imaging unit 37 Communication network 39 Operator terminal 50 Captured image 51 First imaging marker 52 Second imaging marker 53 Spatial reference point

Claims

1. An imaging unit that captures an object to be imaged and acquires a captured image; A display unit that displays the captured image acquired by the imaging unit; A control unit that controls the imaging unit and the display unit, and the control unit recognizes a spatial reference point associated with the object to be imaged; based on the recognized spatial reference point, displays a first imaging marker with a fixed positional relationship with the object to be imaged on the display unit; displays a second imaging marker with a fixed positional relationship with the imaging unit on the display unit; When it is detected that the first imaging marker and the second imaging marker overlap, records the captured image of the object to be imaged. An image acquisition system that performs

2. The image acquisition system according to claim 1, wherein each of the first imaging marker and the second imaging marker is arranged in a square lattice pattern of four.

3. The image acquisition system according to claim 1, wherein each of the first imaging marker and the second imaging marker has a wireframe display form and a spherical shape.

4. The image acquisition system according to claim 1, wherein the first imaging marker and the second imaging marker have one large display range and the other small display range.

5. The first imaging marker and the second imaging marker have different colors, and the control unit When the first imaging marker and the second imaging marker overlap, changes the colors of the first imaging marker and the second imaging marker to be the same color. The image acquisition system according to claim 1.

6. The control unit The image acquisition system according to claim 1, wherein when the first imaging marker and the second imaging marker overlap, the time until recording of the captured image of the object to be imaged is completed is displayed.

7. further comprising a speaker, wherein the control unit causes the speaker to output a shooting completion sound after recording of the captured image of the object to be imaged is completed, according to the image acquisition system of claim 1.

8. wherein the control unit detects a center position of the first imaging marker and a center position of the second imaging marker, and detects an overlap between the first imaging marker and the second imaging marker based on a distance between the center positions, according to the image acquisition system of claim 1.

9. the first imaging marker includes position information indicating a first rotation position of the captured image, the second imaging marker includes position information indicating a second rotation position of the captured image, wherein the control unit after recording of the captured image of the object to be imaged is completed, if the first rotation position and the second rotation position are different, rotates the captured image so that the first rotation position and the second rotation position coincide, according to the image acquisition system of claim 1.

10. An image acquisition method executed by an image acquisition system that captures an object to be imaged and acquires a captured image, the image acquisition system recognizing a spatial reference point associated with the object to be imaged; based on the recognized spatial reference point, displaying a first imaging marker whose positional relationship with the object to be imaged is fixed on a display unit that displays the captured image; displaying a second imaging marker whose positional relationship with an imaging unit that captures the captured image is fixed on the display unit; An image acquisition method that executes a step of recording the captured image of the object to be captured when it is detected that the first imaging marker and the second imaging marker overlap.

Images