Image acquisition system and image acquisition method

Abstract

The present invention comprises: an imaging unit that images an object-to-be-imaged and acquires the captured image; a display unit that displays the captured image acquired by the imaging unit; and a control unit that controls the imaging unit and the display unit. The control unit executes: a step for recognizing a spatial reference point associated with the object-to-be-imaged; a step for displaying, on the display unit on the basis of the recognized spatial reference point, a first imaging marker of which the positional relationship with the object-to-be-imaged is fixed; a step for displaying, on the display unit, a second imaging marker of which the positional relationship with the imaging unit is fixed; and a step for recording the captured image of the object-to-be-imaged when the superposition of the first imaging marker and the second imaging marker is detected.

Description

Image acquisition system and image acquisition method

[0001] The present disclosure relates to an image acquisition system and an image acquisition method.

[0002] Conventionally, there has been known an information processing device including an image capturing unit and a display unit (see, for example, Patent Document 1). The information processing device identifies the position and orientation of the image capturing unit in space using an AR marker as a reference, and displays an object superimposed on the display unit that displays the captured image.

[0003] Japanese Patent Application Laid-Open No. 2023-045749

[0004] The captured image may be used for image judgment, for example. The image used for image judgment needs to be captured so that the object included in the image is at a predetermined shooting position. However, as in Patent Document 1, when an image is captured using an AR marker as a reference, the angle, size, position, etc. of the object fluctuate, making it difficult to obtain an image that is optimal for image judgment.

[0005] Therefore, an object of the present disclosure is to provide an image acquisition system and an image acquisition method that can capture an image of an object to be photographed while properly including the object in the captured image.

[0006] The image acquisition system of the present disclosure includes a photographing unit that photographs an object to be photographed and acquires a photographed image, a display unit that displays the photographed image acquired by the photographing unit, and a control unit that controls the photographing unit and the display unit, and the control unit executes the steps of recognizing a spatial reference point associated with the object to be photographed, displaying a first photographing marker on the display unit that has a fixed positional relationship with the object to be photographed based on the recognized spatial reference point, displaying a second photographing marker on the display unit that has a fixed positional relationship with the photographing unit, and recording the photographed image of the object to be photographed when it is detected that the first photographing marker and the second photographing marker overlap.

[0007] In addition, the image acquisition method disclosed herein is an image acquisition method executed by an image acquisition system that photographs an object to be photographed and acquires a photographed image, and the image acquisition system executes the following steps: recognizing a spatial reference point associated with the object to be photographed; displaying a first photographing marker, which has a fixed positional relationship with the object to be photographed based on the recognized spatial reference point, on a display unit that displays the photographed image; displaying a second photographing marker, which has a fixed positional relationship with a photographing unit that photographs the photographed image, on the display unit; and recording the photographed image of the object to be photographed when it detects that the first photographing marker and the second photographing marker overlap.

[0008] According to the present disclosure, it is possible to capture an object to be photographed while appropriately including it in a captured image.

[0009] Fig. 1 is a diagram of an image acquisition system according to this embodiment. Fig. 2 is a diagram showing an example of a captured image displayed on a display unit. Fig. 3 is a diagram showing an example of a captured image displayed on a display unit. Fig. 4 is a flowchart relating to an image acquisition method according to this embodiment.

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. However, this disclosure is not limited to these embodiments. Furthermore, the components in the following embodiments include those that are easily replaceable by those skilled in the art, or those that are substantially identical. Furthermore, the components described below can be combined as appropriate, and when there are multiple embodiments, the respective embodiments can also be combined.

[0011] [Present Embodiment] The image acquisition system 10 according to this embodiment is used, for example, as a system for recording the details of aircraft maintenance work using images. The image acquisition system 10 also performs image evaluation to determine whether the captured images of the work location are appropriate. The image acquisition system 10 may be used in a system for providing education and training on maintenance work, or in a system for supporting the implementation of maintenance work. The image acquisition system 10 will be described with reference to FIG. 1 .

[0012] (Image Acquisition System) FIG. 1 is a diagram of an image acquisition system according to this embodiment. As shown in FIG. 1, the image acquisition system 10 includes a control unit 31, a storage unit 32, a display unit 33, an input unit 34, and an image capture unit 35. The image acquisition system 10 is a wearable terminal worn by a worker. A worker terminal 39 is also connected to the image acquisition system 10 via a communication network 37. Although not shown, the worker terminal 39 is, for example, a stationary terminal, and like the image acquisition system 10, includes a display unit and an input unit, allowing the worker terminal 39 to input information for recording work.

[0013] The control unit 31 includes an integrated circuit such as a CPU (Central Processing Unit). The control unit 31 executes data processing for recording images of maintenance work. The storage unit 32 is any storage device such as a semiconductor storage device or a magnetic storage device. The storage unit 32 stores images captured by the imaging unit 35, information generated by various processes, etc. The display unit 33 is a display device such as a transmissive liquid crystal display. The input unit 34 is an input device such as a keyboard and a mouse. The imaging unit 35 is a camera, for example, that captures an image of an object to be photographed and acquires the captured image.

[0014] The image acquisition system 10 executes a process of requesting the maintenance worker to acquire a photographed image in order to record an image of the maintenance work location, and executes a display process for the photographed image.

[0015] 2 and 3, the photographed image 50 displayed on the display unit 33 will be described. When acquiring the photographed image 50, the control unit 31 displays the photographed image 50 photographed by the photographing unit 35 on the display unit 33, and also displays a first photographing marker 51 and a second photographing marker 52.

[0016] Here, a spatial reference point 53 in three-dimensional space is provided at the work location that is the object to be photographed. The spatial reference point 53 is, for example, a QR code (registered trademark) attached to the object to be photographed. Note that the spatial reference point 53 may be, for example, a feature point of the object to be photographed, and may be recognizable by any method.

[0017] The first photographing marker 51 is a three-dimensional model object displayed on the display unit 33. The first photographing marker 51 is a marker whose positional relationship with the photographing target is fixed with reference to a spatial reference point 53. Four first photographing markers 51 are provided, and are arranged in a square grid. The first photographing marker 51 is displayed in the form of a wire frame and has a spherical shape. The wire frame is a wire that runs along three-dimensional directions, namely, the vertical direction, the left-right direction, and the depth direction, with the photographing target facing forward. Therefore, by visually checking the first photographing marker 51, the operator can easily grasp the attitude angle of the photographing target in the roll direction, pitch direction, and yaw direction.

[0018] Like the first photographing marker 51, the second photographing marker 52 is a three-dimensional model object displayed on the display unit 33. The second photographing marker 52 is a marker whose positional relationship with the photographing unit 35 is fixed with reference to a spatial reference point 53. The positional relationship between the spatial reference point 53 and the photographing unit 35 is acquired by position detection. In addition, in this embodiment, the positional relationship between the second photographing marker 52 and the photographing unit 35 is fixed with reference to the spatial reference point 53, but the positional relationship between the second photographing marker 52 and the photographing unit 35 may also be fixed with reference to the photographing unit 35. Like the first photographing marker 51, four second photographing markers 52 are provided, arranged in a square grid, and have a spherical shape displayed in a wireframe format.

[0019] Here, the first photographing marker 51 and the second photographing marker 52 have a larger display range and a smaller display range than each other. For example, the size of the display range of the first photographing marker 51 is smaller than that of the second photographing marker 52. In addition, the first photographing marker 51 and the second photographing marker 52 have different colors. As shown in FIG. 3, when the first photographing marker 51 and the second photographing marker 52 are superimposed on each other, their colors change to the same color.

[0020] When the control unit 31 detects that the first photographing marker 51 and the second photographing marker 52 are superimposed, the control unit 31 records the photographed image 50 of the photographing subject. At this time, when the first photographing marker 51 and the second photographing marker 52 are superimposed, the control unit 31 displays the time remaining until recording of the photographed image 50 of the photographing subject is completed. Specifically, as shown in FIG. 3 , the control unit 31 displays a progress bar 54 on the display unit 33. The progress bar 54 represents the time remaining until recording is completed (photography is completed) as a progress rate of the photographing task. Note that the display format of the time remaining until recording is completed is not limited to the progress bar 54, and any display format may be applied.

[0021] Furthermore, the first photographing marker 51 includes position information indicating a first rotation position in the photographed image 50. Since four first photographing markers 51 are arranged, for example, position information of 0° is assigned to the first photographing marker 51 located in the upper right corner shown in Fig. 2. Similarly, position information of 90° is assigned to the first photographing marker 51 located in the lower right corner, position information of 180° is assigned to the first photographing marker 51 located in the lower left corner, and position information of 270° is assigned to the first photographing marker 51 located in the upper left corner.

[0022] Like the first photographing marker 51, the second photographing marker 52 includes position information indicating a second rotation position in the photographed image 50. Since four second photographing markers 52 are arranged, for example, position information of 0° is assigned to the first photographing marker 51 located in the upper right corner shown in Fig. 2. Similarly, position information of 90° is assigned to the first photographing marker 51 located in the lower right corner, position information of 180° is assigned to the first photographing marker 51 located in the lower left corner, and position information of 270° is assigned to the first photographing marker 51 located in the upper left corner.

[0023] If the first rotation position and the second rotation position are different after the control unit 31 has completed recording the captured image 50 of the object to be photographed, the control unit 31 rotates the captured image 50 so that the first rotation position and the second rotation position coincide, and records the rotated captured image 50.

[0024] (Image Acquisition Method) Next, an image acquisition method according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart illustrating the image acquisition method according to this embodiment. When recording an image of a maintenance work location as a photographed object, the control unit 31 of the image acquisition system 10 first recognizes the spatial reference point 53 of the photographed object photographed by the photographing unit 35 (step S1).

[0025] After executing step S1, the control unit 31 calculates the shooting location of the object to be shot in the space of the three-dimensional model based on the recognized spatial reference point (step S2). That is, in step S2, the control unit 31 executes superimposition of the real space and the virtual space based on the spatial reference point.

[0026] After executing step S2, the control unit 31 superimposes and displays the first photography marker 51 on the photographed image 50 photographed by the photographing unit 35 and displayed on the display unit 33 based on the calculated photographing location (step S3). Note that, after executing step S3, the control unit 31 may execute a notification to the effect that recording of the photographed image 50 of the photographing object will be executed.

[0027] After executing step S3, the control unit 31 displays the second photographing marker 52 fixed to the photographing unit 35 superimposed on the photographed image 50 (step S4). Thereafter, the control unit 31 detects that the first photographing marker 51 and the second photographing marker 52 are superimposed on each other, and executes recording of the photographed image 50 (step S5). In step S5, the control unit 31 detects the superimposition of the first photographing marker 51 and the second photographing marker 52 based on the distance between the center position of the first photographing marker 51 and the center position of the second photographing marker 52. Specifically, the control unit 31 determines that the first photographing marker 51 and the second photographing marker 52 are superimposed on each other if the distance is shorter than a preset distance, and determines that the first photographing marker 51 and the second photographing marker 52 are not superimposed on each other if the distance is equal to or greater than the preset distance. In addition, in step S5, when the first shooting marker 51 and the second shooting marker 52 overlap, the control unit 31 displays the progress bar 54 shown in Figure 3 to display the time until recording of the captured image 50 is completed.

[0028] After the recording of the photographed image 50 of the photographing object is completed, the control unit 31 may output a photographing completion sound from a speaker (not shown).

[0029] After executing step S6, the control unit 31 judges the acceptability of the work area by image judgment based on the acquired photographed image 50 (step S6), and ends the process related to the image acquisition method.

[0030] As described above, the image acquisition system 10 and the image acquisition method according to the present embodiment can be understood, for example, as follows.

[0031] The image acquisition system 10 of the first aspect comprises a photographing unit 35 that photographs an object to be photographed and acquires a photographed image, a display unit 33 that displays the photographed image acquired by the photographing unit 35, and a control unit 31 that controls the photographing unit 35 and the display unit 33. The control unit 31 executes the following steps: a step S1 of recognizing a spatial reference point 53 associated with the object to be photographed; a step S3 of displaying a first photographing marker 51, whose positional relationship with the object to be photographed is fixed, on the display unit 33 based on the recognized spatial reference point 53; a step S4 of displaying a second photographing marker 52, whose positional relationship with the photographing unit 35 is fixed, on the display unit 33; and a step S5 of recording the photographed image 50 of the object to be photographed when it detects that the first photographing marker 51 and the second photographing marker 52 overlap.

[0032] According to this configuration, by recording the photographed image 50 with the first photographing marker 51 and the second photographing marker 52 superimposed, it is possible to photograph the photographed object in a state where the positional relationship between the photographing unit 35 and the photographed object is suitable for image determination. Therefore, it is possible to photograph the photographed object by appropriately including it in the photographed image 50.

[0033] As a second aspect, in the image acquisition system 10 according to the first aspect, four of the first photographing markers 51 and four of the second photographing markers 52 are arranged in a square grid pattern.

[0034] According to this configuration, the first photographing marker 51 and the second photographing marker 52 can be aligned appropriately.

[0035] As a third aspect, in the image acquisition system 10 relating to the first or second aspect, each of the first photographing marker 51 and the second photographing marker 52 has a display form of a wire frame and a spherical shape.

[0036] With this configuration, even if the first photographing marker 51 and the second photographing marker 52 overlap, appropriate alignment can be performed while ensuring the visibility of the first photographing marker 51 and the second photographing marker 52.

[0037] As a fourth aspect, in the image acquisition system 10 relating to any one of the first to third aspects, the first photographing marker 51 and the second photographing marker 52 have a larger display range and the other has a smaller display range.

[0038] This configuration makes it easier to align the positions of the first photographing marker 51 and the second photographing marker 52.

[0039] As a fifth aspect, in an image acquisition system 10 relating to any one of the first to fourth aspects, the first photographing marker 51 and the second photographing marker 52 are different colors, and when the first photographing marker 51 and the second photographing marker 52 overlap, the control unit 31 changes the colors of the first photographing marker 51 and the second photographing marker 52 so that they become the same color.

[0040] According to this configuration, it is possible to easily grasp that the first photographing marker 51 and the second photographing marker 52 overlap each other.

[0041] As a sixth aspect, in the image acquisition system 10 relating to any one of the first to fifth aspects, when the first shooting marker 51 and the second shooting marker 52 overlap, the control unit 31 displays the time until recording of the shooting image 50 of the object to be photographed is completed.

[0042] According to this configuration, the operator can be prompted to keep the position of the photographing unit 35 relative to the photographing object in a stationary state until recording is completed.

[0043] In a seventh aspect, the image acquisition system 10 according to any one of the first to sixth aspects further includes a speaker, and the control unit 31 outputs a shooting completion sound from the speaker after recording of the captured image of the object to be photographed is completed.

[0044] According to this configuration, the operator can be prompted to keep the position of the imaging unit 35 relative to the object to be imaged stationary until recording is complete.

[0045] As an eighth aspect, in an image acquisition system 10 relating to any one of the first to seventh aspects, the control unit 31 detects the center position of the first photographing marker 51 and the center position of the second photographing marker 52, and detects the overlap of the first photographing marker 51 and the second photographing marker 52 based on the distance between the center positions.

[0046] According to this configuration, the overlapping of the first photographing marker 51 and the second photographing marker 52 can be easily detected.

[0047] As a ninth aspect, in the image acquisition system 10 relating to any one of the first to eighth aspects, the first photographing marker 51 includes position information indicating a first rotation position of the photographed image 50, and the second photographing marker 52 includes position information indicating a second rotation position of the photographed image 50, and if the first rotation position and the second rotation position are different after recording of the photographed image 50 of the object to be photographed is completed, the control unit 31 rotates the photographed image 50 so that the first rotation position and the second rotation position coincide.

[0048] According to this configuration, even if the photographed image 50 is recorded in a rotated state, it can be recorded in a state where the rotated state is corrected.

[0049] The image acquisition method according to the tenth aspect is an image acquisition method executed by an image acquisition system 10 that photographs an object to be photographed and acquires a photographed image, in which the image acquisition system 10 executes the following steps: step S1: recognizing a spatial reference point 53 associated with the object to be photographed; step S3: displaying a first photographing marker 51, which has a fixed positional relationship with the object to be photographed, on a display unit 33 that displays the photographed image 50 based on the recognized spatial reference point 53; step S4: displaying a second photographing marker 52, which has a fixed positional relationship with a photographing unit 35 that photographs the photographed image 50, on the display unit 33; and step S5: recording the photographed image 50 of the object to be photographed when it is detected that the first photographing marker 51 and the second photographing marker 52 overlap.

[0050] According to this configuration, by recording the photographed image 50 with the first photographing marker 51 and the second photographing marker 52 superimposed, it is possible to photograph the photographed object in a state where the positional relationship between the photographing unit 35 and the photographed object is suitable for image determination. Therefore, it is possible to photograph the photographed object by appropriately including it in the photographed image 50.

[0051] 10 Image acquisition system 31 Control unit 32 Storage unit 33 Display unit 34 Input unit 35 Photography unit 37 Communication network 39 Worker terminal 50 Photographed image 51 First photography marker 52 Second photography marker 53 Spatial reference point

Claims

1. An image acquisition system comprising: an imaging unit which photographs an object to be photographed and acquires an image; a display unit which displays the image acquired by the imaging unit; and a control unit which controls the imaging unit and the display unit, wherein the control unit executes the steps of: recognizing a spatial reference point associated with the object to be photographed; displaying on the display unit a first imaging marker whose positional relationship with the object to be photographed is fixed based on the recognized spatial reference point; displaying on the display unit a second imaging marker whose positional relationship with the imaging unit is fixed; and recording the image of the object to be photographed when it is detected that the first imaging marker and the second imaging marker overlap.

2. The image acquisition system according to claim 1, wherein the first photographing markers and the second photographing markers are each arranged in a square grid of four.

3. The image acquisition system according to claim 1, wherein each of the first and second photographing markers is displayed in a wire frame and has a spherical shape.

4. The image acquisition system according to claim 1, wherein the first shooting marker and the second shooting marker have a larger display range and the other has a smaller display range.

5. The image acquisition system of claim 1, wherein the first photographing marker and the second photographing marker are different colors, and the control unit changes the colors of the first photographing marker and the second photographing marker to the same color when the first photographing marker and the second photographing marker overlap.

6. The image acquisition system of claim 1, wherein the control unit displays the time remaining until recording of the captured image of the object is completed when the first shooting marker and the second shooting marker overlap.

7. An image acquisition system as described in claim 1, further comprising a speaker, wherein the control unit causes the speaker to output a shooting completion sound after recording of the captured image of the subject is completed.

8. The image acquisition system of claim 1, wherein the control unit detects the center position of the first shooting marker and the center position of the second shooting marker, and detects overlap between the first shooting marker and the second shooting marker based on the distance between the center positions.

9. The image acquisition system of claim 1, wherein the first shooting marker includes position information indicating a first rotational position of the captured image, the second shooting marker includes position information indicating a second rotational position of the captured image, and the control unit rotates the captured image so that the first rotational position and the second rotational position coincide with each other if the first rotational position and the second rotational position are different after recording of the captured image of the subject has been completed.

10. An image acquisition method executed by an image acquisition system that photographs an object to be photographed and acquires a photographed image, wherein the image acquisition system executes the following steps: recognizing a spatial reference point associated with the object to be photographed; displaying a first photographing marker, whose positional relationship with the object to be photographed is fixed based on the recognized spatial reference point, on a display unit that displays the photographed image; displaying a second photographing marker, whose positional relationship with a photographing unit that photographs the image, on the display unit; and recording the photographed image of the object to be photographed when it detects that the first photographing marker and the second photographing marker overlap.

Images