Imaging support device, imaging support method, and program

Abstract

To provide an imaging support device, an imaging support method, and a program that can prevent a user from forgetting executing an investigation of a transformed portion.SOLUTION: An imaging support device comprises: transformed portion specification means that specifies a predetermined transformed portion in a predetermined picked-up image; related investigation details specification means that specifies the details of a related investigation according to the attribute of the transformed portion; and related investigation details output means that outputs the details of the related investigation to a user of an imaging apparatus. The related investigation details output means outputs the details of the related investigation superimposed on the picked-up image displayed by the imaging apparatus.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an imaging support device, an imaging support method, and a program. [Background technology]

[0002] Conventionally, there is known a foreign matter inspection device that detects a portion where brightness changes from image data obtained by an imaging device and identifies a foreign matter in a molten film from the detection data of the portion where brightness changes (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2000-168750 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, Patent Document 1 does not propose a mechanism for outputting related investigation contents, which are investigation contents related to the attributes of the mutation site, to a user who inspects foreign matter. Therefore, there is a risk that the user may forget to conduct an investigation related to the mutation site.

[0005] The present invention has been made in consideration of the above-mentioned situation, and aims to provide an imaging support device, an imaging support method, and a program that can prevent a user from forgetting to conduct an investigation related to a mutation site. [Means for solving the problem]

[0006] In order to achieve the above object, one aspect of the present invention is A mutation site identifying means for identifying a predetermined mutation site in a predetermined captured image; A related investigation content specification means for specifying related investigation content according to the attribute of the mutation site; a related investigation content output means for outputting the related investigation content to a user of the imaging device; The imaging support device is provided with:

[0007] Another aspect of the present invention is A mutation site identification step of identifying a predetermined mutation site in a predetermined captured image; A related investigation content specification step of specifying related investigation content according to the attribute of the mutation site; a related investigation content output step of outputting the related investigation content to a user of the imaging device; The imaging assistance method includes the steps of:

[0008] In addition, one aspect of the present invention is a method for providing a computer with: A mutation site identification step of identifying a predetermined mutation site in a predetermined captured image; A related investigation content specification step of specifying related investigation content according to the attribute of the mutation site; a related investigation content output step of outputting the related investigation content to a user of the imaging device; This is a program for executing the above. Effect of the Invention

[0009] According to the present invention, since related investigation contents according to the attributes of the mutation site can be output to the user, it is possible to prevent the user from forgetting to carry out an investigation related to the mutation site. [Brief description of the drawings]

[0010] [Figure 1] 1 is a functional block diagram showing an example of a functional configuration of an imaging support device to which an information processing device according to an embodiment of the present invention is applied. [Diagram 2] 1 is a diagram showing an outline of a system configuration of an imaging support system according to an embodiment of the present invention. [Diagram 3] 1 is a block diagram showing a hardware configuration of an imaging support device according to an embodiment of the present invention. [Figure 4] 3 is a block diagram showing an example of the configuration of the imaging device shown in FIG. 2. [Diagram 5] FIG. 2 is an external view showing an example of a retaining wall. [Figure 6] 6 is a diagram showing a schematic cross section of a retaining wall cut in the y direction along line AA shown in FIG. 5. FIG. [Figure 7] FIG. 4 is a sequence diagram showing an example of an operation procedure of the imaging support system according to the present embodiment. [Figure 8] 5 is a diagram showing an example of an image displayed on a display of the imaging device shown in FIG. 4. [Figure 9] 5 is a flowchart showing an example of an operation procedure of the imaging support device according to the present embodiment. [Figure 10] 13 is a flowchart showing an example of an operation procedure of the imaging support device according to Modification 1 of the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] (Embodiment) Hereinafter, the present embodiment will be described with reference to a number of drawings. For the sake of convenience, some of the drawings show three axes (x-axis, y-axis, and z-axis) that define directions. Fig. 1 is a functional block diagram showing an example of the functional configuration of an imaging support device 1 to which an information processing device according to the present embodiment is applied.

[0012] The imaging support device 1 of this embodiment outputs to the user related investigation contents corresponding to the attributes of a predetermined mutation site in order to support the user's investigation of various diagnosis targets. The imaging support device 1 of this embodiment is not limited to being used to support foreign body inspection in manufacturing plants. The imaging support device 1 of this embodiment can be applied to devices that support the diagnosis of structures such as retaining walls, tunnels, and bridges. A retaining wall is a wall-like structure that is constructed to prevent the collapse of a slope against the lateral pressure of the soil. In this embodiment, a case in which the imaging support device 1 supports the diagnosis of a retaining wall will be described as an example.

[0013] <System configuration> 2 is a diagram showing an outline of the system configuration of an imaging support system S according to this embodiment. The imaging support system S according to this embodiment has an imaging support device 1 that supports a user in diagnosing a retaining wall, and an imaging device 2 that the user carries when inspecting the retaining wall. The user is, for example, a retaining wall diagnostician.

[0014] The imaging support device 1 is, for example, an information processing device such as a server or a PC (Personal Computer). The imaging device 2 is, for example, a portable information processing device such as a smartphone or tablet with an imaging function, or a digital camera with a communication function. The imaging support device 1 and the imaging device 2 are connected to each other so as to be able to communicate with each other via a network 100 such as the Internet. The imaging support device 1 executes various processes in cooperation with the operation of the imaging device 2. Although not shown in FIG. 2, an auxiliary device for investigation may be connected to the imaging device 2 so as to be able to communicate with the imaging device 2. The auxiliary device is, for example, a microscope.

[0015] <Hardware configuration> 3 is a block diagram showing a hardware configuration of the imaging support device 1 according to this embodiment. The imaging support device 1 is an information processing device having a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an output unit 17, an input unit 16, a storage unit 18, a communication unit 19, and a drive 20.

[0016] The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 18 to the RAM 13. The RAM 13 also stores data and the like required for the CPU 11 to execute various processes. The CPU 11, the ROM 12, and the RAM 13 are interconnected via a bus 14. An input / output interface 15 is also connected to this bus 14.

[0017] The input / output interface 15 is connected to an output unit 17, an input unit 16, a storage unit 18, a communication unit 19, and a drive 20. The output unit 17 is composed of a display, a speaker, or the like, and outputs various information as images or sounds. The input unit 16 is composed of a keyboard, a mouse, or the like, and inputs various information. The storage unit 18 is composed of a hard disk drive (HDD) or a solid state drive (SSD), or the like, and stores various data. The communication unit 19 transmits and receives information to and from the imaging device 2 via the network 100 in accordance with a predetermined communication protocol. The communication protocol is, for example, IP (Internet Protocol).

[0018] Removable media 21, which may be a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is appropriately attached to the drive 20. Programs read from the removable media 21 by the drive 20 are installed in the storage unit 18 as necessary. The removable media 21 can also store various data stored in the storage unit 18 in the same manner as the storage unit 18.

[0019] <Imaging device 2> Next, the configuration of the imaging device 2 will be described. In this embodiment, the imaging device 2 will be described as a smartphone. FIG. 4 is a block diagram showing an example of the configuration of the imaging device 2 shown in FIG. 2. The hardware configuration of the imaging device 2 is the same as the hardware configuration shown in FIG. 3. In FIG. 4, the CPU 11, the ROM 12, the RAM 13, the bus 14, and the input / output interface 15 of the configuration shown in FIG. 3 are omitted from the figure. The imaging device 2 has a communication unit 41, a control unit 42, a storage unit 43, an output unit 44, and an input unit 45. The control unit 42 functions as a result of the CPU 11 shown in FIG. 3 executing a program. The output unit 44 has a display 61 that displays an image and a speaker 62 that outputs sound. The input unit 45 has a touch panel 63 for a user to input instructions to the imaging device 2, a camera 64 that photographs the retaining wall 51, and a microphone 65 that collects the user's sound. The touch panel 63 is provided on the surface on which the display 61 displays an image.

[0020] <Outline of the retaining wall surveyed> Before describing in detail the function of the imaging support device 1 according to this embodiment, a retaining wall will be described. Fig. 5 is an external view showing an example of a retaining wall. Fig. 6 is a schematic diagram showing a cross section of a retaining wall 51 cut in the y direction along a line segment AA shown in Fig. 5.

[0021] A retaining wall 51 shown in FIG. 5 is provided with a plurality of drainage holes 52-1 to 52-m (m is an integer of 1 or more) for draining rainwater and groundwater from soil 57. FIG. 5 shows an example where m=6. As many years pass since the construction of the retaining wall 51, various variations occur in the retaining wall 51. Examples of the variations include cracks 54, efflorescence 55, and skirt 53. Cracks 54 are fissures. Efflorescence 55 is a phenomenon in which calcium and sodium components dissolve in water from the concrete of the retaining wall 51 and react with carbon dioxide in the air to turn into white powder. Skirt 53 is a phenomenon in which a part of the retaining wall 51 swells.

[0022] As shown in FIG. 6, a mesh-like backfill material 58 filled with crushed stones is provided between the concrete portion of the retaining wall 51 and the soil 57. A suction prevention material 56 is provided between the concrete portion of the retaining wall 51 and the backfill material 58. A cylindrical pipe is attached to each of the drainage holes 52-1 to 52-6. Rainwater that has soaked into the soil 57 is discharged from the drainage holes 52-1 to 52-6 via the backfill material 58 and the pipe. The suction prevention material 56 serves to prevent the earth and sand of the soil 57 and the crushed stones of the backfill material 58 from being discharged together with the water. In the following, the symbol of any of the drainage holes 52-1 to 52-m is denoted as 52-k (k is any integer from 1 to m).

[0023] <Functional configuration> The functional configuration of the imaging support device 1 according to this embodiment will be described with reference to Fig. 1. In the CPU 11 of the imaging support device 1, a mutation site identification unit 31, a related investigation content identification unit 32, a related investigation content output unit 33, an investigation completion detection unit 34, a task completion restriction unit 35, a related investigation content association unit 36, and an investigation completion notification unit 37 function during operation.

[0024] The memory unit 18 stores past captured image information and related investigation content information. The past captured image information is information on captured images recorded in chronological order for the diagnosis target. When the diagnosis target is a retaining wall 51, the memory unit 18 stores one or more captured images of the retaining wall 51 captured in the past. The related investigation content information is information on related investigation content according to the attributes of the mutated portion. The mutated portion is, for example, skirt steak 53, efflorescence 55, cracks 54, etc.

[0025] When the mutated site is skirt steak 53, the relevant inspection contents are the position, height, size, etc. of skirt steak 53. When the mutated site is efflorescence 55, the relevant inspection contents are the presence or absence of cracks along the water path, the width of the efflorescence (white powdery part), and the thickness of the efflorescence. When the mutated site is efflorescence 55, the relevant inspection contents may optionally include the height of the water level. When the mutated site is crack 54, the relevant inspection contents are the length of crack 54, the width of each of the upper and lower ends of crack 54, the presence or absence of a step of crack 54 (difference in height between the left and right sides), and the depth of crack 54. The following mainly describes the case where the mutated site is skirt steak 53.

[0026] The mutation site identifying unit 31 performs machine learning in advance to identify the mutation site from many captured images serving as samples based on multiple teacher data for each of a plurality of types of mutation sites. When the mutation site identifying unit 31 receives a predetermined captured image from the imaging device 2, it identifies the predetermined mutation site in the predetermined captured image. The captured image is, for example, an image of a retaining wall 51. The captured image may be a still image or a video image.

[0027] When the mutated part is skirt steak 53, the mutated part identifying unit 31 can determine whether it is skirt steak 53 or not by determining whether there is a height difference in an image captured from the side of skirt steak 53. When the mutated part is efflorescence 55, the mutated part identifying unit 31 can determine whether efflorescence 55 is present or not from the difference in contrast between the part with efflorescence 55 and the part without efflorescence 55 in the color of the surface of the retaining wall 51. The principle of determination will be briefly explained. The color of most of the retaining wall 51 becomes yellowish when exposed to sunlight, wind and rain for a long period of time, but the color of the efflorescence 55 is pure white like paper. Therefore, the mutated part identifying unit 31 can determine that the part with a lighter color than the overall color of the retaining wall 51 is efflorescence 55. Furthermore, the efflorescence 55 is formed white on the surface of the retaining wall 51 in the shape of a trace like that of flowing water. Since the contrast is slightly different between the inside and outside of the boundary between the part with efflorescence 55 and its surroundings, the mutated part identifying unit 31 identifies the efflorescence 55 in the retaining wall 51.

[0028] Furthermore, as a method of identifying the mutation site, the mutation site identifying unit 31 may identify the mutation site by using past captured image information stored in the storage unit 18. A specific example will be described. Upon receiving an image captured by the imaging device 2, the mutation site identifying unit 31 reads out from the storage unit 18 the most recent captured image that has the same angle of view as the received captured image and that has been recorded in the past. The mutation site identifying unit 31 compares the received captured image with the captured image read out from the storage unit 18 to identify the mutation site.

[0029] The related investigation content identifying unit 32 identifies the related investigation content according to the attributes of the mutated area. When the mutated area is the skirt steak 53, the related investigation content identifying unit 32 refers to the related investigation content information stored in the memory unit 18, and identifies the position, height, and size of the skirt steak 53 as the related investigation content. Height H1 shown in FIG. 6 corresponds to the height of the skirt steak 53. Height H1 corresponds to the maximum value of the bulge of the skirt steak 53 in the direction perpendicular to the slope of the retaining wall 51. In FIG. 6, the hatching pattern of the bulging portion of the skirt steak 53 has been omitted to make height H1 easier to see.

[0030] The related investigation content output unit 33 outputs the related investigation content to the user of the imaging device 2. For example, the related investigation content output unit 33 causes the display 61 of the imaging device 2 to display the related investigation content as a symbol. When the image of the retaining wall 51 shown in FIG. 5 is displayed on the display 61, the related investigation content output unit 33 causes the characters "position, height and size" to be displayed at the position of the skirt steak 53. In this manner, the related investigation content output unit 33 may output the related investigation content by superimposing it on the captured image displayed by the imaging device 2.

[0031] The related investigation content associating unit 36 ​​stores the related investigation content in association with a predetermined mutation site. This function will be described with reference to the retaining wall 51 shown in FIG. 5, taking as an example a case where the predetermined imaging site is the skirt steak 53. Before starting the investigation of the retaining wall 51, the user attaches stickers with numbers "1" to "3" in order near the skirt steak 53, the cracks 54, and the efflorescence 55. Then, when the user operates the imaging device 2 to capture an image of the entire retaining wall 51, the control unit 42 transmits the captured image of the entire retaining wall 51 to the imaging support device 1. The related investigation content associating unit 36 ​​recognizes the skirt steak 53, the cracks 54, and the efflorescence 55 identified by the mutation site identifying unit 31 from the captured image, and the numbers "1" to "3" written on the stickers. Next, when the user brings the camera 64 of the imaging device 2 close to the skirt steak 53, the control unit 42 transmits the sticker with the number "1" and the first mutation site image including the skirt steak 53 to the imaging support device 1. Thereafter, when the user operates the imaging device 2 to input the position, height, and size of the skirt steak 53, the control unit 42 transmits first related investigation result information including the investigation results of the related investigation contents of the skirt steak 53 to the imaging support device 1. When the related investigation contents associating unit 36 ​​receives the first mutation site image and the first related investigation result information from the imaging device 2, it associates the image of the skirt steak 53 with the investigation results of the related investigation contents in correspondence with the number "1" written on the sticker, and stores them in the storage unit 18. In this way, the related investigation contents associating unit 36 ​​associates each of the images of the skirt steak 53, the cracks 54, and the efflorescence phenomenon 55 with the investigation results of the related investigation contents in correspondence with the numbers "1" to "3", and stores them in the storage unit 18.

[0032] The investigation completion detection unit 34 detects the investigation completion for the related investigation content. When the investigation completion detection unit 34 detects the investigation completion for the related investigation content, the task completion restriction unit 35 permits the user to complete the task in response to the detection of the investigation completion. For example, the task completion restriction unit 35 outputs information to the user of the imaging device 2 that the task has been completed.

[0033] When the investigation completion detection unit 34 detects that the investigation regarding the related investigation content has been completed, the investigation completion notification unit 37 outputs a notification to that effect to the user. For example, the investigation completion notification unit 37 causes the display 61 of the imaging device 2 of the user to display a “Completed” mark.

[0034] <Processing content> Next, the operation of the imaging support device 1 according to this embodiment will be described. Fig. 7 is a sequence diagram showing an example of the operation procedure of the imaging support system S according to this embodiment.

[0035] In step S1, when the user operates the imaging device 2 to capture an image of the entire retaining wall 51, the control unit 42 of the imaging device 2 transmits the captured image of the retaining wall 51 to the imaging support device 1. In step S2, when the mutation site identifying unit 31 receives the captured image of the retaining wall 51 from the imaging device 2, it analyzes the captured image and identifies the mutation site. Here, a case will be described in which the mutation site identifying unit 31 identifies skirt steak 53 as the mutation site from the captured image of the retaining wall 51.

[0036] In step S3, the related investigation content specifying unit 32 specifies related investigation content according to the attribute of the mutation site specified by the mutation site specifying unit 31. Specifically, the related investigation content specifying unit 32 refers to the related investigation content information stored by the storage unit 18, and specifies the position, height, and size as the related investigation content of the skirt steak 53. In step S4, the related investigation content output unit 33 outputs the related investigation content to the user of the imaging device 2. Specifically, the related investigation content output unit 33 causes the display 61 of the imaging device 2 to display information on the position, height, and size of the skirt steak 53 as the related investigation content for the user. In this way, the related investigation content (position, height, size, etc.) related to the specified mutation site (skirt steak) can be output to the user of the imaging device 2. Therefore, it is possible to prevent the user from forgetting to carry out the related investigation content of the skirt steak 53.

[0037] Fig. 8 is a diagram showing an example of an image displayed on the display 61 of the imaging device 2 shown in Fig. 4. Fig. 8 shows an image in which a user, before inspecting the retaining wall 51, attaches stickers with numbers "1" to "3" in order near the skirt 53, crack 54, and efflorescence 55 of the retaining wall 51, and then photographs the entire retaining wall 51, and then enlarges the area including the skirt 53.

[0038] 8, the related investigation content output unit 33 displays information on the related investigation content of the skirt steak 53 by superimposing it on the captured image of the retaining wall 51 displayed on the display 61 of the imaging device 2. In this way, the related investigation content output unit 33 may output the related investigation content by superimposing it on the captured image displayed by the imaging device 2. In this case, the related investigation content is output on the captured image in which the mutation site is displayed, so that the user can be more accurately prevented from forgetting to conduct the investigation.

[0039] 7, the combination of the mutated site and the related investigation contents is the skirt steak 53 and the related investigation contents according to the attributes of the skirt steak 53 (position, height, size, etc.), but the combination of the mutated site and the related investigation contents is not limited to the combination of the skirt steak 53 and the related investigation contents according to the attributes of the skirt steak 53. When the mutated site is a crack 54, the related investigation contents are the length of the crack 54, the width of each of the upper and lower ends of the crack 54, the presence or absence of a step (height difference between the left and right) of the crack 54, and the depth of the crack 54, etc. When the mutated site is efflorescence 55, the related investigation contents are the presence or absence of cracks along the water path of the efflorescence 55, the width of the efflorescence portion, the thickness of the efflorescence portion, etc.

[0040] 7, the device to which the related investigation contents are output is the imaging device 2, but the device to which the related investigation contents are output is not limited to the imaging device 2. The device to which the related investigation contents are output may be a terminal carried by the user and may be another information processing terminal communicatively connected to the imaging device 2. The other information processing terminal is, for example, a wearable terminal such as smart glasses or a smart watch. Furthermore, the output of the related investigation contents to the user is not limited to a display by an image, but may be an audio output via the speaker 62.

[0041] Next, an operation of the imaging support device 1 for associating a mutation site with related investigation details will be described. Fig. 9 is a flowchart showing an example of an operation procedure of the imaging support device 1 according to this embodiment. Steps S11 to S13 are similar to steps S2 to S4 described with reference to Fig. 7, and therefore detailed description of these steps will be omitted.

[0042] A case will be described below as an example in which a user attaches stickers with numbers "1" to "3" in order near the skirt 53, cracks 54, and efflorescence 55 before starting an investigation of the retaining wall 51. When the user operates the imaging device 2 to capture an image of the entire retaining wall 51, the control unit 42 transmits the captured image of the entire retaining wall 51 to the imaging support device 1. In step S11, when the mutation site identification unit 31 receives the captured image of the retaining wall 51 from the imaging device 2, it analyzes the captured image and identifies the skirt 53, cracks 54, and efflorescence 55. The related investigation content associating unit 36 ​​recognizes the skirt 53, cracks 54, and efflorescence 55 identified by the mutation site identification unit 31 from the captured image, and the numbers "1" to "3" written on the stickers.

[0043] In step S12, the related investigation content specification unit 32 refers to the related investigation content information stored by the storage unit 18 and specifies the related investigation content related to each of the skirt steak 53, the crack 54, and the efflorescence phenomenon 55. When the user approaches the skirt steak 53 while the camera 64 of the imaging device 2 is taking a video, the control unit 42 transmits a sticker with the number "1" and a first mutation site image including the skirt steak 53 to the imaging support device 1. The related investigation content association unit 36 ​​receives the first mutation site image from the imaging device 2. In step S13, the related investigation content output unit 33 displays the related investigation content according to the attribute of the skirt steak 53 on the display 61 of the imaging device 2. For example, as shown in FIG. 8, the related investigation content output unit 33 displays the information of the related investigation content by superimposing it on the captured image of the retaining wall 51 displayed on the display 61 of the imaging device 2.

[0044] When the user operates the imaging device 2 to input the position, height, and size of the skirt steak 53, the control unit 42 transmits first related investigation result information including the investigation results of the related investigation content of the skirt steak 53 to the imaging support device 1. In step S14, the related investigation content associating unit 36 ​​receives the first related investigation result information including the investigation results of the related investigation content of the skirt steak 53 from the imaging device 2. In step S15, based on the received first mutation site image and first related investigation result, the related investigation content associating unit 36 ​​associates the image of the skirt steak 53 with the investigation results of the related investigation content in accordance with the number "1" written on the sticker, and stores them in the memory unit 18.

[0045] In step S16, the investigation completion detection unit 34 judges whether the investigation of all related investigation contents has been completed. As a result of the judgment in step S16, the investigation completion detection unit 34 has not acquired the investigation results of the related investigation contents corresponding to the crack 54 and the efflorescence phenomenon 55, so the process returns to step S11. When the user approaches the crack 54 in a state in which the camera 64 of the imaging device 2 is shooting a video, the control unit 42 transmits a second mutation site image including the sticker with the number "2" and the crack 54 to the imaging support device 1. In step S11, the mutation site identification unit 31 analyzes the captured image received from the imaging device 2 to identify the crack 54. In step S12, the related investigation content identification unit 32 refers to the related investigation content information stored by the storage unit 18 and identifies the related investigation content related to the crack 54. In step S13, the related investigation content output unit 33 causes the display 61 of the imaging device 2 to display information on the related investigation content of the crack 54.

[0046] When the user operates the imaging device 2 to input the length of the crack 54, the widths of the upper and lower ends, the presence or absence of a step (height difference between the left and right sides), and the depth, the control unit 42 transmits second related investigation result information including the investigation results of the related investigation content of the crack 54 to the imaging support device 1. In step S14, the related investigation content associating unit 36 ​​receives the second related investigation result information including the investigation results of the related investigation content of the crack 54 from the imaging device 2. In step S15, based on the received second mutation site image and the second related investigation result, the related investigation content associating unit 36 ​​associates the image of the crack 54 with the investigation results of the related investigation content in accordance with the number "2" written on the sticker and stores them in the storage unit 18.

[0047] Thereafter, for efflorescence 55 as well, the related investigation content associating unit 36 ​​associates the mutation site with the investigation results of the related investigation content and stores them in the memory unit 18. Thereafter, in the judgment of step S16, when the investigation completion detection unit 34 judges that the investigation of all related investigation content has been completed, the related investigation content associating unit 36 ​​ends the processing. In this manner, the related investigation content associating unit 36 ​​associates each of the images of skirt steak 53, cracks 54, and efflorescence 55 with the investigation results of the related investigation content in correspondence with the numbers "1" to "3", and stores them in the memory unit 18.

[0048] As described with reference to FIG. 9, the related investigation contents are automatically associated with the image of the specified mutation site, so that the user can organically associate the related investigation contents with the specified mutation site, and perform subsequent diagnosis more smoothly.

[0049] Also, a method for storing the mutation site in association with the related investigation content has been described with reference to FIG. 9, but the method is not limited to the method described with reference to FIG. 9. An example of another method will be described. The user taps a prompt (for example, the message shown in FIG. 8) displayed near the captured image of a predetermined mutation site displayed on the display 61 of the imaging device 2 to input a number that the user has assigned. Next, the user operates the imaging device 2 to input the investigation results of the related investigation content, and causes the imaging support device 1 to transmit the investigation results of the related investigation content with the input number attached. The related investigation content associating unit 36 ​​associates the image of the mutation site with the investigation results of the related investigation content using the input number as an identification marker, and stores them in the storage unit 18. In this way, the related investigation content associating unit 36 ​​may associate the predetermined mutation site with the related investigation content input by the user in the storage unit 18 using the number assigned by the user in response to the prompt displayed near the captured image of the predetermined mutation site as an identification marker. In this case, the user can save the trouble of attaching a sticker with a number written on it to the retaining wall 51.

[0050] <Advantageous Effects of the Present Embodiment> According to the above-described embodiment, since related investigation contents according to the attributes of the mutation site can be output to the user, it is possible to prevent the user from forgetting to carry out an investigation related to the mutation site.

[0051] Furthermore, according to this embodiment, the related investigation contents are automatically associated with an image of a specified mutation site, so that the user can organically associate the related investigation contents with the specified mutation site, allowing the user to perform subsequent diagnosis more smoothly.

[0052] Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the scope that can achieve the object of the present invention are included in the present invention.

[0053] (Variation 1) The present modified example 1 is an example for more accurately preventing the user from forgetting to carry out the survey. Fig. 10 is a flowchart showing an example of the operation procedure of the modified example 1 of the imaging support device 1 according to the present embodiment. Since steps S11 to S13 are similar to steps S2 to S4 described with reference to Fig. 7, and steps S14 to S16 are similar to steps S14 to S16 described with reference to Fig. 9, detailed explanations of these processes will be omitted.

[0054] As a result of the determination in step S16, the related investigation content associating unit 36 ​​determines that the investigation of all related investigation contents has been completed. In step S17, the task completion restricting unit 35 outputs information to the user that the task completion is permitted. For example, the task completion restricting unit 35 causes the display 61 of the imaging device 2 to display information that the task has been completed, or notifies the user by voice via the speaker 62. According to this modification example 1, the user cannot complete the task until the permission to complete the task is notified, so that the user can be more accurately prevented from forgetting to conduct the investigation.

[0055] (Variation 2) This modified example 2 is also an example that more accurately prevents the user from forgetting to conduct a survey. In this modified example 2, in step S17 shown in FIG. 10, when the survey completion detection unit 34 detects that the survey of the related survey content has been completed, the survey completion notification unit 37 outputs a notice to that effect to the user. For example, the survey completion notification unit 37 displays a "Done" mark on the display 61 of the user's imaging device 2. According to this modified example 2, the user is notified that the survey has been completed, so that the user can more reliably recognize that the survey has been completed. This makes it possible to more accurately prevent the user from forgetting to conduct the survey. Note that modified examples 1 and 2 may be combined.

[0056] (others) Also, for example, the above-mentioned series of processes can be executed by hardware or software. In other words, the above-mentioned functional configuration is merely an example and is not particularly limited. That is, it is sufficient that the information processing system is provided with a function capable of executing the above-mentioned series of processes as a whole, and the type of functional block used to realize this function is not particularly limited to the above-mentioned example. In addition, the location of the functional block is not particularly limited and may be arbitrary. For example, the functional block of the imaging support device 1 may be transferred to another device such as the imaging device 2. Conversely, the functional block of the other device may be transferred to the imaging support device 1, etc. In addition, one functional block may be configured as a single piece of hardware, may be configured as a single piece of software, or may be configured as a combination of them.

[0057] When a series of processes is executed by software, a program constituting the software is installed in a computer or the like from a network or a recording medium. The computer may be a computer built into dedicated hardware. The computer may also be a computer capable of executing various functions by installing various programs, such as a server, a general-purpose smartphone, or a personal computer.

[0058] The recording medium containing such a program may be configured not only as a removable medium (not shown) distributed separately from the device main body in order to provide the program to the user, etc., but also as a recording medium etc. provided to the user in a state in which the medium is pre-installed in the device main body. Since the program can be distributed via a network, the recording medium may be mounted on or accessible to a computer connected or connectable to the network.

[0059] In this specification, the steps describing the program recorded on the recording medium include not only processes performed in chronological order, but also processes that are not necessarily performed in chronological order but are performed in parallel or individually. In addition, in this specification, the term "system" refers to an overall device composed of multiple devices, multiple means, etc.

[0060] In other words, the information processing device to which the present invention is applied can take various forms having the following configurations. That is, A mutation site identifying means (e.g., a mutation site identifying unit 31) that identifies a predetermined mutation site in a predetermined captured image; A related investigation content specifying means (e.g., a related investigation content specifying unit 32) for specifying related investigation content according to the attribute of the mutation site; a related investigation content output unit (e.g., a related investigation content output unit 33) that outputs the related investigation content to a user of the imaging device; The imaging support device is provided with: This makes it possible to output to the user relevant investigation content according to the attributes of the mutation site, thereby preventing the user from forgetting to carry out an investigation related to the mutation site.

[0061] The related investigation content output means may output the related investigation content by superimposing it on a captured image displayed by the imaging device. This allows the related investigation content to be output to the captured image in which the mutation site is displayed, making it possible to more accurately prevent the user from forgetting to carry out the investigation.

[0062] Also, a survey completion detection means (e.g., a survey completion detection unit 34) for detecting the completion of a survey on the related survey content; The system may further include a task completion restriction means (for example, a task completion restriction unit 35) that permits the user to complete the task in response to detection of completion of the survey related to the related survey content. This makes it possible to more accurately prevent the user from forgetting to carry out the survey since the user cannot complete the task until permission to complete the task is notified.

[0063] The apparatus may further include a related investigation content associating means (for example, a related investigation content associating unit 36) that associates the related investigation content with the predetermined mutation site and stores the associated investigation content. This automatically associates the related investigation content with an image of the specified mutation site, allowing the user to organically associate the related investigation content with the specified mutation site, making subsequent diagnosis, etc., smoother.

[0064] Also, a survey completion detection means for detecting the completion of a survey regarding the related survey content; The system may further include a survey completion notification means (eg, survey completion notification unit 37) that outputs a notification to a user when the survey completion detection means detects that the survey regarding the related survey content has been completed. This notifies the user that the survey has been completed, so the user can more reliably recognize that the survey has been completed, and thus more reliably prevents the user from forgetting to complete the survey.

[0065] In addition, the information processing method to which the present invention is applied is A mutation site identification step of identifying a predetermined mutation site in a predetermined captured image; A related investigation content specification step of specifying related investigation content according to the attribute of the mutation site; a related investigation content output step of outputting the related investigation content to a user of the imaging device; The imaging assistance method includes the steps of:

[0066] In addition, the program to which the present invention is applied is installed on a computer. A mutation site identification step of identifying a predetermined mutation site in a predetermined captured image; A related investigation content specification step of specifying related investigation content according to the attribute of the mutation site; a related investigation content output step of outputting the related investigation content to a user of the imaging device; This is a program for executing the above. [Explanation of symbols]

[0067] 1: Imaging support device 2: Imaging device 11: CPU 18: Memory unit 19: Communication unit 31: Mutation site identification unit 32: Related investigation content identification unit 33: Related investigation content output unit 34: Investigation completion detection unit 35: Task completion restriction unit 36: Related investigation content association unit 37: Investigation completion notification unit

Claims

1. A mutation site identifying means for identifying a predetermined mutation site in a predetermined captured image; A related investigation content specification means for specifying related investigation content according to the attribute of the mutation site; a related investigation content output means for outputting the related investigation content to a user of the imaging device; An imaging support device comprising:

2. the related investigation content output means outputs the related investigation content by superimposing it on the captured image displayed by the imaging device. The imaging support device according to claim 1 .

3. a survey completion detection means for detecting the completion of a survey regarding the related survey content; and a task completion restriction means for permitting the user to complete the task in response to detection of completion of the survey related to the related survey content. The imaging support device according to claim 1 .

4. The method further includes: storing the associated investigation content in association with the predetermined mutation site; The imaging support device according to claim 1 .

5. a survey completion detection means for detecting the completion of a survey regarding the related survey content; and a survey completion notification means for outputting a notification to a user when the survey completion detection means detects that the survey on the related survey content has been completed. The imaging support device according to claim 1 .

6. A mutation site identification step of identifying a predetermined mutation site in a predetermined captured image; A related investigation content specification step of specifying related investigation content according to the attribute of the mutation site; a related investigation content output step of outputting the related investigation content to a user of the imaging device; An imaging assistance method comprising:

7. On the computer, A mutation site identification step of identifying a predetermined mutation site in a predetermined captured image; A related investigation content specification step of specifying related investigation content according to the attribute of the mutation site; a related investigation content output step of outputting the related investigation content to a user of the imaging device; A program for executing.

Images