Display method and program

Abstract

A display method includes: a first display step for displaying, on a display unit (22), an image being captured by an imaging unit (27) and a guide for including, in an imaging range, the lower end in the vertical direction of a long structure that deteriorates due to rust, during an operation in a first mode for imaging the lower end by the imaging unit (27); and an output step for outputting a still image captured by the imaging unit (27) when a prescribed trigger condition is satisfied.

Description

Display method and program

[0001] The present invention relates to a display method and a program.

[0002] Techniques relating to the maintenance of infrastructure facilities have been proposed. Patent Document 1 discloses a report management device that can effectively utilize reports relating to infrastructure facilities such as street lights.

[0003] Japanese Patent Application Laid-Open No. 2021-117878

[0004] When managing a structure together with a still image showing the structure, the still image may need to meet certain requirements, such as showing a specific part of the structure.

[0005] The present invention provides a display method and the like that can assist in taking a still image that meets predetermined requirements.

[0006] A display method according to one aspect of the present invention is a display method executed by an information terminal equipped with a photographing unit and a display unit, and includes a first display step of displaying on the display unit an image being photographed by the photographing unit and a guide for including the lower end in the vertical direction of a long structure that is deteriorating due to rust, during operation of a first mode for photographing the lower end in the vertical direction of the long structure that is deteriorating due to rust, using the photographing unit, and an output step of outputting a still image photographed by the photographing unit when a predetermined trigger condition is satisfied.

[0007] A program according to one aspect of the present invention is a program for causing the information terminal to execute the display method.

[0008] The display method and the like of the present invention can assist in taking still images that meet predetermined requirements.

[0009] FIG. 1 is a block diagram showing a functional configuration of an information processing system according to a first embodiment. FIG. 2 is a diagram showing an example of a user photographing a street light using an information terminal. FIG. 3 is a flowchart showing an example of a GUI display operation. FIG. 4 is a diagram showing an example of a GUI displayed on a display unit during operation in a first mode. FIG. 5 is a diagram showing an example of a GUI displayed on a display unit during operation in a second mode. FIG. 6 is a diagram showing an example of a GUI displayed on a display unit during operation in a third mode. FIG. 7 is a diagram showing an example of a GUI displayed on a display unit during operation in a fourth mode. FIG. 8 is a diagram showing an example of a GUI displayed on a display unit during operation in a fifth mode. FIG. 9 is a diagram showing an example of a GUI for transmitting image information. FIG. 10 is a diagram showing an example of a GUI for inputting a reason why a boundary portion cannot be photographed. FIG. 11 is a diagram showing an example of a GUI indicating the state of rust occurrence. FIG. 12 is a diagram showing a modified example of a GUI displayed on a display unit during operation in the first mode. FIG. 13 is a diagram showing an example of mosaic processing. FIG. 14 is a block diagram showing a functional configuration of an information processing system according to a second embodiment. FIG. 15 is a diagram showing an example of a user photographing a street light using an information terminal. FIG. 16 is a sequence diagram of a first operation example of an information processing system according to the second embodiment. FIG. 17 is a diagram showing an example of a photographing GUI (Graphical User Interface). FIG. 18 is a diagram showing an example of a deterioration degree input GUI. FIG. 19 is a diagram showing an example of a comment input GUI. FIG. 20 is a sequence diagram of a second operation example of an information processing system according to the second embodiment. FIG. 21 is a diagram showing an example of a selection GUI. FIG. 22 is a diagram showing an example of a display screen for posted information. FIG. 23 is a diagram showing an example of a display screen for new posted information. FIG. 24 is a diagram showing an example of a display screen for posted information and handling information. FIG. 25 is a block diagram showing the functional configuration of an information processing system according to a first modification of the second embodiment. FIG. 26 is a block diagram showing the functional configuration of an information processing system according to a second modification of the second embodiment. FIG. 27 is a block diagram showing the functional configuration of an information processing system according to a third modification of the second embodiment. FIG. 28 is a block diagram showing the functional configuration of an information processing system according to a fourth modification of the second embodiment.FIG. 29 is a block diagram showing a functional configuration of an information processing system according to Modification 5 of Embodiment 2. FIG. 30 is a block diagram showing a functional configuration of an information processing system according to Embodiment 3. FIG. 31 is a diagram showing an example of a user photographing a street light using an information terminal. FIG. 32 is a diagram showing a first example of a still image photographed using the information terminal. FIG. 33 is a diagram showing a second example of a still image photographed using the information terminal. FIG. 34 is a sequence diagram of the first half of an example of an operation of the information processing system according to Embodiment 3. FIG. 35 is a diagram showing an example of a display screen for structure management information. FIG. 36 is a sequence diagram of the second half of an example of an operation of the information processing system according to Embodiment 3. FIG. 37 is a diagram showing an example of a display screen for structure management information including maintenance management information. FIG. 38 is a block diagram showing a functional configuration of an information processing system according to Modification 1 of Embodiment 3. FIG. 39 is a block diagram showing a functional configuration of an information processing system according to Modification 2 of Embodiment 3.

[0010] Hereinafter, the embodiments will be described in detail with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection forms, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0011] It should be noted that the drawings are schematic diagrams and are not necessarily strict illustrations. In addition, in the drawings, substantially the same components are denoted by the same reference numerals, and overlapping descriptions may be omitted or simplified.

[0012] (First Embodiment) [Configuration] First, a description will be given of the configuration of an information processing system according to the first embodiment. Fig. 1 is a block diagram showing the functional configuration of the information processing system according to the first embodiment.

[0013] The information processing system 10 is a system that collects still images of long structures such as utility poles and street lamps (road lights) and manages the condition of the structures based on the collected still images. The information processing system 10 includes multiple information terminals 20, an image management system 30, a diagnostic system 40, a structure management system 50, and a local government terminal 60. These devices and systems will be described in detail below.

[0014] Each of the multiple information terminals 20 is an information terminal owned by a general user (such as a citizen). The general user (hereinafter also simply referred to as a user) uses the information terminal 20 to capture a still image of a structure and transmits image information of the captured still image to the image management system 30. FIG. 2 is a diagram showing an example of a user capturing an image of a street light (an example of a structure) using the information terminal 20. The information terminal 20 is, for example, a portable information terminal such as a smartphone or a tablet terminal. The information terminal 20 includes an operation receiving unit 21, a display unit 22, an information processing unit 23, a memory unit 24, a communication unit 25, a position measurement unit 26, and an image capturing unit 27.

[0015] The operation reception unit 21 receives operations from the user and is realized by, for example, a touch panel.

[0016] The display unit 22 displays images (moving images or still images) and is realized by a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel.

[0017] The information processing unit 23 performs processing such as displaying an image on the display unit 22 in response to an operation received by the operation receiving unit 21. The information processing unit 23 is realized, for example, by a microcomputer, but may also be realized by a processor. The functions of the information processing unit 23 are realized, for example, by the microcomputer or processor constituting the information processing unit 23 executing a computer program stored in the storage unit 24.

[0018] The storage unit 24 is a storage device that stores computer programs and the like executed by the information processing unit 23. The computer programs stored in the storage unit 24 include dedicated application programs (hereinafter also referred to as dedicated apps) that allow the information terminal 20 to access the image management system 30. The storage unit 24 is realized by, for example, a semiconductor memory.

[0019] The communication unit 25 is a communication circuit (communication module) that enables the information terminal 20 to communicate with the image management system 30 via a mobile communication network and a wide area communication network such as the Internet. The communication unit 25 is, for example, a wireless communication circuit that performs wireless communication. There are no particular limitations on the communication standard used for communication by the communication unit 25. The communication unit 25 may be a communication circuit that performs wireless communication in accordance with a communication standard such as Wi-Fi (registered trademark).

[0020] The position measurement unit 26 measures the current position (coordinates) of the information terminal 20 and outputs position information indicating the measured current position. The position measurement unit 26 is realized by, for example, a Global Navigation Satellite System (GNSS) module such as a Global Positioning System (GPS) module.

[0021] The photographing unit 27 is a camera that photographs an image (moving image or still image) of a structure, and is realized by an image sensor, a photographing optical system, and the like.

[0022] The image management system 30 is a system that receives image information from the information terminal 20, including a still image of a structure captured by the information terminal 20 and location information of the information terminal 20 at the time the still image was captured, and stores (manages) the received image information. The image management system 30 also transmits (provides) the image information to the diagnostic system 40 by communicating with the diagnostic system 40. The image management system 30 is realized by one or more server devices (cloud servers), but may also be realized serverlessly.

[0023] The diagnostic system 40 receives image information from the image management system 30 and diagnoses the degree of deterioration of a structure shown in a still image included in the received image information. The diagnostic system 40 is equipped with a machine learning model that can, for example, input an image and output a score indicating the degree of deterioration of the structure shown in the still image, and diagnoses the degree of deterioration of the structure using the machine learning model. The diagnostic system 40 also communicates with the structure management system 50 to transmit (provide) to the structure management system 50 image information and individual information about the structure including the diagnosis result of the degree of deterioration of the structure shown in the still image included in the image information. The diagnostic system 40 is implemented by one or more server devices (cloud servers), but may also be implemented serverlessly.

[0024] The structure management system 50 receives individual information about structures from the diagnostic system 40 and stores structure management information (a management ledger for multiple structures) that aggregates a large number of pieces of individual information about the received structures. The structure management system 50 also communicates with a local government terminal 60 to transmit (provide) individual information about structures specified by the local government terminal to the local government terminal 60. The structure management system 50 is implemented by one or more server devices (cloud servers), but may also be implemented serverlessly.

[0025] The local government terminal 60 is an information terminal used by local government employees. By accessing the structure management system 50 using the local government terminal 60, the local government employees can receive individual information about structures managed by the local government from the structure management system 50. The local government terminal 60 is, for example, a stationary information terminal such as a personal computer, but may also be a portable information terminal such as a smartphone or tablet terminal.

[0026] Here, a street light 70 shown in Fig. 2 will be described. The street light 70 is an example of a long structure (lighting device) that deteriorates due to rust, and is an example of a structure to be managed by the structure management system 50. In the following first embodiment, the structure will be described as a street light 70. The street light 70 includes a support pole 71, a lighting fixture 72, and a cover 73 that can be opened during maintenance (inspection). The cover 73 is, for example, a lid that can be opened to allow a maintenance worker to access a power supply device (power supply box) that supplies power to the lighting fixture 72 (light source).

[0027] [GUI Display Operation] As described above, the diagnostic system 40 diagnoses the deterioration level of the street light 70 shown in a still image based on the still image captured by the information terminal 20 running a dedicated app. When the diagnostic system 40 diagnoses the deterioration level of the still image using a machine learning model, the accuracy of the deterioration level diagnosis depends on how the street light 70 appears in the still image. In other words, the still image needs to appropriately depict the structure. However, because the still images are captured by general users with their cooperation, there is a possibility that the way the street light 70 appears in the still image will vary. In other words, in order to improve the accuracy of the deterioration level diagnosis, it is necessary to reduce the variation in the quality of the still images captured by the information terminal 20 and improve the quality of the still images.

[0028] Therefore, the dedicated application (information processing unit 23) serves as a GUI (Graphical User Interface) for capturing still images, and superimposes a guide on the moving image being captured by the image capturing unit 27, thereby suppressing variations in the quality of the still images. The display operation of such a GUI will be described below. Fig. 3 is a flowchart showing an example of the display operation of the GUI.

[0029] The information processing unit 23 of the information terminal 20 performs a first mode operation for capturing an image of the lower end portion in the vertical direction of the street light 70 using the imaging unit 27 (S11). During operation in the first mode, the information processing unit 23 displays, on the display unit 22, a moving image being captured by the imaging unit 27 and a guide for including the lower end portion of the street light 70 in the capturing range. Fig. 4 is a diagram showing an example of a GUI displayed on the display unit 22 during operation in the first mode.

[0030] As shown in Fig. 4, during operation in the first mode, text and a dashed frame 22a are displayed as a guide for including the bottom end of the street light 70 in the shooting range. In the example of Fig. 4, in addition to the dashed frame 22a, a dashed frame 22b is also displayed for including the boundary portion of the bottom end with the ground surface (i.e., the bottommost end of the support pole 71) in the shooting range. The dashed frame 22a and the dashed frame 22b are an example of a guide for specifying the position of the bottom end of the street light 70 in the moving image.

[0031] In step S11, it is sufficient that at least one of the text and the dashed frame 22 a is displayed as a guide, and it is not essential that both the text and the dashed frame 22 a are displayed. In addition, as another guide, a semi-transparent color filter may be displayed within the shooting range.

[0032] When the operation acceptance unit 21 accepts an operation on the capture button (at the bottom of the screen in FIG. 4 ) during operation in the first mode, the information processing unit 23 uses the capture unit 27 to capture a still image at the time the operation is accepted. The information processing unit 23 then generates image information that links the captured still image with the location information of the information terminal 20 at the time the still image was captured, and stores the generated image information in the storage unit 24. In other words, the information processing unit 23 outputs the image information to the storage unit 24. The location information is measured by the position measurement unit 26.

[0033] Next, the information processing unit 23 performs a second mode operation for capturing an image of the entire street light 70 by the imaging unit 27 (S12). During operation in the second mode, the information processing unit 23 displays, on the display unit 22, a moving image being captured by the imaging unit 27 and a guide for including the entire street light 70 in the imaging range. Fig. 5 is a diagram showing an example of a GUI displayed on the display unit 22 during operation in the second mode.

[0034] 5, during operation in the second mode, text and a dashed frame are displayed as a guide for including the entire street light 70 in the shooting range. Note that in step S12, it is sufficient that at least one of the text and the dashed frame is displayed as a guide, and it is not essential that both the text and the dashed frame are displayed.

[0035] When the operation acceptance unit 21 accepts an operation on the capture button (at the bottom of the screen in FIG. 5 ) during operation in the second mode, the information processing unit 23 captures a still image at the timing when the operation was accepted using the capture unit 27. The information processing unit 23 generates image information that links the captured still image with the location information of the information terminal 20 at the time the still image was captured, and stores the generated image information in the storage unit 24. In other words, the information processing unit 23 outputs the image information to the storage unit 24.

[0036] Next, the information processing unit 23 performs a third mode operation for capturing an image of the lighting fixture 72 using the imaging unit 27 (S13). During operation in the third mode, the information processing unit 23 displays, on the display unit 22, a moving image being captured by the imaging unit 27 and a guide for including the lighting fixture 72 in the capturing range. Fig. 6 is a diagram showing an example of a GUI displayed on the display unit 22 during operation in the third mode.

[0037] 6, during operation in the third mode, text and a dashed frame are displayed as a guide for including the lighting fixture 72 in the shooting range. Note that in step S13, it is sufficient that at least one of the text and the dashed frame is displayed as a guide; it is not essential that both the text and the dashed frame are displayed.

[0038] When the operation acceptance unit 21 accepts an operation on the capture button (at the bottom of the screen in FIG. 6 ) during operation in the third mode, the information processing unit 23 captures a still image at the timing when the operation was accepted using the capture unit 27. The information processing unit 23 generates image information that links the captured still image with the location information of the information terminal 20 at the time the still image was captured, and stores the generated image information in the storage unit 24. In other words, the information processing unit 23 outputs the image information to the storage unit 24.

[0039] Next, the information processing unit 23 performs a fourth mode operation to capture the identification information (hereinafter also referred to as the luminaire number) of the luminaire 72 using the image capturing unit 27 (S14). During operation in the fourth mode, the information processing unit 23 displays on the display unit 22 a moving image being captured by the image capturing unit 27 and a guide for including the identification information of the luminaire 72 in the capture range. Fig. 7 is a diagram showing an example of a GUI displayed on the display unit 22 during operation in the fourth mode. The identification information of the luminaire 72 is indicated, for example, on the support pole 71.

[0040] 7, during operation in the fourth mode, text and a dashed frame are displayed as a guide for including the identification information of the lighting fixture 72 in the shooting range. Note that in step S14, it is sufficient that at least one of the text and the dashed frame is displayed as a guide; it is not essential that both the text and the dashed frame are displayed.

[0041] When the operation acceptance unit 21 accepts an operation on the capture button (at the bottom of the screen in FIG. 7 ) during operation in the fourth mode, the information processing unit 23 captures a still image at the timing when the operation was accepted using the capture unit 27. The information processing unit 23 generates image information that links the captured still image with the location information of the information terminal 20 at the time the still image was captured, and stores the generated image information in the storage unit 24. In other words, the information processing unit 23 outputs the image information to the storage unit 24.

[0042] Next, the information processing unit 23 performs a fifth mode operation for capturing an image of the cover 73 (also referred to as a lid) that is opened during maintenance using the imaging unit 27 (S15). During operation in the fifth mode, the information processing unit 23 displays, on the display unit 22, a moving image being captured by the imaging unit 27 and a guide for including the cover 73 in the imaging range. Fig. 8 is a diagram showing an example of a GUI that is displayed on the display unit 22 during operation in the fifth mode.

[0043] 8, during operation in the fifth mode, text and a dashed frame are displayed as a guide for including cover 73 in the shooting range. Note that in step S15, it is sufficient that at least one of the text and the dashed frame is displayed as a guide, and it is not essential that both the text and the dashed frame are displayed.

[0044] When the operation acceptance unit 21 accepts an operation on the capture button (at the bottom of the screen in FIG. 8 ) during operation in the fifth mode, the information processing unit 23 captures a still image at the timing when the operation was accepted using the capture unit 27. The information processing unit 23 generates image information that links the captured still image with the location information of the information terminal 20 at the time the still image was captured, and stores the generated image information in the storage unit 24. In other words, the information processing unit 23 outputs the image information to the storage unit 24.

[0045] Next, the information processing unit 23 communicates with the image management system 30 using the communication unit 25, thereby transmitting (outputting) the image information (still images) generated and stored in steps S11 to S15 to the image management system 30 (S16). FIG. 9 is a diagram showing an example of a GUI for transmitting image information. In the example of FIG. 9, the information processing unit 23 transmits the image information to the image management system 30 using the communication unit 25, triggered by the operation reception unit 21 accepting an operation on the capture button (at the bottom of the screen in FIG. 9). As a result, the still images captured in steps S11 to S15 are uploaded to the image management system 30.

[0046] As described above, the dedicated app (information processing unit 23) can suppress variation in the quality of still images by superimposing a guide on the moving image being captured by the capturing unit 27 as a GUI for capturing still images.

[0047] 3 is an example and is not particularly limited. The order in which the first to fifth modes are performed may be selected by the user, for example.

[0048] Furthermore, it is not essential that five types of image information corresponding to the five modes, 1st mode to 5th mode, be transmitted, but it is sufficient that at least one type of image information be transmitted. Since the still image capturing the lower end of the street light 70, which is captured in the 1st mode, is mainly used to diagnose the degree of deterioration, it is sufficient that at least image information corresponding to the 1st mode be transmitted.

[0049] The lower end of the street light 70 is prone to deterioration such as rust due to urine from dogs and cats and being covered by vegetation. In order to prevent the collapse of the street light 70, it is important to diagnose the degree of deterioration using a still image of the lower end of the street light 70.

[0050] [Variation 1: Processing Related to Re-Shooting] In the GUI display operation described above, although a guide is displayed on the display unit 22, it is up to the user to decide whether or not a still image showing the street light 70 is captured according to the guide. However, after a still image is captured, the information processing unit 23 determines whether or not the still image satisfies predetermined requirements, and if it determines that the predetermined requirements are not satisfied, the information processing unit 23 may ask the user to re-capture the still image or may cancel the output of the still image (storage in the memory unit 24). The predetermined requirement here is whether or not a still image showing the street light 70 is captured according to the guide.

[0051] For example, when the dashed frame 22a is displayed as a guide in the first mode, the information processing unit 23 performs edge detection (contour extraction) processing of the street light 70 that appears in the captured still image. When the detected edge is located within the dashed frame 22a, the information processing unit 23 can determine that a still image has been captured in which the street light 70 appears as indicated by the guide. In other words, the information processing unit 23 can determine that the captured still image satisfies predetermined requirements.

[0052] In this case, the predetermined requirement is that the street light 70 is contained within the dashed frame 22a, but the street light 70 may be allowed to extend beyond the dashed frame 22a by a predetermined range. In other words, the predetermined requirement may be, for example, that the street light 70 does not extend beyond the guide beyond a predetermined range in the still image.

[0053] If the information processing unit 23 determines that the captured still image satisfies the predetermined requirements, it terminates the operation in the first mode. If the information processing unit 23 determines that the captured still image does not satisfy the predetermined requirements, it at least one of continues the operation in the first mode and stops outputting the still image.

[0054] This allows the image management system 30 to collect from the information terminal 20 only still images in which the street light 70 is captured, as guided.

[0055] [Variation 2: Input of reason why the lower end cannot be photographed] When the lower end of the street light 70 is covered with vegetation, in the first mode, it may be impossible for the information terminal 20 to photograph the boundary between the lower end of the street light 70 and the ground surface on which the street light 70 is installed. In such a case, the information processing unit 23 may display a screen (GUI) on the display unit 22 for inputting the reason why the boundary cannot be photographed. Fig. 10 is a diagram showing an example of a GUI for inputting the reason why the boundary cannot be photographed.

[0056] 10 is sent to the image management system 30 in addition to or instead of the image information, so that the image management system 30 can manage the reason why the boundary cannot be photographed.

[0057] Similarly, if the identification information of the lighting fixture 72 is not attached to the support 71 (for example, if it has peeled off and been lost), a screen (GUI) for inputting the reason why the identification information of the lighting fixture 72 cannot be photographed may be displayed on the display unit 22, and text corresponding to the reason may be input.

[0058] [Variation 3: Improving the display quality of street lights in still images] Street lights 70 are elongated structures that are long in the vertical direction. Therefore, as shown in Figures 4 to 8 above, during operation in the first to fifth modes, information processing unit 23 displays the moving image being captured by imaging unit 27 with the long side direction of display unit 22 of information terminal 20 as the vertical direction and the short side direction as the horizontal direction. This allows information terminal 20 to display street lights 70 large in still images.

[0059] While the dedicated app is running, the information processing unit 23 may control the photographing unit 27 to permit optical zooming of the photographing unit 27 but not permit digital zooming of the photographing unit 27. In other words, during operation in the first to fifth modes, optical zooming of the photographing unit 27 may be permitted but digital zooming of the photographing unit 27 may not be permitted. This allows the information terminal 20 to capture a still image in which the street light 70 is clearly visible.

[0060] [Variation 4: Output of Image Information] In the above-described GUI display operation, the capture of a still image and the output of image information including the captured still image to the storage unit 24 (in other words, the storage of image information to the storage unit 24) are triggered by a predetermined user operation (operation on the capture button). However, the capture of a still image and the output of image information may also be triggered by the passage of a predetermined time from a certain timing by timer processing. The capture of a still image and the output of image information may also be automatically triggered by the street light 70 shown in the moving image satisfying the above-described predetermined requirement (the requirement as to whether the street light 70 is shown as guided). In this case, the information processing unit 23 determines whether the predetermined requirement is satisfied. In this way, the capture of a still image and the output of image information may be performed when the predetermined trigger requirement is satisfied.

[0061] [Modification 5: Display of Rust State] When the lower end of the street light 70 is displayed on the display unit 22 during first mode operation, the information processing unit 23 may display the state of rust occurrence at the lower end on the display unit 22. Fig. 11 is a diagram showing an example of a GUI showing the state of rust occurrence.

[0062] 11 , the state of rust occurrence is scored using an indicator. The score indicating the state of rust occurrence is an example of information about the state of rust. In this case, the memory unit 24 of the information terminal 20 stores a first machine learning model as part of a dedicated app, and the information processing unit 23 uses the first machine learning model to score the state of rust occurrence at the bottom end of the street light 70 currently displayed on the display unit 22.

[0063] The first machine learning model is a trained model that has learned a large amount of training data that associates still images (or moving images) of the lower end of the street light 70 with scores of the rust occurrence state of the lower end shown in the still images. The first machine learning model can input a still image showing the lower end of the street light 70 that is currently displayed on the display unit 22, and output a score that indicates the rust occurrence state.

[0064] Furthermore, the information processing unit 23 may display the extent of rust occurrence as the state of rust occurrence at the lower end on the display unit 22. For example, the information processing unit 23 may display the extent of rust occurrence by changing the color of a specific color in a still image (or moving image) of the lower end of the street light 70 displayed on the display unit 22. The extent of rust occurrence is another example of information related to the state of rust.

[0065] In this case, the second machine learning model is stored in the memory unit 24 of the information terminal 20 as part of a dedicated app, and the information processing unit 23 uses the second machine learning model to identify the area of ​​rust occurrence in a still image (or moving image) of the lower end of the street light 70.

[0066] The second machine learning model is a trained model that has learned a large amount of training data that associates still images (or moving images) of the lower end of the street light 70 with the range of rust occurrence at the lower end shown in the still images. The second machine learning model is able to input a still image showing the lower end of the street light 70 that is currently displayed on the display unit 22 and identify the range of rust occurrence in the still image.

[0067] In this way, during operation in the first mode, the information processing unit 23 can support the user in understanding the state of rust on the street light 70 by displaying information on the display unit 22 regarding the state of rust on the street light 70. Note that the information processing unit 23 may also display information on the display unit 22 regarding the state of rust on parts other than the lower end of the street light 70, such as the cover 73.

[0068] Furthermore, the information about the rust state of the street light 70 is an example of information about the deterioration state of the street light 70, and the information processing unit 23 may display information about the deterioration state other than the rust state on the display unit 22. Examples of the deterioration state other than the rust state include the degree of deformation (straightness, etc.) of the support pole 71 of the street light 70 and the inclination angle of the support pole 71 with respect to the ground surface. By displaying the information about the deterioration state of the street light 70 on the display unit 22, the information processing unit 23 can assist the user in understanding the deterioration state of the street light 70.

[0069] [Variation 6: Linking Images to Location Information] In the above-described first embodiment, in order to link a still image to location information, it has been described that the current location of the information terminal 20 is measured by the position measurement unit 26 when the still image is captured. However, the method of linking a still image to location information is not limited to the method using the position measurement unit 26.

[0070] For example, a map may be displayed on the display unit 22 of the information terminal 20, and the user may plot points on the displayed map to specify the installation positions of the street lights 70.

[0071] Furthermore, for example, the image management system 30 may provide list information of the installation positions (coordinates) of pre-registered street lights 70 to the information terminal 20 running the dedicated app, and a GUI for selecting the installation positions based on the list information may be displayed on the display unit 22 of the information terminal 20. The selection GUI is, for example, a GUI in which icons are displayed at the installation positions of the street lights 70 on a map. By selecting an icon, the user can input the installation positions of the street lights 70 that appear in the captured still image. In this case, since the installation positions of the street lights 70 are managed in the image management system 30, the information terminal 20 may transmit identification information of the selected icon to the image management system 30 instead of transmitting position information.

[0072] [Variation 7: Display of Still Images Captured in the Past] In the above-described first embodiment, it is desirable that the street light 70 is captured at the same angle of view in multiple still images captured at different times and featuring the same street light 70 as a subject, which are managed by the image management system 30. This allows for fixed-point observation of the street light 70, making it easier to determine whether the street light 70 has deteriorated.

[0073] Therefore, a still image captured in the past may be displayed in the GUI for capturing an image of the street light 70, as a reference for the angle of view of the street light 70 to be captured in the future. Fig. 12 is a diagram showing such a modified example of the GUI displayed on the display unit 22 of the information terminal 20 during operation in the first mode.

[0074] 12, a still image captured in the past and a moving image capturing the current street light 70 are displayed side by side at the same size. By displaying such a GUI on the display unit 22, the information processing unit 23 of the information terminal 20 can support the user in capturing an image of the street light 70 with the same angle of view as the past still image.

[0075] It is not essential that the still image captured in the past and the moving image showing the current street light 70 are displayed side by side at the same size. For example, the still image captured in the past may be smaller than the moving image and displayed somewhere around the moving image. Also, the still image captured in the past may be made semi-transparent and superimposed on the moving image. Furthermore, although FIG. 12 shows the GUI for the first mode, the still image captured in the past may be displayed in the GUIs for the second to fifth modes.

[0076] A further explanation will be given regarding the method for displaying still images captured in the past. For example, when a user is about to capture a still image, the user selects the position of the street light 70 to be captured from among points on the map displayed on the display unit 22 of the information terminal 20. The information processing unit 23 transmits a request for image information, including position information corresponding to the selected point, to the image management system 30 using the communication unit 25. This allows the information processing unit 23 to obtain image information of a past still image of the street light 70 to be captured from the image management system 30 and display the image information on the display unit 22.

[0077] [Variation 8: Mosaic Processing] There is a possibility that people, houses, cars, and other surrounding objects located around the street light 70 may end up appearing in the still image. Furthermore, if the surface of the street light 70 is glossy, the photographer or surrounding objects (hereinafter also referred to as surrounding objects, etc.) may end up appearing in the street light 70 itself in the still image.

[0078] As described above, local government officials can access the structure management system 50 using the local government terminal 60 and view still images stored (managed) in the structure management system 50, but it may not be desirable from the perspective of protecting personal information if surrounding objects, etc. are captured in the still images.

[0079] Therefore, in a still image provided from the structure management system 50 to the local government terminal 60, surrounding objects and the like reflected in the still image may be pixelated. Fig. 13 is a diagram showing an example of pixelation processing.

[0080] For example, in the structure management system 50, an original still image without mosaic processing ((a) in FIG. 13 ) and a still image with mosaic processing ((b) in FIG. 13 ) are stored as a pair. For example, when a local government employee accesses the structure management system 50, they are asked to enter their user ID. Employees who are determined to have administrator authority based on their user ID are permitted to view the original still image, while employees who are determined to have administrator authority are prohibited from viewing the original still image and are only permitted to view the still image with mosaic processing. Storing the original still image prevents information necessary for managing the street light 70 from being accidentally lost due to mosaic processing.

[0081] The mosaic processing may be performed manually by an administrator of the structure management system 50, or may be performed automatically by the structure management system 50. The structure management system 50 can perform mosaic processing on detected objects, for example, by performing object detection processing (e.g., person detection or face detection) on a still image.

[0082] It is not essential that the original still image and the mosaic-processed still image are stored as a pair. For example, the structure management system 50 may store only the original still image, and the mosaic process may be performed each time a still image is provided.

[0083] Furthermore, peripheral objects and the like reflected in a still image may be pixelated in the still image provided from the image management system 30 to the information terminal 20. For example, in a past still image displayed on the GUI of Fig. 12, peripheral objects and the like reflected in the still image may be pixelated.

[0084] In this case, the original still image and the still image after mosaic processing may be stored as a pair in the image management system 30, or only the original still image may be stored in the image management system 30, and the mosaic processing may be performed each time a still image is provided.

[0085] [Effects, etc.] Hereinafter, examples of inventions obtained from the disclosure of this specification will be described, and effects, etc. obtained from the exemplified inventions will be explained.

[0086] Invention 1 is a display method executed by an information terminal 20 equipped with a photographing unit 27 and a display unit 22, the display method including: a first display step of displaying, on the display unit 22, an image being photographed by the photographing unit 27 and a guide for including the lower end in the vertical direction of a long structure that is deteriorating due to rust, during operation in a first mode for photographing the lower end in the vertical direction of the long structure by the photographing unit 27; and an output step of outputting a still image photographed by the photographing unit 27 when a predetermined trigger condition is satisfied. The output of the still image in the output step may be interpreted as output to a storage unit 24 provided in the information terminal 20, or as output (transmission) to an image management system 30.

[0087] Such a display method can assist in taking a still image that shows the lower end of a structure by displaying a guide on the display unit 22.

[0088] Invention 2 is a display method of Invention 1, further including a second display step of displaying on the display unit 22 the image being photographed by the photographing unit 27 and a guide for including the entire structure in the photographing range during operation of the second mode for photographing the entire structure by the photographing unit 27.

[0089] Such a display method can assist in taking a still image showing the entire structure by displaying a guide on the display unit 22.

[0090] Invention 3 is the display method of Invention 1 or 2, in which the structure is a lighting device (street light 70) having a support 71 and a lamp 72, and the display method further includes a third display step of displaying on the display unit 22, during operation of a third mode for photographing the lamp 72 by the photographing unit 27, an image being photographed by the photographing unit 27 and a guide for including the lamp 72 in the photographing range.

[0091] This display method can assist in taking a still image in which the lighting fixture 72 appears by displaying a guide on the display unit 22.

[0092] Invention 4 is a display method of any of Inventions 1 to 3, in which the structure is a lighting device (street light 70) having a support 71 and a lamp 72, and the display method further includes a fourth display step of displaying on the display unit 22 an image being captured by the photographing unit 27 and a guide for including the identification information in the photographing range during operation of a fourth mode in which identification information related to the lamp 72 is captured by the photographing unit 27.

[0093] This display method can assist in taking a still image showing the identification information about the lighting fixture 72 by displaying a guide on the display unit 22 .

[0094] Invention 5 is a display method of any of Inventions 1 to 4, in which the structure is a lighting device (street light 70) having a pole 71, a lamp 72, and a cover 73 provided on the pole 71, and the display method further includes a fifth display step of displaying, in a fifth mode in which the cover 73 is photographed by the photographing unit 27, an image being photographed by the photographing unit 27 and a guide for keeping the cover 73 within the photographing range on the display unit 22.

[0095] Such a display method can assist in taking a still image showing the identification information related to the cover 73 by displaying a guide on the display unit 22 .

[0096] A sixth aspect of the present invention is the display method of any one of the first to fifth aspects of the present invention, wherein the first display step displays information about the deterioration state of the structure on the display unit 22.

[0097] Such a display method can help understand the state of deterioration of a structure.

[0098] Invention 7 is the display method of invention 6, in which the state of deterioration of the structure includes the state of rust in the structure.

[0099] Such a display method can help understand the state of rust in a structure.

[0100] Invention 8 is the display method of invention 7, in which the state of rust includes the range in which rust has occurred in the structure.

[0101] Such a display method can help to grasp the extent of rust in a structure.

[0102] A ninth aspect of the present invention is the display method of any one of the first to eighth aspects of the present invention, wherein the guide displayed in the first display step includes a guide for specifying the position of the bottom end of the structure in the image.

[0103] Such a display method makes it possible to specify the position of the bottom end of a structure in the image.

[0104] Invention 10 is a display method of any of Inventions 1 to 9, further including a step of, if the lower end of the structure shown in the still image does not satisfy predetermined requirements during operation in the first mode, continuing operation in the first mode and / or canceling the output step.

[0105] Such a display method can prevent a still image that does not satisfy predetermined requirements from being captured.

[0106] Invention 11 is a display method of any of Inventions 1 to 10, further including a step of displaying on the display unit 22 a screen for inputting the reason why it is not possible to photograph the boundary between the lower end of the structure and the ground surface on which the structure is installed, when it is not possible to photograph the boundary.

[0107] Such a display method can help manage the reasons why the border cannot be photographed.

[0108] Invention 12 is a display method according to any one of Inventions 1 to 11, in which, in the first display step, the long side direction of the display unit 22 of the information terminal 20 is vertical and the short side direction is horizontal, and the image being captured by the photographing unit 27 is displayed.

[0109] This type of display method allows structures to be displayed large in still images.

[0110] A thirteenth aspect of the present invention is the display method of any one of the first to twelfth aspects, wherein optical zoom of the photographing unit 27 is permitted but digital zoom of the photographing unit 27 is not permitted during operation in the first mode.

[0111] This display method makes it possible to capture still images in which the structure is clearly visible.

[0112] A fourteenth aspect of the present invention is the display method of any one of the first to thirteenth aspects of the present invention, wherein the predetermined trigger condition includes a condition that a predetermined operation is performed by the user.

[0113] Such a display method can output a still image captured by the image capturing unit 27 when a predetermined operation is performed by the user.

[0114] A fifteenth aspect of the present invention is a program for causing an information terminal 20 to execute any one of the display methods of the first to fourteenth aspects.

[0115] According to such a program, the information terminal 20 can display a guide on the display unit 22 to assist in taking a still image that shows the bottom end of a structure.

[0116] (Embodiment 2) [Configuration] First, a description will be given of the configuration of an information processing system according to embodiment 2. Fig. 14 is a block diagram showing the functional configuration of the information processing system according to embodiment 2.

[0117] The information processing system 110 is a system that accepts posts of images or comments showing the status of outdoor structures from general users and makes the posted contents public. Examples of outdoor structures include utility poles, street lights, fences, and playground equipment.

[0118] The information processing system 110 includes a plurality of information terminals 120, a post management system 130, a diagnostic system 140, a structure management system 150, and a local government terminal 160. These devices and systems will be described in detail below.

[0119] Each of the multiple information terminals 120 is owned by a general user (such as a citizen). The general user (hereinafter simply referred to as "user") uses the information terminal 120 to capture a still image showing the condition of a structure, and transmits posted information including the captured still image to the posting management system 130. FIG. 15 is a diagram showing an example of a user capturing an image of a street light (an example of a structure) using the information terminal 120. The user can also input a comment about the condition of the structure into the information terminal 120 and include the input comment in the posted information.

[0120] The information terminal 120 is a portable information terminal such as a smartphone or a tablet terminal, and includes an operation receiving unit 121, a display unit 122, an information processing unit 123, a storage unit 124, a communication unit 125, a position measurement unit 126, and an image capturing unit 127.

[0121] The operation reception unit 121 receives operations from a user and is realized by, for example, a touch panel.

[0122] The display unit 122 displays images (moving images or still images) and is realized by a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel.

[0123] The information processing unit 123 performs processing such as displaying an image on the display unit 122 in response to an operation received by the operation receiving unit 121. The information processing unit 123 is realized, for example, by a microcomputer, but may also be realized by a processor. The functions of the information processing unit 123 are realized, for example, by the microcomputer or processor constituting the information processing unit 123 executing a computer program stored in the storage unit 124.

[0124] The storage unit 124 is a storage device that stores computer programs and the like executed by the information processing unit 123. The computer programs stored in the storage unit 124 include a dedicated application program (hereinafter also referred to as a dedicated app) that enables the information terminal 120 to access the post management system 130. The storage unit 124 is realized by, for example, a semiconductor memory.

[0125] The communication unit 125 is a communication circuit (communication module) that enables the information terminal 120 to communicate with the post management system 130 via a mobile communication network and a wide area communication network such as the Internet. The communication unit 125 is, for example, a wireless communication circuit that performs wireless communication. There are no particular limitations on the communication standard used for communication by the communication unit 125. The communication unit 125 may be a communication circuit that performs wireless communication in accordance with a communication standard such as Wi-Fi (registered trademark).

[0126] The position measurement unit 126 measures the current position (coordinates) of the information terminal 120 and outputs position information indicating the measured current position. The position measurement unit 126 is realized, for example, by a GNSS (Global Navigation Satellite System) module such as a GPS (Global Positioning System) module. The posted information described above includes position information indicating the current position of the information terminal 120 when the still image included in the posted information was captured, and the current position indicated by this position information is measured, for example, by the position measurement unit 126.

[0127] The image capturing unit 127 is a camera that captures images (moving images or still images) of structures. The image capturing unit 127 is realized by an image sensor, an optical system for capturing images, and the like.

[0128] The post management system 130 is a system that receives posted information generated by the information terminal 120 from the information terminal 120 and stores (manages) the received posted information. The post management system 130 is a system that provides a social networking service (SNS), but may also be a system that provides a bulletin board system (BBS), or any other system that can manage posted information. The post management system 30 also communicates with the diagnostic system 140 to transmit (provide) image information included in the posted information to the diagnostic system 140, and communicates with the structure management system 150 via a wide area communication network to transmit the posted information to the structure management system 150.

[0129] The post management system 130 is implemented by one or more server devices (cloud servers), but may also be implemented serverlessly. The post management system 130 includes a communication unit 131, an information processing unit 132, and a storage unit 133.

[0130] The communication unit 131 is a communication circuit (communication module) that enables the post management system 130 to communicate with the information terminal 120, the diagnostic system 140, and the structure management system 150 via a wide area communication network. The communication unit 131 is, for example, a wired communication circuit that performs wired communication, but may also be a wireless communication circuit that performs wireless communication. There are no particular limitations on the communication standard for the communication performed by the communication unit 131.

[0131] The information processing unit 132 performs information processing related to the management and provision of posted information. The information processing unit 132 is realized, for example, by a microcomputer, but may also be realized by a processor. The information processing unit 132 includes, as functional components, a management unit 134 and an output unit 135. The functions of the management unit 134 and the output unit 135 are realized, for example, by the microcomputer or processor constituting the information processing unit 132 executing a computer program stored in the storage unit 133.

[0132] The storage unit 133 is a storage device that stores computer programs executed by the information processing unit 132, posted information, etc. The storage unit 133 is realized by, for example, a hard disc drive (HDD), but is also realized by a semiconductor memory.

[0133] The diagnostic system 140 receives image information from the post management system 130 and diagnoses the degree of deterioration of a structure shown in a still image included in the received image information. The diagnostic system 140 is equipped with a machine learning model that can input, for example, a still image and output a score indicating the degree of deterioration of the structure shown in the still image, and diagnoses the degree of deterioration of the structure using the machine learning model. The diagnostic system 140 also communicates with the structure management system 150 to transmit (provide) the diagnosis result of the degree of deterioration of the structure to the structure management system 150. The diagnostic system 140 is implemented by one or more server devices (cloud servers), but may also be implemented serverlessly.

[0134] In addition, after receiving image information from the post management system 130, the diagnostic system 140 may transmit the deterioration level diagnostic results to the post management system 130, and the post management system 130 may transmit the posted information including the deterioration level diagnostic results to the structure management system 150.

[0135] The structure management system 150 receives posted information from the posting management system 130, generates individual information for structures based on the received posted information, and stores structure management information (a management ledger for multiple structures) that aggregates a large number of the generated individual information for structures. The structure management system 150 also receives the results of a structure deterioration assessment from the assessment system 140, and includes the received results of a structure deterioration assessment in the individual information for the structure. The structure management system 150 also communicates with the local government terminal 160, transmitting (providing) the individual information for structures specified by the local government terminal to the local government terminal 160. The structure management system 150 is implemented by one or more server devices (cloud servers), but may also be implemented serverlessly.

[0136] The local government terminal 160 is an information terminal used by local government employees. By using the local government terminal 160 to access the structure management system 150, the local government employees can receive individual information about structures managed by the local government from the structure management system 150. The local government terminal 60 is, for example, a stationary information terminal such as a personal computer, but may also be a portable information terminal such as a smartphone or tablet terminal.

[0137] [Operation Example 1: Operation when accepting a post from a user] As described above, the information processing system 110 can accept posts such as still images or comments showing the state of a structure installed outdoors from a user. Hereinafter, an operation example 1 of such an information processing system 110 will be described. Fig. 16 is a sequence diagram of operation example 1 of the information processing system 110.

[0138] When a user performs a predetermined operation on the operation acceptance unit 121 of the information terminal 120 running the dedicated app, the information processing unit 123 displays a GUI for capturing a still image (a capture GUI) on the display unit 122 based on the predetermined operation accepted by the operation acceptance unit 121 (S21). Fig. 17 is a diagram showing an example of the capture GUI.

[0139] When the operation accepting unit 121 accepts an operation on the image capturing button (at the bottom of the screen in FIG. 17 ) while the image capturing GUI is displayed, the information processing unit 123 captures a still image at the timing when the operation is accepted using the image capturing unit 127 (S22). The position measuring unit 126 measures the current position of the information terminal 120 when the still image is captured, and generates position information indicating the measured current position (S23).

[0140] It is not essential that the location information be measured (generated) by the location measurement unit 126, and the location information (the installation location of the structure) may be manually input by the user. For example, a map may be displayed on the display unit 122, and the user may specify the installation location of the structure by plotting a point on the map.

[0141] Furthermore, the post management system 130 may provide a list of installation positions (coordinates) of registered structures to the information terminal 120 running the dedicated app, and a GUI for selecting an installation position based on the list information may be displayed on the display unit 122 of the information terminal 120. The selection GUI may be, for example, a GUI in which icons are displayed at the installation positions of structures on a map. By selecting an icon, the user can input the installation position of the structure shown in the captured still image. In this case, since the installation positions of the street lights 80 are managed in the post management system 130, there is no need for the position measurement unit 126 to generate position information. Instead, the information processing unit 123 may transmit post information including identification information of the selected icon instead of position information in step S28 described below.

[0142] The information processing unit 123 also displays a GUI (deterioration level input GUI) on the display unit 122 for inputting the state of the structure (e.g., the degree of deterioration) (S24). FIG. 18 is a diagram showing an example of the deterioration level input GUI. The GUI in FIG. 18 allows the user to select the degree of deterioration of the structure from multiple options (not deteriorated, slightly deteriorated, deteriorated, and significantly deteriorated) prepared in advance. The information processing unit 123 generates deterioration level information indicating the degree of deterioration selected through the GUI shown in FIG. 18 (S25).

[0143] 18. For example, if the structure is a utility pole or a street light, the degree of rust at the base of the pole may be provided as an option. Also, hashtags indicating the degree of deterioration of the structure may be provided as options.

[0144] The information processing unit 123 also displays a GUI (comment input GUI) on the display unit 122 for freely inputting a comment about the state of the structure (e.g., the degree of deterioration) in text (S26). FIG. 19 is a diagram showing an example of the comment input GUI. The GUI in FIG. 19 includes an operation area for inputting the state of the structure in text. The information processing unit 123 generates text information for the text input through the GUI shown in FIG. 19 (S27). When inputting text, predictive conversion may be performed, or hashtags similar to the input words may be suggested to the user.

[0145] The information processing unit 123 generates posting information including image information of the still image taken in step S22, location information generated in step S23, deterioration level information generated in step S25, text information generated in step S27, and poster information, and transmits the generated posting information to the posting management system 130 (S28).

[0146] The communication unit 131 of the post management system 130 receives the posted information. The management unit 134 stores the received posted information in the storage unit 133 (S29). In the posted information, first information regarding the state of a structure that can be visually confirmed by a user of the information terminal 120, such as image information, deterioration level information, and comment information, is associated with poster information and location information. Since the location of the information terminal 120 when the still image indicated by the location information was captured is considered to be the installation location of the structure, the management unit 134 can be said to manage the first information in association with the installation location of the structure by storing the posted information in the storage unit 133. Similarly, since the first information is associated with poster information in the posted information, the management unit 134 can be said to manage the first information in association with poster information (information about the user) by storing the posted information in the storage unit 133.

[0147] In this way, the information processing system 110 can accept posts from users, such as still images showing the condition of outdoor structures. Note that it is not essential that the posted information include all four types of information, i.e., image information, deterioration level information, comment information, and location information; the posted information may include at least one of the above four types of information as needed. In other words, the posted information may include one or more of the four types of information.

[0148] Furthermore, although it has been explained that the deterioration level information is input by the user, the information processing unit 132 of the posting management system 130 can also inquire about the degree of deterioration of the structure (deterioration level information) from the diagnostic system 140 by using the communication unit 131 to send image information included in the posted information to the diagnostic system 140.

[0149] [Operation Example 2: Operation When Providing Posted Information to a User] A user of the information processing system 110 can access the post management system 130 using the information terminal 120 to view posted information previously posted by that user or posted information previously posted by other users. In other words, the information processing system 110 can provide posted information to the user. Below, an operation example 2 of such an information processing system 110 will be described. FIG. 20 is a sequence diagram of operation example 2 of the information processing system 110. Note that in the following description of operation example 2, the information terminal 120 of a general user will be referred to as information terminal 120a, and the information terminal 120 of the structure manager will be referred to as information terminal 120b.

[0150] When a user performs a predetermined operation on the operation acceptance unit 121 of the information terminal 120a running the dedicated app, the information processing unit 123 displays a GUI for selecting a structure (selection GUI) on the display unit 122 based on the predetermined operation accepted by the operation acceptance unit 121 (S31). Fig. 21 is a diagram showing an example of the selection GUI.

[0151] 21 , for example, the selection GUI displays a map of the area around the current location of the information terminal 120a, and displays an icon of the structure at the installation position of the structure on the map. As described above, the posted information of the structure includes location information. The information processing unit 123 communicates with the post management system 130 using the communication unit 125 and acquires, from the post management system 130, location information of structures located around the current location of the information terminal 120a, thereby enabling the icon of the structure to be displayed at the installation position on the map.

[0152] While the selection GUI is displayed, the operation accepting unit 121 accepts a user's operation to select a structure (S32). Based on the selection operation accepted by the operation accepting unit 121, the information processing unit 123 transmits a request for providing posted information about the selected structure to the post management system 130 using the communication unit 125 (S33).

[0153] The communication unit 131 of the post management system 130 receives the request to provide the posted information. The output unit 135 uses the communication unit 131 to transmit (output) the posted information of the structure selected in step S32 (more specifically, the first information indicating the state of the structure included in the posted information) to the information terminal 120a (S34).

[0154] The communication unit 125 of the information terminal 120a receives the posted information. The information processing unit 123 displays (visualizes) the received posted information on the display unit 122 (S35). Fig. 22 is a diagram showing an example of a display screen for posted information. The display screen in Fig. 22 displays image information, deterioration level information, and comment information for the selected structure.

[0155] Here, the user can make a new post by further inputting the latest image information, deterioration level information, and comment information for the selected structure. For example, the information processing unit 123 generates new posted information by performing the same process as in Operation Example 1 (S36), and transmits the generated new posted information to the post management system 130 (S37). Note that in step S36, the GUI (FIGS. 17 to 19) for inputting the state of the structure, as described in Operation Example 1, is displayed.

[0156] The communication unit 131 of the post management system 130 receives new posted information. The management unit 134 associates the received new posted information with the old posted information and stores it in the storage unit 133 (S38). In other words, the management unit 134 manages the new first information included in the received new posted information by associating it with the installation location of the structure and the old first information (such as an image of the structure previously taken by the user or another user). In this way, the post management system 130 can manage changes in the state of the same structure (changes in state over time).

[0157] Incidentally, not only general users but also managers of structures can access the posting management system 130 using the information terminal 120b and view newly posted information. FIG. 23 is a diagram showing an example of a display screen for newly posted information. The manager may be a local government (a local government employee) or a business operator contracted by the local government (an employee of the business operator). If the manager is a local government, the manager may also access the posting management system 130 using the local government terminal 160.

[0158] 23 , when new posted information (first information) indicates that the condition of a structure is poor, the administrator or the like can post the status of the response to the structure. For example, the information processing unit 123 of the information terminal 120b generates response information indicating the status of the response and transmits the generated response information to the posting management system 130 using the communication unit 125 (S39). Although not shown, when the response information is generated, a GUI for inputting the response information is also displayed.

[0159] The communication unit 131 of the posting management system 130 receives the handling information. The management unit 134 associates the received handling information with the series of posted information and stores it in the storage unit 133 (S40).

[0160] After the processing of step S30 is performed, when the posted information is displayed on the display unit 122 of the information terminal 120a, the output unit 135 of the posting management system 130 transmits (outputs) the handling information to the information terminal 120a in addition to the posted information. As a result, the handling information is displayed on the display unit 122 in addition to the posted information. Figure 24 is a diagram showing an example of a display screen for the posted information and the handling information. Note that the handling information may be information that indicates at least one of a planned handling and the details of the handling that has been completed.

[0161] In this way, the information processing system 110 can provide users with posted information about the status of structures. The information processing system 110 can also accept new posts from users about structures for which posted information exists, and can accept response information from managers of the structures.

[0162] 22 to 24, the poster's name and the poster's authority information (in other words, attribute information) are displayed as poster information included in the posted information. In the examples of FIGS. 22 to 24, the poster's authority is of three types: "general," "expert," and "administrator." "General" indicates that the poster is a general user, "expert" indicates that the poster has specialized knowledge about structures, and "administrator" indicates that the poster is responsible for managing the structure. The poster's authority is set, for example, when registering a user after a dedicated app is installed on the information terminal 120.

[0163] In this way, if the posted information managed by the posting management system 130 includes information regarding the poster's authority and the poster's authority is displayed on the display screen of the posted information, users viewing the posted information can view the contents of the posted information after understanding the poster's authority.

[0164] The scope of viewable posted information may be changed depending on the authority. For example, a user with "administrator" authority may be able to view all posted information, a user with "expert" authority may be able to view posted information by users with "expert" authority and posted information by users with "general" authority, and a user with "general" authority may be able to view only posted information by users with "general" authority.

[0165] [Modification of Selection GUI] In the selection GUI described in the above operation example 2, for example, icons of all structures that are located within a predetermined range of the current location of the information terminal 120a on the map and for which posted information exists are displayed, but icons of only some of the structures may also be displayed. In other words, the icons displayed may be narrowed down.

[0166] For example, a user may specify a filtering condition so that only icons that meet the specified condition are displayed, and icons that do not meet the specified condition are hidden. Narrowing down the icons displayed in the selection GUI. The filtering condition may be, for example, a condition that the structure is one for which the user himself / herself has posted a status in the past, or a condition that there is a post by a poster with "expert" or "administrator" authority.

[0167] The filtering condition may also be that there is response information. The filtering condition may also be that there is a new post within the most recent predetermined period (for example, one month). The filtering condition may also be that the deterioration level is above or below a predetermined value. If the user has "administrator" authority, the filtering condition may also be that the structure is under the management of the user.

[0168] Furthermore, the display mode of the icons may be changed instead of being narrowed down. For example, the icons of structures that meet specified conditions may be displayed in a different display mode from other icons. For example, the icons of structures that meet specified conditions may be displayed in a flashing manner, or at least one of the shape, size, and display color may be changed.

[0169] In this way, if it is possible to narrow down the structures displayed in the selection GUI, or if it is possible to display some of the structures displayed in the selection GUI in a different display format from other structures, there is an advantage in that it makes it easier to select structures.

[0170] [System Configuration Variation 1] The information processing system 110 may include an image management system that provides an SNS service specialized for posting images, in addition to the post management system 130 that provides an SNS service that allows posting of images and comments. Fig. 25 is a block diagram showing the functional configuration of such an information processing system according to Variation 1.

[0171] As shown in FIG. 25, the information processing system 110a according to the first variant includes a plurality of information terminals 120, a post management system 130, a diagnostic system 140, a structure management system 150, a local government terminal 160, and an image management system 170.

[0172] The image management system 170 is a system that receives image information from the information terminal 120, including a still image of a structure captured by the information terminal 120 and location information of the information terminal 120 at the time the still image was captured, and stores (manages) the received image information. The image management system 170 also communicates with the diagnostic system 140 to transmit (provide) the image information to the diagnostic system 140. The image management system 170 is realized by one or more server devices (cloud servers).

[0173] In the information processing system 110a, the diagnostic system 140 receives image information from the image management system 170 and diagnoses the degree of deterioration of the structure shown in the still image included in the received image information. The post management system 130 does not transmit (provide) image information to the diagnostic system 140, but does transmit (provide) posted information to the structure management system 150. In this way, the post management system 130 may be realized as a system that does not transmit image information to the diagnostic system 140. In the operation examples 1 and 2 of the above-mentioned second embodiment, the information processing system 110 may be read as the information processing system 110a.

[0174] [Second Modification of System Configuration] In the information processing system 110, the structure management system 150 may have the same functions as the post management system 130. Fig. 26 is a block diagram showing the functional configuration of such an information processing system according to the second modification.

[0175] 26 , an information processing system 110b according to the second modification includes a plurality of information terminals 120, a diagnostic system 140, a structure management system 150, a local government terminal 160, and an image management system 170. The structure management system 150 includes a communication unit 151, an information processing unit 152, and a storage unit 153, and the information processing unit 152 includes, as functional components, a management unit 154 and an output unit 155.

[0176] In this way, the structure management system 150 may have the same functions as the post management system 130. In the operation examples 1 and 2 of the second embodiment, the information processing system 110, the post management system 130, the communication unit 131, the information processing unit 132, the memory unit 133, the management unit 134, and the output unit 135 may be read as the information processing system 110b, the structure management system 150, the communication unit 151, the information processing unit 152, the memory unit 153, the management unit 154, and the output unit 155.

[0177] [Modification 3 of System Configuration] The block diagram showing the functional configuration of the information processing system 110 may be modified as shown in Fig. 27. Fig. 27 is a block diagram showing the functional configuration of an information processing system according to Modification 3.

[0178] In the information processing system 110 of Fig. 14, image information is provided from the post management system 130 to the diagnostic system 140, and the diagnostic system 140 provides the diagnosis result of the deterioration level of the structure to the structure management system 150. In contrast, in the information processing system 110c of Fig. 27, image information and post information are provided from the post management system 130 to the structure management system 150, and the structure management system 150 provides the image information to the diagnostic system 140 and can obtain the diagnosis result of the deterioration level of the structure in response.

[0179] [Fourth Modification of System Configuration] The block diagram showing the functional configuration of the information processing system 110a may be modified as shown in Fig. 28. Fig. 28 is a block diagram showing the functional configuration of an information processing system according to a fourth modification.

[0180] In the information processing system 110a of Fig. 25, image information is provided from the image management system 170 to the diagnostic system 140, and the diagnostic system 140 provides the diagnosis results of the deterioration level of the structure to the structure management system 150. In contrast, in the information processing system 110d of Fig. 28, image information is provided from the image management system 170 to the structure management system 150, and the structure management system 150 provides the image information to the diagnostic system 140, and in response can obtain the diagnosis results of the deterioration level of the structure.

[0181] [Fifth Modification of System Configuration] The block diagram showing the functional configuration of the information processing system 110b may be modified as shown in Fig. 29. Fig. 29 is a block diagram showing the functional configuration of an information processing system according to a fifth modification.

[0182] In the information processing system 110b of Fig. 26, image information is provided from the image management system 170 to the diagnostic system 140, and the diagnostic system 140 provides the diagnosis results of the degree of deterioration of the structure to the structure management system 150. In contrast, in the information processing system 110e of Fig. 29, image information is provided from the image management system 170 to the structure management system 150, and the structure management system 150 provides the image information to the diagnostic system 140, and in response can obtain the diagnosis results of the degree of deterioration of the structure.

[0183] [Other Modifications] The post management system 130 manages posted information related to outdoor structures, but may also manage posted information related to outdoor facilities. The post management system 130 may manage posted information related to objects including at least one of structures and facilities.

[0184] [Effects, etc.] Hereinafter, examples of inventions obtained from the disclosure of this specification will be described, and effects, etc. obtained from the exemplified inventions will be explained.

[0185] Invention 1 is a management system including a communication unit that receives first information regarding the state of an object installed outdoors, which can be visually confirmed by a user of the information terminal 120 by communicating with the information terminal 120, and a management unit that manages the received first information in association with the installation position of the object. Each of the post management system 130 included in the information processing system 110, the post management system 130 included in the information processing system 110a, and the structure management system 150 included in the information processing system 110b is an example of a management system.

[0186] Such a management system can manage the status of objects installed outdoors.

[0187] Invention 2 is a management system of Invention 1, in which the management unit manages the received first information by correlating it with the installation location of the object and the state of the object previously input by the user or another user.

[0188] Such a management system can manage an object installed outdoors by associating the latest state of the object with the past state of the object.

[0189] A third aspect of the present invention is the management system of the first or second aspect, wherein the first information includes information indicating a degree of deterioration of the object.

[0190] Such a management system can manage the degree of deterioration of an object.

[0191] A fourth aspect of the present invention is the management system according to any one of the first to third aspects, wherein the first information includes an image showing the state of the object.

[0192] Such a management system can manage images showing the state of an object.

[0193] Invention 5 is the management system of any one of Inventions 1 to 4, wherein the communication unit further receives second information indicating a response status for the state of the object, the management unit further manages the received second information in association with the first information, and the management system further includes an output unit that outputs at least one of the first information and the second information. The response information of the above-mentioned embodiment 2 is an example of the second information. Note that when the management system corresponds to the posting management system 130, the output unit (output unit 135) may output at least one of the first information and the second information to the information terminal 120 or to the structure management system 150.

[0194] Such a management system can manage not only the state of an object but also the status of a response to the state of the object, and can provide at least one of the state of the object and the status of the response.

[0195] A sixth aspect of the present invention is the management system of any one of the first to fifth aspects of the present invention, wherein the management unit manages the received first information in association with the installation position of the object and information about the user.

[0196] Such a management system can manage the state of an object in association with information about the user who input (posted) the state of the object.

[0197] A seventh aspect of the present invention is the management system of the sixth aspect, wherein the information about the user includes authority information about the user.

[0198] Such a management system can manage the state of an object in association with the authority information of the user who input (posted) the state of the object.

[0199] Invention 8 is a management method executed by a computer system, which includes a step S28 of receiving first information regarding the status of an object installed outdoors, which can be visually confirmed by a user of the information terminal 120, by communicating with the information terminal 120, and a step S29 of managing the received first information in association with the installation location of the object.

[0200] This management method makes it possible to manage the state of an object installed outdoors.

[0201] A ninth aspect of the present invention is a program for causing a computer system to execute the management method of the eighth aspect of the present invention.

[0202] Such a program enables the computer system to manage the status of objects installed outdoors.

[0203] Invention 10 is a display method executed by an information terminal 120 having a display unit 122, the display method including: a step S31 of displaying on the display unit 122 a first screen including a map of the surroundings of the information terminal 120, the first screen being for selecting an object installed outdoors; and a step S36 of displaying on the display unit 122 a second screen for inputting the state of the selected object, the state of the object being visually recognizable by the user of the information terminal 120. The selection GUI ( FIG. 21 ) of the second embodiment above is an example of the first screen, and the GUIs ( FIGS. 17 to 19 ) for inputting the state of the object are examples of the second screen. Note that the first screen and the second screen may be displayed on the same display screen.

[0204] Such a display method can assist the user in inputting the state of the object into the information terminal 120.

[0205] An eleventh aspect of the present invention is the display method of the tenth aspect, wherein the input state of the object is associated with states of the object previously input by the user of the information terminal 120 or other users.

[0206] Such a display method can assist the user in inputting the state of an object into the information terminal 120 in order to match the latest state of the object placed outdoors with the past state of the object.

[0207] A twelfth aspect of the present invention is the display method of the tenth or eleventh aspect of the present invention, wherein a plurality of options for selecting the state of the object are displayed on the second screen.

[0208] Such a display method can assist the user in inputting the state of the object into the information terminal 120 by displaying a plurality of options.

[0209] A thirteenth aspect of the present invention is the display method of any one of the tenth to twelfth aspects, wherein the plurality of options includes information indicating the degree of deterioration of the object.

[0210] Such a display method can assist the user in inputting the state of the object into the information terminal 120 by displaying a plurality of options indicating the degree of deterioration of the object.

[0211] Invention 14 is the display method of any one of Inventions 10 to 13, wherein an operation area for inputting the state of the object by characters is displayed on the second screen.

[0212] Such a display method can assist the user in freely inputting the state of the object into the information terminal 120 .

[0213] Invention 15 is the display method of any one of Inventions 10 to 14, further including the step of displaying on display unit 122 a third screen for photographing an image showing the state of the object using information terminal 120. The photographing GUI (FIG. 17) of the above-mentioned second embodiment is an example of the third screen.

[0214] Such a display method can assist the user in taking an image showing the state of an object using the information terminal 120.

[0215] Invention 16 is a display method according to any one of Inventions 10 to 15, which is capable of narrowing down the objects displayed on the first screen, or of displaying some of the objects displayed on the first screen in a different display mode from other objects.

[0216] Such a display method can simplify the selection of an object by the user.

[0217] A seventeenth aspect of the present invention is a program for causing an information terminal 120 to execute any one of the display methods of the tenth to sixteenth aspects.

[0218] According to such a program, the information terminal 120 can assist the user in inputting the state of the object into the information terminal 120 .

[0219] [Summary] -Problem- To provide a posting management system that can manage the status of objects installed outdoors.

[0220] -Solution- The posting management system 130 includes a communication unit 131 that receives first information regarding the status of an object installed outdoors, which can be visually confirmed by the user of the information terminal 120, by communicating with the information terminal 120, and a management unit 134 that manages the received first information by correlating it with the installation position of the object (FIG. 14).

[0221] (Embodiment 3) [Configuration] First, a description will be given of the configuration of an information processing system according to embodiment 3. Fig. 30 is a block diagram showing the functional configuration of the information processing system according to embodiment 3.

[0222] The information processing system 210 is a system that manages the status of structures installed outdoors. Examples of such structures include utility poles, streetlights, fences, and playground equipment. Management of outdoor structures is often performed in an analog manner, and maintenance may be performed irregularly. The information processing system 210 can manage the status of the structures, particularly the maintenance status, and can achieve planned maintenance of the structures. Planned maintenance of structures ultimately reduces maintenance costs and can prevent the occurrence of defects in the structures and the occurrence of unsafe conditions caused by defects in the structures.

[0223] The information processing system 210 includes multiple information terminals 220, an image management system 230, a diagnostic system 240, a structure management system 250, a local government system 260, and multiple construction management systems 270. The information processing system 210 includes one local government system 260 corresponding to one local government, but may include multiple local government systems 260 corresponding to multiple local governments. The information processing system 210 also includes multiple construction management systems corresponding to multiple maintenance businesses, but may also include one construction management system 270 corresponding to one maintenance business, and it is sufficient that the information processing system 210 includes at least one construction management system 270. These devices or systems will be described in detail below.

[0224] Each of the multiple information terminals 220 is an information terminal owned by a general user (such as a citizen). The general user (hereinafter also simply referred to as a user) uses the information terminal 220 to capture a still image showing the state of a structure, and transmits (posts) image information of the captured still image to the image management system 230. FIG. 31 is a diagram showing an example of a user capturing an image of a street light (an example of a structure) using the information terminal 220. FIGS. 32 and 33 are diagrams showing examples of a captured still image. The information terminal 220 is, for example, a portable information terminal such as a smartphone or a tablet terminal.

[0225] The still image may be an image of the entire structure or an image of a main part of the structure. Fig. 32 is a diagram showing an example of an image of the entire structure, and Fig. 33 is a diagram showing an example of an image of a main part of the structure. The still image in Fig. 33 shows the base of a street light, which can be considered a main part because it is a part that is prone to deterioration in street light management.

[0226] The image information also includes location information of the information terminal 220 when the still image included in the image information was captured (in other words, location information of a structure shown in the still image). The location information is measured, for example, by a location measurement unit (not shown) included in the information terminal 220. The location measurement unit is realized, for example, by a GNSS (Global Navigation Satellite System) module such as a GPS (Global Positioning System) module.

[0227] The image management system 230 is a system that receives image information from the information terminal 220, including a still image of a structure captured by the information terminal 220 and location information of the information terminal 220 at the time the still image was captured, and stores (manages) the received image information. The image management system 230 also transmits (provides) the image information to the structure management system 250 by communicating with the structure management system 250 via a wide area communication network. The image management system 230 is implemented by one or more server devices (cloud servers), but may also be implemented serverlessly.

[0228] The diagnostic system 240 receives image information from the structure management system 250 and diagnoses the degree of deterioration of the structure shown in the still image included in the received image information. The diagnostic system 240 is equipped with a machine learning model that can input, for example, a still image and output a score indicating the degree of deterioration of the structure shown in the still image, and diagnoses the degree of deterioration of the structure using the machine learning model. The diagnostic system 240 also transmits (provides) the diagnosis result of the degree of deterioration of the structure to the structure management system 250. The diagnostic system 240 is implemented by one or more server devices (cloud servers), but may also be implemented serverlessly.

[0229] The structure management system 250 receives image information from the image management system 230 and transmits the received image information to the diagnostic system 240, thereby receiving the diagnosis results of the structure's deterioration level from the diagnostic system 240. The structure management system 250 also stores the received image information and structure management information including the received diagnosis results of the deterioration level in a storage unit 253, and transmits (provides) the structure management information to the local government system 260 as needed. The structure management system 250 is realized by one or more server devices (cloud servers), but may also be realized serverlessly. The structure management system 250 includes a communication unit 251, an information processing unit 252, and a storage unit 253.

[0230] The communication unit 251 is a communication circuit (communication module) that enables the structure management system 250 to communicate with the image management system 230, the diagnosis system 240, the local government system 260, and the construction management system 270 via a wide area communication network. The communication unit 251 is, for example, a wired communication circuit that performs wired communication, but may also be a wireless communication circuit that performs wireless communication. There are no particular limitations on the communication standard for communication performed by the communication unit 251.

[0231] The information processing unit 252 processes information related to the management of the structure. The information processing unit 252 is realized, for example, by a microcomputer, but may also be realized by a processor. The information processing unit 252 includes, as functional components, an acquisition unit 254, a management unit 255, and an output unit 256. The functions of the acquisition unit 254, the management unit 255, and the output unit 256 are realized, for example, by the microcomputer or processor constituting the information processing unit 252 executing a computer program stored in the storage unit 253.

[0232] The storage unit 253 is a storage device that stores the computer programs executed by the information processing unit 252, structure management information, etc. The storage unit 253 is realized by, for example, a hard disk drive (HDD), but may also be realized by a semiconductor memory.

[0233] The municipal system 260 is a system used by municipal employees. By using the municipal system 260 to access the structure management system 250, the municipal employees can receive structure management information for structures managed by the municipality from the structure management system 250. The municipal system 260 is realized, for example, by a municipal terminal and a municipal server, but may also be realized as a standalone municipal terminal. The municipal terminal is, for example, a stationary information terminal such as a personal computer, but may also be a portable information terminal such as a smartphone or tablet terminal. The municipal system 260 includes an operation reception unit 261, a display unit 262, a communication unit 263, an information processing unit 264, and a memory unit 265.

[0234] The operation reception unit 261 receives operations from local government employees. The operation reception unit 261 is realized by, for example, a touch panel.

[0235] The display unit 262 displays an image (moving image or still image) and is realized by a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel.

[0236] The communication unit 263 is a communication circuit (communication module) that enables the local government system 260 to communicate with the structure management system 250 and the construction management system 270 via a wide area communication network. The communication unit 263 is, for example, a wired communication circuit that performs wired communication, but may also be a wireless communication circuit that performs wireless communication. There are no particular limitations on the communication standard for communication performed by the communication unit 263.

[0237] The information processing unit 264 performs information processing related to the visualization of the structure management information provided by the structure management system 250. The information processing unit 264 is realized, for example, by a microcomputer, but may also be realized by a processor. The information processing unit 264 includes an output unit 266 as a functional component. The function of the output unit 266 is realized, for example, by the microcomputer or processor constituting the information processing unit 264 executing a computer program stored in the storage unit 265.

[0238] The storage unit 265 is a storage device that stores computer programs and the like executed by the information processing unit 264. The storage unit 265 is realized by, for example, an HDD, but may also be realized by a semiconductor memory.

[0239] The construction management system 270 is a system used by a business operator who performs maintenance on structures, and manages the status and schedule of maintenance on structures. Maintenance here is used in a broad sense, and includes various types of maintenance work, such as inspection work, maintenance work, part or entire replacement work, and decommissioning work. The multiple construction management systems 270 included in the information processing system 210 are used, for example, by different businesses. The construction management system 270 is implemented by one or more server devices (cloud servers), but may also be implemented serverlessly. The construction management system 270 includes a communication unit 271, an information processing unit 272, and a storage unit 273.

[0240] The communication unit 271 is a communication circuit (communication module) that enables the construction management system 270 to communicate with the structure management system 250 and the local government system 260 via a wide area communication network. The communication unit 271 is, for example, a wired communication circuit that performs wired communication, but may also be a wireless communication circuit that performs wireless communication. There are no particular limitations on the communication standard for the communication performed by the communication unit 271.

[0241] The information processing unit 272 processes information related to management of the maintenance status of the structure. The information processing unit 272 is realized, for example, by a microcomputer, but may also be realized by a processor. The functions of the information processing unit 272 are realized, for example, by the microcomputer or processor constituting the information processing unit 272 executing a computer program stored in the storage unit 273.

[0242] The storage unit 273 is a storage device that stores computer programs and the like executed by the information processing unit 272. The storage unit 273 is realized by, for example, an HDD, but may also be realized by a semiconductor memory.

[0243] [Operation Example] Next, an operation example of the information processing system 210 will be described. First, the first half of the operation example will be described with reference to Fig. 34. Fig. 34 is a sequence diagram of the first half of the operation example of the information processing system 210.

[0244] A local government employee or the like performs a predetermined registration operation to register a structure that is under the management of the local government, and the operation reception unit 261 of the local government system 260 receives the registration operation (S41). The information processing unit 264 generates registration information for the structure to be managed and transmits the generated registration information to the structure management system 250 using the communication unit 263 (S42). The registration information includes identification information (e.g., a management number or name, etc.) of the structure to be registered, its type (e.g., a street light, a utility pole, or playground equipment, etc.), the name of the manufacturer, the dimensions of the structure (e.g., the overall height of the structure or the height of a specific part of the structure, etc.), the installation location, the installation status (e.g., whether the structure is installed on the road surface), and the installation date. The information included in the registration information is an example of third information related to the attributes of the structure.

[0245] The communication unit 251 of the structure management system 250 receives the registration information. The information processing unit 252 can identify the structure to be registered based on the received registration information. Then, the information processing unit 252 communicates with the image management system 230 using the communication unit 251, and transmits an image provision request to the image management system 230 to obtain image information of the structure to be managed (S43). The image provision request includes information indicating the installation location of the structure to be registered. The image information is an example of first information related to the state of the structure.

[0246] The image management system 230 receives the image provision request and identifies image information of the structure to be managed by comparing information indicating the installation location included in the received image provision request with position information of the structure shown in the still image included in the image information managed by the image management system 230. The image management system 230 transmits the identified image information to the structure management system 250 (S44). Note that as a result of the comparison, there may be cases where image information of the structure to be managed does not exist.

[0247] The communication unit 251 of the structure management system 250 receives the image information. The information processing unit 252 communicates with the diagnosis system 240 using the communication unit 251 to send a diagnosis request including image information of the structure to be managed to the diagnosis system 240 (S45), and receives, in response, from the diagnosis system 240, a diagnosis result of the degree of deterioration of the structure to be managed (S46). In other words, the diagnosis result of the degree of deterioration is deterioration level information of the structure, and is an example of first information related to the state of the structure.

[0248] The acquisition unit 254 of the structure management system 250 acquires the registration information received in step S41, the image information received in step S44, and the deterioration level diagnosis result received in step S46 (S47). The management unit 255 generates structure management information by associating the image information and the deterioration level diagnosis result with the acquired registration information, and stores the generated structure management information in the storage unit 253 (S48).

[0249] The management unit 255 may extract structures that require maintenance based on the results of the deterioration level diagnosis, and include information indicating the necessity of maintenance in the structure management information. The management unit 255 can determine whether or not maintenance is necessary by, for example, comparing the results of the deterioration level diagnosis (score) with a threshold value. This determination is made by the diagnosis system 240, and the determination result of whether or not maintenance is necessary may be included in the results of the deterioration level diagnosis.

[0250] Thereafter, the local government employee or the like performs a predetermined viewing operation to view at least a part of the structure management information, and the operation reception unit 261 of the local government system 260 accepts the viewing operation (S49). The information processing unit 264 uses the communication unit 263 to send a viewing request to the structure management system 250 (S50).

[0251] The communication unit 251 of the structure management system 250 receives the viewing request. The output unit 256 transmits (outputs) the structure management information specified in the viewing request (more specifically, information for visualizing the structure management information) to the local government system 260 using the communication unit 251 (S51).

[0252] The communication unit 263 of the local government system 260 receives the structure management information. The output unit 266 displays the received structure management information on the display unit 262 (S52). In other words, the output unit 266 outputs the structure management information as display information. Figure 35 is a diagram showing an example of a display screen for the structure management information.

[0253] Next, the second half of the operation example will be described with reference to Fig. 36. Fig. 36 is a sequence diagram of the second half of the operation example of the information processing system 210.

[0254] A local government employee or the like who has viewed the display screen in Figure 35 requests a maintenance company to perform maintenance on a structure that is thought to require maintenance. For example, the local government employee or the like performs a predetermined request operation to request maintenance, and the operation reception unit 261 of the local government system 260 receives the request operation (S53). The information processing unit 264 generates request information for the structure that is the target of maintenance, and transmits the generated request information to the construction management system 270 using the communication unit 263 (S54). The request information includes identification information (e.g., a management number or name, etc.) of the structure that is the target of maintenance, the details of the maintenance, etc.

[0255] The communication unit 271 of the construction management system 270 receives the request information. The information processing unit 272 generates maintenance management information that associates the received request information with the scheduled date of maintenance (construction), etc., and stores the information in the storage unit 273 (S55). While the maintenance is being carried out, the maintenance management information includes information indicating the content of the maintenance, whether or not maintenance is scheduled to be carried out, and the implementation status (a status indicating progress, the scheduled date of maintenance, etc.), and after the maintenance has been carried out, the maintenance management information includes the date of maintenance implementation. The maintenance management information is an example of second information related to the maintenance of a structure.

[0256] The maintenance management information may also include still images of the structure before maintenance, still images of the structure during maintenance, and still images of the structure after maintenance. The still images here do not mean still images provided by the image management system 230, but still images taken by the maintenance contractor. While still images are provided irregularly by the image management system 230, the still images included in the maintenance management information are taken in accordance with the timing of maintenance and added to the maintenance management information.

[0257] Note that a still image of the structure before maintenance, included in the image information provided by the image management system 230, is an example of information included in the first information, relating to a first state of the structure before maintenance is performed. A still image of the structure after maintenance, included in the image information provided by the image management system 230, is an example of information included in the first information, relating to a second state of the structure after maintenance is performed.

[0258] A still image of the structure before maintenance included in the maintenance management information is an example of information included in the second information about a first state of the structure before maintenance is performed. A still image of the structure after maintenance included in the maintenance management information is an example of information included in the second information about a second state of the structure after maintenance is performed. A still image of the structure during maintenance included in the maintenance management information is an example of information included in the second information about a third state of the structure while maintenance is being performed.

[0259] Thereafter, the information processing unit 252 of the structure management system 250 communicates with the construction management system 270 using the communication unit 251 to send an information provision request to the construction management system 270 to obtain maintenance management information for the structure to be managed (S56). The information provision request includes identification information for the structure to be managed.

[0260] The communication unit 271 of the construction management system 270 receives the information provision request. The information processing unit 272 identifies the maintenance management information for the structure to be managed by comparing the identification information for the structure included in the information provision request with the identification information for the structure included in the maintenance management information, and assigns information about the name of the business operator performing the maintenance to the identified maintenance management information. The information processing unit 272 transmits (outputs) the maintenance management information to which the information about the business operator name has been assigned to the structure management system 250 (S57). Note that as a result of the comparison, there may be cases where maintenance management information for the structure to be managed does not exist.

[0261] The communication unit 251 of the structure management system 250 receives the maintenance management information. The acquisition unit 254 acquires the maintenance management information received in step S57 (S58). The management unit 255 associates the maintenance management information with the structure management information and stores the information in the storage unit 253 (S59). In other words, the management unit 255 updates the structure management information.

[0262] After the structure management information has been updated in this manner, if processing similar to steps S50 to S52 is performed, the output unit 256 can transmit (output) the structure management information including the maintenance management information to the local government system 260. As a result, the maintenance management information is displayed as part of the structure management information on the display unit 262 of the local government system 260. Figure 37 is a diagram showing an example of a display screen for structure management information including maintenance management information.

[0263] The image provision request in step S43 is sent periodically or in response to instructions from the local government system 260 (such as a local government employee), and the image information included in the structure management information is added or updated as appropriate, and the deterioration level is updated as appropriate. The image information may include a still image showing the structure before maintenance is performed, or may include a still image showing the structure after maintenance is performed. In other words, the image information can be said to include at least one of information regarding a first state of the structure before maintenance is performed and information regarding a second state of the structure after maintenance is performed. In other words, the structure management information includes historical information about the state of the structure.

[0264] Similarly, the information request in step S56 is sent periodically or in response to an instruction from the local government system 260 (such as a local government employee), and the maintenance management information included in the structure management information is added or updated as appropriate. The maintenance management information may include information about a first state of the structure before maintenance is performed, information about a second state of the structure after maintenance is performed, or information about a third state of the structure while maintenance is being performed. In other words, the maintenance management information includes at least one of information about the first state, information about the second state, and information about the third state. In other words, the structure management information includes maintenance history information for the structure.

[0265] As described above, the structure management system 250 includes an acquisition unit 254 that acquires first information on the state of the structure and second information on maintenance of the structure, a management unit 255 that generates structure management information in which the acquired first information and second information are associated with each other, and an output unit 256 that outputs the generated structure management information. The first information is, for example, image information or a diagnosis result of the degree of deterioration, and the second information is, for example, maintenance management information.

[0266] Such an information processing system 210 can support structure maintenance by outputting structure management information that associates the state of a structure with its maintenance status. If the structure state history information and structure maintenance history information contained in the structure management information are visualized, it becomes easier for local government officials and others to plan structure maintenance. Planned structure maintenance can ultimately reduce maintenance costs and prevent the occurrence of structure malfunctions and unsafe conditions caused by structure malfunctions.

[0267] [Modification 1 of System Configuration] A description will be given of Modification 1 of the functional configuration of the information processing system 210. Fig. 38 is a block diagram showing the functional configuration of the information processing system according to Modification 1.

[0268] For example, if the structure management system 250 and the diagnostic system 240 are systems used by the same operator, a configuration is conceivable in which the diagnostic system 240 acquires image information from the image management system 230 and outputs the acquired image information and the deterioration level diagnosis results to the structure management system 250. The information processing system 210a shown in Figure 38 corresponds to such a configuration, with the diagnostic system 240 located between the image management system 230 and the structure management system 250.

[0269] The information processing system 210a has the advantage of being able to reduce the amount of communication required to obtain the results of a deterioration level diagnosis, compared to the information processing system 210. In the information processing system 210a, the image management system 230 may provide image information to both the diagnosis system 240 and the structure management system 250, and in this case, the process of transmitting image information from the diagnosis system 240 to the structure management system 250 can be omitted.

[0270] [Modification 2 of System Configuration] A description will be given of Modification 2 of the functional configuration of the information processing system 210. Fig. 39 is a block diagram showing the functional configuration of the information processing system according to Modification 2.

[0271] For example, there may be a case where the business operator using the structure management system 250 and the business operator using the construction management system 270 are not affiliated, and the structure management system 250 is unable to directly obtain maintenance management information from the construction management system 270. The information processing system 210b shown in Figure 39 has a functional configuration that takes such a case into account, and the dashed line connecting the structure management system 250 and the construction management system 270 has been omitted.

[0272] In the information processing system 210b, the maintenance management information, or information substantially identical thereto, is provided from the local government system 260 to the structure management system 250. For example, the maintenance management information stored in the construction management system 270 is provided to the structure management system 250 via the local government system 260, but it is also possible for a local government employee or the like to manually input information substantially identical to the maintenance management information stored in the construction management system 270 into the local government system 260, and the manually inputted substantially identical information be provided to the structure management system 250.

[0273] Furthermore, if one of the local government system 260 and the construction management system 270 can output information (data) to the other in a format that can be read by the other, then the other of the local government system 260 and the construction management system 270 can provide the other with information that is substantially identical to the information managed by that system.

[0274] [Modifications Related to Position Information of Structures] In the above third embodiment, it has been described that the position information of structures shown in still images is measured by a position measurement unit included in the information terminal 220. However, it is not essential that the position information of structures shown in still images is measured by a position measurement unit.

[0275] For example, a map may be displayed on the display unit of the information terminal 220, and the user may plot a point on the displayed map to specify the installation location of the structure.

[0276] Furthermore, for example, the image management system 230 may provide a list of installation positions (coordinates) of pre-registered structures to the information terminal 220 running the dedicated app, and a GUI for selecting an installation position based on the list information may be displayed on the display unit 22 of the information terminal 220. The selection GUI may be, for example, a GUI in which an icon is displayed at the installation position of the structure on a map. By selecting an icon, the user can input the installation position of the structure shown in the captured still image. In this case, since the installation positions of the structures are managed in the image management system 230, the information terminal 220 may transmit identification information of the selected icon to the image management system 230 instead of transmitting position information.

[0277] [Other Modifications] In the third embodiment, the diagnostic system 240 diagnosed the current level of deterioration of the structure shown in the still image, but the diagnostic system 240 may also predict the future level of deterioration. For example, if time-series data of the diagnostic results of the deterioration level is available, the future level of deterioration may be predicted by calculating an approximate straight line or an approximate straight line based on the time-series data of the deterioration level.

[0278] In this case, the acquisition unit 254 of the structure management system 250 can acquire a predicted deterioration level, such as the deterioration level n months from now, and the management unit 255 can include the predicted deterioration level in the structure management information. Note that the deterioration level prediction can also be performed by the structure management system 250.

[0279] Furthermore, in the above-described third embodiment, it is assumed that a login process is performed when a local government employee or the like accesses the structure management system 250 using the local government system 260. In such a case, the output unit 256 of the structure management system 250 may manage the login ID and the authority information of the employee or the like who has the login ID, and change the scope of provision of the structure management information depending on the authority of the employee or the like.

[0280] For example, if the output unit 256 determines that an employee or the like is a general user based on the login ID, it may output only a portion of the structure management information, and if the output unit 256 determines that an employee or the like is a management user based on the login ID, it may output all of the structure management information. In other words, the output unit 256 may output structure management information according to the authority of the user to whom the information is output. Note that the authority information associated with the login ID is set when the login ID is issued, etc.

[0281] In the third embodiment, the structure management system 250 manages structures installed outdoors, but it may also manage equipment installed outdoors. The structure management system 250 may manage at least one of structures and equipment.

[0282] [Effects, etc.] Hereinafter, examples of inventions obtained from the disclosure of this specification will be described, and effects, etc. obtained from the exemplified inventions will be explained.

[0283] Invention 1 is a management system including an acquisition unit 254 that acquires first information on the state of an object including at least one of a structure and equipment, and second information on maintenance of the object, a management unit 255 that generates management information of the object in which the acquired first information and second information are associated, and an output unit 256 that outputs the generated management information. The structure management information of the above-mentioned embodiment 3 is an example of management information of an object, and the structure management system of the above-mentioned embodiment 3 is an example of a management system.

[0284] Such a management system can output information relating to the maintenance of an object.

[0285] Invention 2 is a management system of Invention 1, in which the management unit 255 extracts objects requiring maintenance based on the acquired first information, and the management information includes information regarding the objects requiring maintenance.

[0286] Such a management system can prompt maintenance of an object by outputting information about the object requiring maintenance.

[0287] Invention 3 is a management system of Invention 1 or 2, in which the first information includes at least one of information regarding a first state of the object before maintenance is performed and information regarding a second state of the object after maintenance is performed.

[0288] Such a management system can provide information about the condition of an object before or after maintenance.

[0289] A fourth aspect of the present invention is the management system of the third aspect, wherein the first information includes information based on an image of the object.

[0290] Such a management system can provide information based on images in which objects appear.

[0291] Invention 5 is the management system of Invention 4, wherein the image of the object includes an image of the entire object and an image of a main part of the object.

[0292] Such a management system can provide an overall image of the object as well as an image of the main part of the object.

[0293] A sixth aspect of the present invention is the management system of any one of the first to fifth aspects of the present invention, wherein the second information includes information relating to a prediction of the state of the object.

[0294] Such a management system can provide information about the future state of an object.

[0295] A seventh aspect of the present invention is the management system according to any one of the first to sixth aspects of the present invention, wherein the second information includes information relating to a schedule for carrying out maintenance on the object.

[0296] Such a management system can provide information regarding scheduled maintenance for an object.

[0297] An eighth aspect of the present invention is the management system of the seventh aspect, wherein the second information includes information regarding whether or not construction work is planned to be carried out on the object.

[0298] Such a management system can provide information on whether construction work is scheduled for an object.

[0299] Invention 9 is a management system according to any one of Inventions 1 to 8, wherein the second information includes at least one of information relating to a first state of the object before maintenance is performed, information relating to a second state of the object after maintenance is performed, and information relating to a third state of the object while maintenance is being performed.

[0300] Such a management system can provide information on the condition of the object as the maintenance progresses.

[0301] A tenth aspect of the present invention is the management system according to any one of the first to ninth aspects, wherein the second information includes information relating to the name of a company that performs maintenance.

[0302] Such a management system can provide information regarding the name of the company that will carry out the maintenance.

[0303] An eleventh aspect of the present invention is the management system according to any one of the first to tenth aspects of the present invention, wherein the acquisition unit 254 further acquires third information relating to attributes of the object.

[0304] Such management systems can provide information about the attributes of objects.

[0305] A twelfth aspect of the present invention is the management system of the eleventh aspect, wherein the third information includes information relating to the type of the object.

[0306] Such a management system can provide information about the type of object.

[0307] A thirteenth aspect of the present invention is the management system of the eleventh or twelfth aspect, wherein the third information includes information relating to a manufacturer of the object.

[0308] Such a management system can provide information about the manufacturer of an object.

[0309] A fourteenth aspect of the present invention is the management system of any one of the eleventh to thirteenth aspects, wherein the third information includes information relating to the height of at least a part of the object.

[0310] Such a management system may provide information about the height of at least part of an object.

[0311] A fifteenth aspect of the present invention is the management system according to any one of the eleventh to fourteenth aspects of the present invention, wherein the third information includes information relating to the installation status of the object.

[0312] Such a management system can provide information about the installation status of the object.

[0313] A sixteenth aspect of the present invention is the management system according to any one of the first to fifteenth aspects of the present invention, wherein the output unit 256 outputs management information according to the authority of the user of the output destination.

[0314] Such a management system can change the content of the management information to be provided depending on the user authority of the output destination.

[0315] The seventeenth aspect of the present invention is a control system including a communication unit (263) that communicates with a management system that outputs management information in which first information on the state of an object including at least one of a structure and equipment and second information on maintenance of the object are associated with each other, and an output unit (266) that outputs the management information as display information. The local government system (260) of the third embodiment is an example of a control system.

[0316] Such a control system can display management information (information related to maintenance) of an object.

[0317] Invention 18 is a construction management system 270 having a communication unit 271 that transmits second information to a management system that outputs management information in which first information regarding the state of an object including at least one of a structure and equipment is associated with second information regarding the maintenance of the object.

[0318] Such a construction management system 270 can provide information regarding the maintenance of an object to the management system.

[0319] Invention 19 is an information processing method executed by a computer system such as a management system, and includes an acquisition step of acquiring first information regarding the state of an object including at least one of a structure and equipment, and second information regarding maintenance of the object, a generation step of generating management information for the object in which the acquired first information and second information are associated, and an output step of outputting the generated management information.

[0320] Such an information processing method can output information relating to the maintenance of an object.

[0321] Invention 20 is an information processing method executed by a computer system such as a control system, and includes a communication step of communicating with a management system that outputs management information in which first information regarding the state of an object including at least one of a structure and equipment and second information regarding maintenance of the object are associated, and an output step of outputting the management information as display information.

[0322] Such an information processing method can display management information (information related to maintenance) of an object.

[0323] Invention 21 is an information processing method executed by a computer system, which includes a step of transmitting second information to a management system that outputs management information in which first information regarding the state of an object including at least one of a structure and equipment is associated with second information regarding maintenance of the object.

[0324] Such an information processing method can provide information regarding the maintenance of an object to a management system.

[0325] A twenty-second aspect of the present invention is a program for causing a computer system to execute any one of the information processing methods of the nineteenth to twenty-first aspects.

[0326] According to such a program, the management system can output information regarding the maintenance of the object, the control system can display management information (information regarding maintenance) of the object, and the construction management system 270 can provide information regarding the maintenance of the object to the management system.

[0327] Invention 23 is a management system comprising an acquisition unit 254 that acquires first information regarding the state of an object including at least one of a structure and equipment, a management unit 255 that generates management information for the object including the acquired first information, and an output unit 256 that outputs the generated management information.

[0328] In this way, the structure management information only needs to include at least the first information, and does not necessarily need to include the second information. Inventions obtained from the disclosure of this specification include inventions of methods or programs having substantially the same technical features as invention 23.

[0329] [Summary] -Problem- To provide a management system that can output information related to the maintenance of an object.

[0330] -Solution- The structure management system 250 includes an acquisition unit 254 that acquires first information regarding the state of the structure and second information regarding the maintenance of the structure, a management unit 255 that generates management information for the object in which the acquired first information and second information are associated, and an output unit 256 that outputs the generated management information (Figure 30).

[0331] (Other Embodiments) Although the first to third embodiments have been described above, the present invention is not limited to the first to third embodiments.

[0332] For example, in the above-described first to third embodiments, a system such as an information processing system is realized by a plurality of devices, but it may also be realized by a single device. In this manner, the system in this specification may be configured by a single device or by a plurality of devices. When a system is realized by a plurality of devices, the components of the system may be distributed among the plurality of devices in any manner.

[0333] In addition, in the first to third embodiments, the processing performed by a specific processing unit may be performed by another processing unit. Furthermore, the order of multiple processing operations may be changed, or multiple processing operations may be performed in parallel.

[0334] Furthermore, in the above-described first to third embodiments, each component may be realized by executing a software program suitable for that component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0335] Furthermore, each component may be realized by hardware. Each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0336] Furthermore, the general or specific aspects of the present invention may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0337] For example, the present invention may be realized as the information terminal of the first embodiment. The present invention may also be realized as a display method executed by the information terminal. The present invention may also be realized as a program for causing the information terminal to execute the display method, or as a computer-readable non-transitory recording medium on which such a program is recorded.

[0338] For example, the present invention may be realized as a computer system such as the information processing system, post management system, or structure management system of the second embodiment. The present invention may also be realized as a management method executed by a computer system. The present invention may also be realized as a program for causing a computer system to execute the management method, or as a computer-readable non-transitory recording medium on which such a program is recorded.

[0339] For example, the present invention may be realized as a computer system such as the information processing system of the third embodiment, a structure management system, a local government system, or a construction management system. The present invention may also be realized as an information processing method executed by a computer system. The present invention may also be realized as a program for causing a computer system to execute the information processing method, or as a computer-readable non-transitory recording medium on which such a program is recorded.

[0340] In addition, the present invention also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art would think of, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope of the present invention.

[0341] 10, 110, 110a, 110b, 110c, 110d, 110e, 210, 210a, 210b Information processing system 20, 120, 120a, 120b, 220 Information terminal 21, 121, 261 Operation acceptance unit 22, 122, 262 Display unit 22a, 22b Dashed frame (guide) 23, 123, 132, 152, 252, 264, 272 Information processing unit 24, 124, 133, 153, 253, 265, 273 Storage unit 25, 125, 131, 151, 251, 263, 271 Communication unit 26, 126 Position measurement unit 27, 127 Photography unit 30, 170, 230 Image management system 40 Diagnostic system 50, 150, 250 Structure management system 60, 160 Municipal terminal 70 Street light 71 Support pole 72 Lighting fixture 73 Cover 130 Post management system 134, 154 Management unit 135, 155 Output unit 140, 240 Diagnostic system 254 Acquisition unit 255 Management unit 256, 266 Output unit 260 Municipal system 270 Construction management system

Claims

1. A display method executed by an information terminal including a photographing unit and a display unit, during the operation of a first mode for photographing a lower end portion in the vertical direction of a long structure deteriorated by rust with the photographing unit, a first display step of displaying, on the display unit, an image being photographed by the photographing unit and a guide for including the lower end portion in a photographing range; and an output step of outputting a still image photographed by the photographing unit when a predetermined trigger condition is satisfied.

2. The display method according to claim 1, further including a second display step of displaying, on the display unit, an image being photographed by the photographing unit and a guide for including the whole of the structure in a photographing range during the operation of a second mode for photographing the whole of the structure with the photographing unit.

3. The structure is a lighting device including a support column and a lamp, and the display method further includes a third display step of displaying, on the display unit, an image being photographed by the photographing unit and a guide for including the lamp in a photographing range during the operation of a third mode for photographing the lamp with the photographing unit.

4. The structure is a lighting device including a support column and a lamp, and the display method further includes a fourth display step of displaying, on the display unit, an image being photographed by the photographing unit and a guide for including identification information regarding the lamp in a photographing range during the operation of a fourth mode for photographing the identification information regarding the lamp with the photographing unit.

5. The structure is a lighting device including a support column, a lamp, and a cover provided on the support column, and the display method further includes a fifth display step of displaying, on the display unit, an image being photographed by the photographing unit and a guide for fitting the cover in a photographing range in a fifth mode for photographing the cover with the photographing unit.

6. The display method according to claim 1, wherein information regarding a deterioration state of the structure is displayed on the display unit in the first display step.

7. The display method according to claim 6, wherein the deterioration state of the structure includes a rust state in the structure.

8. The display method according to claim 7, wherein the rust state includes a range where the rust has occurred in the structure.

9. The display method according to claim 1, wherein the guide displayed in the first display step includes a guide for designating the position of the lower end portion of the structure in the image.

10. The display method according to claim 1, further comprising a step of performing at least one of continuing the operation in the first mode and stopping the output step when the lower end portion of the structure shown in the still image does not satisfy a predetermined requirement during the operation in the first mode.

11. The display method according to claim 1, further comprising a step of displaying, on the display unit, a screen for inputting a reason why the boundary portion between the lower end portion of the structure and the ground surface on which the structure is installed cannot be photographed when the boundary portion cannot be photographed.

12. In the first display step, the image being photographed by the photographing unit is displayed with the long side direction of the display unit of the information terminal as the vertical direction and the short side direction as the horizontal direction. The display method according to claim 1.

13. During the operation in the first mode, the optical zoom of the photographing unit is permitted, and the digital zoom of the photographing unit is not permitted. The display method according to claim 1.

14. The predetermined trigger condition includes a condition that a predetermined operation is performed by the user. The display method according to claim 1.

15. A program for causing the information terminal to execute the display method according to any one of claims 1 to 14.

Images