Inspection support methods and programs for large-scale outdoor facilities

Abstract

This invention provides an inspection support method and program for large-scale outdoor facilities that facilitates inspection work within the facility and the recording of inspection results, as well as easy viewing of those results. [Solution] Server 4 generates response necessity information based on on-site image information transmitted from the first terminal 2, stores the on-site image information, location information indicating the shooting location of the on-site image, and the response necessity information in association, and the second terminal 3 displays map AT and selectable marks M at positions corresponding to the shooting location of the on-site image P, depending on the response necessity, based on initial display information transmitted from server 4, and when a mark M is selected, displays the on-site image P corresponding to the selected mark M on map A based on detailed display information transmitted from server 4.

Description

Technical Field

[0001] The present invention relates to an inspection support method and program for large-scale outdoor facilities.

Background Art

[0002] In various large-scale outdoor facilities such as factories and water purification plants, it is extremely important in the operation and maintenance of the facility to inspect the inside of the facility and grasp the situation inside the facility, such as locations that require repair.

[0003] To facilitate such inspections inside the facility, various systems for assisting inspectors have been developed. For example, Patent Document 1 discloses an inspection support system using an unmanned moving body such as a drone capable of remote operation.

[0004] In the inspection support system disclosed in this Patent Document 1, an inspector as an operator operates an unmanned moving body to perform imaging at the site, and data captured by a virtual reality device such as a 3D VR camera worn by the inspector can be displayed so as to be visually recognizable simultaneously with information such as an inspection history regarding the photographed location.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] By the way, when the facility to be inspected is a large-scale outdoor facility such as a steel mill or an oil refinery, the number of locations that require inspection becomes extremely large and covers a wide area, so the amount of work required for the inspector to perform the inspection has become extremely large.

[0007] Furthermore, while it is necessary to record the inspection results for each area within the facility that has been inspected, this task of recording inspection results is enormous and requires a great deal of effort from the workers.

[0008] Furthermore, when using the inspection results, such as when formulating inspection work plans based on the recorded results, the process of finding and viewing desired inspection results, such as those of high importance (i.e., those requiring priority attention), from among the vast amount of results, as well as locating those specific sections, was extremely cumbersome.

[0009] Therefore, the present invention has been made in view of the above-mentioned problems, and aims to provide an inspection support method and program for large-scale outdoor facilities that facilitates inspection work and recording of inspection results within large-scale outdoor facilities, as well as facilitates viewing of inspection results. [Means for solving the problem]

[0010] To solve the above problems, the present invention provides a first terminal that takes a field image at a site, generates image information of the field image and location information indicating the shooting location, and transmits it to a server; the server determines the degree of need for action at the site based on the image information and generates action need information; the server stores the image information, location information, and action need information in association with map information of a map including the shooting location; the second terminal generates initial request information requesting initial display information for a predetermined site and transmits it to the server; and the server stores the map information and location information according to the initial request information. Steps include: extracting the report and the response necessity information, generating initial display information that displays a user-selectable mark on the map at the shooting location on the map, along with the map, and transmitting it to the second terminal; the second terminal displaying information on the display means based on the initial display information; when the user makes the selection operation on the mark displayed on the second terminal, generating detailed request information requesting detailed display information for the selected mark and transmitting it to the server; and the server responding to the detailed request information, The selected mark is enlarged to display an enlarged map of the area around the mark, Extract the image information associated with the position information corresponding to the selected mark, expansion The inspection support method for large-scale outdoor facilities is characterized by comprising the steps of: generating detailed display information that overlays the site image onto a map and transmitting it to the second terminal; and the second terminal displaying information on the display means based on the detailed display information.

[0011] In the above-described inspection support method for large-scale outdoor facilities, it is preferable that the first terminal is capable of modifying the response necessity information generated by the server, and when such modification is performed, it generates modification operation information and sends it to the server, and the server updates and saves the response necessity information based on the modification operation information.

[0012] Furthermore, the present invention includes the steps of: a first terminal taking a field image at a site, generating response necessity information regarding the level of response required at the site based on the image information of the field image, location information indicating the shooting location, and user setting operations, and transmitting it to a server; the server associating the image information, location information, and response necessity information with map information of a map including the shooting location and storing it; a second terminal generating initial request information requesting initial display information regarding a predetermined site and transmitting it to the server; and the server, in response to the initial request information, generating the map information, location information, and response necessity information The process includes: extracting necessity information, generating initial display information that displays a user-selectable mark on the map at the shooting location, along with the map, according to the corresponding necessity information, and transmitting it to the second terminal; the second terminal displaying information on a display means based on the initial display information; when the user makes the selection operation on the mark displayed on the second terminal, the second terminal generates detailed request information requesting detailed display information for the selected mark and transmitting it to the server; and the server responding to the detailed request information, The selected mark is enlarged to display an enlarged map of the area around the mark, Extract the image information associated with the position information corresponding to the selected mark, expansion The inspection support method for large-scale outdoor facilities is characterized by comprising the steps of: generating detailed display information that overlays the site image onto a map and transmitting it to the second terminal; and the second terminal displaying information on the display means based on the detailed display information.

[0013] In the above-described inspection support method for large-scale outdoor facilities, it is preferable that when the server receives the detailed request information from the second terminal, it displays an enlarged map of the map around the selected mark, and generates detailed display information that displays the site image on the enlarged map, and transmits it to the second terminal.

[0014] In the above-described inspection support method for large-scale outdoor facilities, the first terminal further generates shooting date and time information indicating the date and time the on-site image was taken and transmits it to the server together with the image information and the location information, the server stores the shooting date and time information in association with the map information, the image information, the location information and the response necessity information, the second terminal can generate chart request information requesting a chart based on a predetermined response necessity and shooting period and transmit it to the server, the server extracts the predetermined response necessity information and the shooting date and time information within the shooting period based on the chart request information to generate chart display information including the display mode of the chart and transmits it to the second terminal, and the second terminal preferably displays the chart on the display means based on the chart display information.

[0015] Furthermore, the present invention includes the steps of: storing image information of a site image, which is an image of the site transmitted from a first terminal, and location information indicating the shooting location, in association with map information of a map including the shooting location; determining the degree of need for response at the site based on the image information and generating response need information; storing the response need information in association with the map information, the image information, and the location information; extracting the map information, the location information, and the response need information in accordance with initial request information concerning a predetermined site transmitted from a second terminal, generating initial display information that displays a mark on the shooting location on the map, which differs according to the response need information and can be selected by the user, along with the map, and transmitting it to the second terminal; and receiving detailed request information from the second terminal requesting detailed display information generated in accordance with the selection operation by the user, in accordance with the detailed request information. The selected mark is enlarged to display an enlarged map of the area around the mark, Extract the image information associated with the position information corresponding to the selected mark, expansion The process of generating detailed display information that overlays the aforementioned site image onto a map and transmitting it to the second terminal, It is characterized by being a program that causes a computer to execute something.

[0016] Furthermore, the present invention includes the steps of: storing image information of a site image, which is an image of the site transmitted from a first terminal, location information indicating the shooting location, and response necessity information indicating the degree of need for response at the site, in association with map information of a map including the shooting location; extracting the map information, location information, and response necessity information in response to initial request information concerning a predetermined site transmitted from a second terminal, generating initial display information that displays a user-selectable mark at the shooting location on the map, along with the map, according to the response necessity information, and transmitting it to the second terminal; and receiving detailed request information from the second terminal requesting detailed display information generated in response to the user's selection operation, in accordance with the detailed request information. The selected mark is enlarged to display an enlarged map of the area around the mark, Extract the image information associated with the position information corresponding to the selected mark, expansion The program is characterized by causing a computer to perform the steps of generating detailed display information that overlays the aforementioned on-site image onto a map and transmitting it to the second terminal.

[0017] In the above program, it is preferable that the detailed display information causes the second terminal to display an enlarged map of the map around the selected mark, and also displays the on-site image on the enlarged map. [Effects of the Invention]

[0018] According to the present invention, which has the configuration described above, inspection work within the facility and recording of inspection results are made easier, and viewing of the inspection results is also made easier. In particular, the location of areas requiring attention and the details of the necessary actions can be viewed at a glance on a map as a result of the inspection, making it possible to create an efficient response plan. [Brief explanation of the drawing]

[0019] [Figure 1] This is a schematic diagram showing an inspection support system in which an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention is implemented. [Figure 2](A) is a front view and (B) is a rear view schematically showing a first terminal used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. [Figure 3] It is a block diagram of a first terminal used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. [Figure 4] It is a functional block diagram of a control unit provided in a first terminal used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. [Figure 5] It is a perspective view schematically showing a second terminal used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. [Figure 6] It is a block diagram of a second terminal used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. [Figure 7] It is a functional block diagram of a control unit provided in a second terminal used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. [Figure 8] It is a block diagram of a server used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. [Figure 9] It is a functional block diagram of a control unit provided in a server used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. [Figure 10] It is a schematic diagram showing a map of a facility to which an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention is applied. [Figure 11] It is a schematic diagram showing a site image of a site to which an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention is applied. [Figure 12] It is a schematic diagram showing a state in which a map and a mark are displayed on a display of a second terminal used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. [Figure 13] It is a schematic diagram showing a state in which a selection operation is performed on a mark displayed on the display of FIG. 12 and a site image is displayed on an enlarged map. [Figure 14]This flowchart shows the process in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. [Figure 15] This is a schematic diagram showing a chart created in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. [Modes for carrying out the invention]

[0020] Hereinafter, with reference to the drawings, an inspection support system that implements an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention will be described.

[0021] Figure 1 is a schematic diagram showing an inspection support system 1 in which an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention is implemented. As shown in Figure 1, the inspection support system 1 is configured to include a plurality of first terminals 2 (2A, 2B, ..., 2N), a second terminal 3, and a server 4 connected to the first terminals 2 and the second terminals 3 via a network N.

[0022] The first terminal 2 generates and transmits on-site image information, location information, shooting date and time information, and various operation information to the server 4, and receives various information from the server 4 and displays (draws) based on that information. The second terminal 3 generates and transmits various operation information to the server 4, and receives various information from the server 4 and displays (draws) based on that information. Other information processing related to the inspection support system 1 (such as the generation of information for drawing by the first terminal 2 and the second terminal 3) is performed by the server 4. Various drawings by the first terminal 2 and the second terminal 3 based on various information from the server 4 may be performed via software or applications dedicated to the inspection support system 1 installed on the first terminal 2 and the second terminal 3, or they may be performed via a known internet browser pre-installed on the first terminal 2 and the second terminal 3 without using software or applications dedicated to the inspection system 1.

[0023] Figure 2 schematically shows a first terminal 2 used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention, where (A) is a front view and (B) is a rear view. In this embodiment, the first terminal 2 is assumed to be a smartphone and is a terminal that can capture images via a shooting means and generate image information of the image, location information indicating the location where the image was taken, and shooting date and time information indicating the date and time the image was taken, etc., via an installed program, and transmit them to the server 4. As shown in Figure 2, the first terminal 2 comprises a terminal body 20, a touch display 26 as an operating means and display means provided on the front side of the terminal body 20, and a camera 27 as a shooting means provided on the back side of the terminal body 20. In this invention, the first terminal 2 is not limited to a smartphone, but can be any type of portable terminal that can be used on site, such as a tablet terminal or a notebook computer.

[0024] Figure 3 is a block diagram of the configuration of the first terminal 2 used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. As shown in Figure 3, the first terminal 2 is configured to include a control unit 21 that controls the entire first terminal 2, a storage means 25, a touch display 26, a camera 27, and a communication means 28.

[0025] The control unit 21 is configured to control the entire first terminal 2, and the CPU (Central Processing Unit) 22, which is the processor, loads the program stored in the ROM (Read-only memory) 23 into the RAM (Random Access Memory) 24 and executes it, thereby performing various functions such as the image information generation unit 211, which will be described later. In this invention, the processor is not limited to a CPU, and any other processor such as a GPU (Graphics Processing Unit) may be used. The same applies to the processors of the control unit 31 of the second terminal 3 and the control unit 41 of the server 4, which will be described later.

[0026] The storage means 25 is configured using a storage device such as a magnetic hard disk or a semiconductor storage device. The storage means 25 stores various information necessary for the first terminal 2 to function, as well as image information of the captured image, the date and time information of the image, and the location information of the image.

[0027] The touch display 26 functions as a display means (first terminal display means) that provides various information to the user, and also functions as an operating means (first terminal operating means) that enables various operations by touching the touch display 26.

[0028] Camera 27 is a photographic means equipped with an optical configuration such as a lens and a photosensitive element.

[0029] The communication means 28 is a network interface that enables wired and / or wireless communication, and communicates with the server 4 via the network N. The communication means 28 passes various information received from the server 4 to the control unit 21, and also transmits information generated by the control unit 21 to the server 4.

[0030] Figure 4 is a functional block diagram of a control unit 21 provided in a first terminal 2 used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. As shown in Figure 4, the control unit 21 functions as an image information generation unit 211, a location information generation unit 212, a shooting date and time information generation unit 213, an operation information generation unit 214, and a display unit 215, etc., by having the CPU 22 load a program stored in the ROM 23 into the RAM 24 and execute it.

[0031] The image information generation unit 211 captures on-site images of the site via the camera 27 and generates image information, which is data from the on-site images.

[0032] The location information generation unit 212 generates location information (geotags) indicating the location where the field image was taken, using a GNSS (Global Navigation Satellite System) such as GPS (Global Positioning System, not shown), which is a location information generation means provided in the first terminal 2.

[0033] The shooting date and time information generation unit 213 generates shooting date and time information indicating the shooting date and time of the on-site image using a timer function (not shown) provided in the first terminal 2.

[0034] The operation information generation unit 214 generates operation information corresponding to operations performed by the user via the touch display 26. The operation information generated by the operation information generation unit 214 is used for the operation of the machine and, if necessary, is transmitted to the server 4 for information processing by the server 4.

[0035] The display unit 215 displays a GUI (Graphical User Interface) via the touch display 26 based on various display information, and performs drawing based on display information, which is various drawing information received from the server 4.

[0036] For example, display information transmitted from server 4 to first terminal 2 includes display information for a setting operation screen that allows selecting the site type for a site image. When a user performs a setting operation via the setting operation screen displayed on touch display 26, the operation information generation unit 214 generates site type information indicating the selected site type as the operation information. The generated site type information is transmitted to server 4 and stored in storage means 45 by storage unit 411 (described later) in association with map information, site image information, and location information.

[0037] Furthermore, the display information transmitted from the server 4 to the first terminal 2 includes the display information of the operation screen for changing the response necessity information. When a user performs a response necessity operation via the operation screen displayed on the touch display 26, the operation information generation unit 214 generates change operation information indicating the change operation. The generated change operation information is transmitted to the server 4, and the response necessity information stored in the storage means 45 is updated and saved based on the change operation information.

[0038] Figure 5 is a schematic perspective view showing a second terminal 3 used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. In this embodiment, the second terminal 3 is assumed to be a notebook computer and is capable of requesting various information from the server 4 via an installed program, as described later, and of performing various drawings based on display information received from the server 4.

[0039] As shown in Figure 5, the second terminal 3 comprises a terminal body 30, a display 36 as a display means (second terminal display means) provided on the terminal body 30, and an operating means (second terminal operating means) 37 such as a keyboard. In this invention, the second terminal 3 is not limited to a notebook computer, but can be a tablet computer, a desktop computer, or a smartphone with the same device configuration as the first terminal 2, or any other type of terminal.

[0040] Furthermore, in this invention, a single terminal may be used as both the first terminal 2 and the second terminal 3. For example, a single tablet terminal may be used as the first terminal 2 during inspection work and as the second terminal 3 when viewing the inspection results. In this case, the terminal is equipped with all the functions required for both the first terminal 2 and the second terminal 3.

[0041] Figure 6 is a block diagram of the configuration of a second terminal 3 used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. As shown in Figure 6, the second terminal 3 is configured to include a control unit 31 that controls the entire second terminal 3, a storage means 35, a display 36, an operating means 37, and a communication means 38.

[0042] The control unit 31 performs various functions, such as the request information generation unit 311 described later, by having the CPU 32 load the program stored in the ROM 33 into the RAM 34 and execute it.

[0043] The storage means 35 is configured using a storage device such as a magnetic hard disk or a semiconductor storage device. The storage means 35 stores various information necessary for the second terminal 3 to function.

[0044] The display 36 functions as a means of providing various information to the user.

[0045] The operating means 37 is an input device such as a keyboard or mouse that accepts various input operations from the user, and the information of the input operations is passed to the control unit 31 and used for various information processing. The operating means 37 may also include an audio input means including a microphone. Furthermore, it may be a touch display that integrates the display means and input means, as in the first terminal 2.

[0046] The communication means 38 is a network interface that enables wired and / or wireless communication, and communicates with the server 4 via the network N. The communication means 38 passes various information received from the server 4 to the control unit 31, and also transmits information generated by the control unit 31 to the server 4.

[0047] Figure 7 is a functional block diagram of a control unit 31 provided in a second terminal 3 used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. As shown in Figure 7, the control unit 31 functions as a request information generation unit 311 and a display unit 312, etc., by having the CPU 32 load a program stored in ROM 33 into RAM 34 and execute it.

[0048] The request information generation unit 311 generates request information that requests the server 4 to send various types of information based on operations performed by the user via the operation means 37. The request information includes initial request information that requests initial display information related to a predetermined site, as described later, and detailed request information that requests detailed display information. The generated request information is transmitted to the server 4 via the communication means 38. The request information also includes identification information that identifies each second terminal 3. The identification information of the second terminal 3 may be, for example, the user's login information (user ID and password, etc.) entered by the user via the operation means 37 when establishing a connection between the second terminal 3 and the server 4.

[0049] The display unit 312 displays various information, such as a map of the site and images of the site, on the display 36 based on display information (initial display information and detailed display information, etc.) that indicates the display format of various information received from the server 4. Details of the display by the display unit 312 will be described later.

[0050] Figure 8 is a block diagram of the server configuration used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. As shown in Figure 8, the server 4 is configured to include a control unit 41 that controls the entire server 4, a storage means 45, and a communication means 46. The server 4 may be configured as a single device, or it may be a cloud server, or a combination of these.

[0051] The control unit 41 performs various functions, such as the storage unit 411 and the information extraction unit 412, which will be described later, by having the CPU 42 load the program stored in the ROM 43 into the RAM 44 and execute it.

[0052] The storage means 45 is configured using a storage device such as a magnetic hard disk or a semiconductor storage device. The storage means 45 stores various information necessary for the server 4 to function. The storage means 45 also stores map information of one or more large outdoor facilities to be inspected, such as steel mills, petrochemical complexes, factories, parks, and public facilities. Furthermore, the storage means 45 stores image information, location information, and shooting date and time information transmitted from the first terminal 2 in association with each other. In addition, the storage means 45 also stores various information used in the inspection support system 1, such as site type information, work progress information, and response necessity information, in association with the image information, location information, and shooting date and time information.

[0053] The communication means 46 is a network interface that enables wired and / or wireless communication, and communicates with the first terminal 2 and the second terminal 3 via the network N. The communication means 46 passes various information received from the first terminal 2 and the second terminal 3 to the control unit 41, and also transmits information generated by the control unit 41 to the first terminal 2 and the second terminal 3. The information transmitted from the server 4 to the second terminal 3 includes various types of information such as map information, image information, location information, shooting date and time information, display information, site type information, response necessity information, work progress information, and chart display information.

[0054] Figure 9 is a functional block diagram of a control unit 41 provided in a server 4 used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. As shown in Figure 9, the control unit 41 functions as a storage unit 411, an information extraction unit 412, a display information generation unit 413, and a response necessity determination unit 414, etc., by having the CPU 42 load a program stored in ROM 43 into RAM 44 and execute it.

[0055] The storage unit 411 stores image information, location information, and shooting date and time information received from the first terminal 2 in the storage means 45 in an updatable manner, associating them with each other. The storage unit 411 also stores site type information, work progress information, and response necessity information in the storage means 45 in an updatable manner, associating them with the image information, location information, and shooting date and time information.

[0056] The information extraction unit 412 extracts various types of information from the storage means 45, such as map information, image information, location information, shooting date and time information, site type information, response necessity information, and work progress information, in response to the request information received from the second terminal 3. This information extracted by the information extraction unit 412 is transmitted to the second terminal 3 (and to the first terminal 2 if necessary) via the communication means 46.

[0057] The information extraction unit 412 extracts map information based on identification information indicating the affiliation of each second terminal 3. That is, the information extraction unit 412 identifies which facility the second terminal 3 is used for inspection based on the identification information, extracts the map information for that facility, and transmits the extracted map information and various related information to the second terminal 3. The identification information may be, for example, the ID and password entered into the server 4 via the second terminal 3 when logging into the inspection support system 1.

[0058] The display information generation unit 413 generates display information indicating the display format of various information such as map information, image information, and location information, for display on the touch display 26, which is the display means (first terminal display means) of the first terminal 2, or the display 36, which is the display means (second terminal display means) of the second terminal. In other words, in the inspection support system 1, the display of the inspection support functions of the inspection support system 1 on the first terminal 2 and the second terminal 3 is performed according to the display information generated by the display information generation unit 413 of the server 4.

[0059] The display information generated by the display information generation unit 413 includes initial display information that displays a map on the display 36 of the second terminal 3 based on map information and displays a mark that can be selected via the operating means 37 at a position corresponding to the shooting location on the map, and detailed display information that displays an enlarged map of the area around the selected mark on the display 36 and displays a field image associated with the shooting location corresponding to the mark on the enlarged map.

[0060] The response necessity determination unit 414 uses known methods such as machine learning and deep learning to determine the response necessity of the site based on on-site image information and generates response necessity information indicating that response necessity. In this embodiment, four levels of response necessity are provided: A, indicating a state requiring emergency response; B, indicating a dangerous state that may require response in the future; C, indicating that no response is yet required but observation should be continued; and D, indicating no problem.

[0061] As described above, by providing the response necessity determination unit 414, the response necessity of inspection work at each site can be automatically determined based on the site images taken by the first terminal 2, and this response necessity is shared with the user using the second terminal 3 via the server 4. By sharing the response necessity information with the user, it becomes easier for the user to plan the inspection work. The specific determination of the response necessity by the response necessity determination unit 414 will be described later.

[0062] Next, we will describe a specific example of inspection support provided by the inspection support system 1 having the configuration described above.

[0063] Figure 10 is a schematic diagram showing a map AT of a facility CE to which an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention is applied. In this embodiment, the facility CE shown in Figure 11 is a large-scale steel mill located in a port area and is equipped with various structures such as belt conveyors BE that are spread throughout the facility CE. Map information showing such a map AT is stored in advance in the storage means 45 of the server 4.

[0064] Figure 11 is a schematic diagram showing a site image P of a site where an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention is applied. In the site image P shown in Figure 11, ore that has fallen from the gap G at the joint of adjacent belt conveyors BE has accumulated as fallen ore F on the ground below the belt conveyors BE. When fallen ore F accumulates, it may cause various problems, such as contact with and damage to the belt conveyors BE. Therefore, it is necessary to constantly monitor the amount of fallen ore F and remove it as needed.

[0065] When such a site image P is captured by the camera 27 of the first terminal 2, which is a smartphone, the image information generation unit 211 of the first terminal 2 generates image information representing the site image P. In addition, the location information generation unit 212 generates location information of the shooting location using the GPS installed in the first terminal 2, and the shooting date and time information generation unit 213 generates shooting date and time information using the timer function installed in the first terminal 2. The image information, location information, and shooting date and time information are transmitted to the server 4 and finally stored in the storage means 45 in association with each other, although this association itself may be performed in the first terminal 2.

[0066] Then, the response necessity determination unit 414 of the server 4, which has received the image information of the site image P, determines the response necessity based on the site image P. In the example of the site image P shown in Figure 11, the response necessity determination unit 414 calculates the height H of the fallen ore F from the site image P and determines the response necessity by comparing the calculated height H with one or more thresholds.

[0067] For example, the response necessity determination unit 414 determines that if the height of the fallen ore F is less than 10 cm, the response necessity is D (no problem); if it is 10 cm or more but less than 20 cm, the response necessity is C (requires observation); if it is 20 cm or more but less than 30 cm, the response necessity is B (dangerous); and if it is 30 cm or more, the response necessity is A (requires urgent action). The response necessity determination unit 414 then generates response necessity information indicating the response necessity result, and the generated response necessity information is stored in the storage means 45 by the storage unit 411, associated with image information and location information.

[0068] The determination of the need for action by the action necessity determination unit 414 can be performed using various known methods depending on the type of on-site image P. For example, if the on-site image P represents the appearance of the device, the action necessity determination unit 414 may determine the need for action by using AI (artificial intelligence) to determine the degree of damage based on the shape of the device surface.

[0069] As described above, the on-site images P, etc., captured by the first terminal 2 and stored in the server 4 can be viewed using the second terminal 3. Specifically, when the server 4 and the second terminal 3 are connected, and the ID and password as login information are entered into the server 4 via the second terminal 3, the request information generation unit 311 of the second terminal 3 first generates initial request information, which is request information to send initial display information to the server 4, and this initial request information is sent to the server 4.

[0070] Then, the information extraction unit 412 of the server 4 identifies the facility in which the second terminal 3 is used based on the identification information of the second terminal 3, extracts map information of the facility, and extracts location information associated with the map information.

[0071] The display information generation unit 413 then displays the map AT on the display 36 of the second terminal 3 based on the extracted map information, and generates initial display information to display a mark M that can be selected via the operating means 37 at a position on the map AT corresponding to the shooting position, based on the extracted location information.

[0072] Figure 12 is a schematic diagram showing the state in which a map AT and marks M are displayed on the display 36 of a second terminal 3 used in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention. The display unit 312 of the second terminal 3, which has received initial display information from the server 4, displays, based on the initial display information, a map AT showing the facility CE, selectable marks M (M1, M2, M3) at positions corresponding to one or more location information associated with the map AT, and a setting menu S for various settings on the display 36, as shown in Figure 12.

[0073] In the example shown in Figure 12, three different marks, M1, M2, and M3, are displayed as Mark M, corresponding to the level of need for action. Mark M1 indicates a need for action level C, Mark M2 indicates a need for action level B, and Mark M3 indicates a need for action level A. Mark M is displayed at locations corresponding to various sites, such as the conveyor belt BE within the map AT. By displaying different Mark M according to the level of need for action, users can easily select and view the information they need, and they can grasp the locations requiring action and the level of need for action at a glance.

[0074] The settings menu S is a menu button that displays a selection screen for performing various setting operations to change the display mode of the display 36. The selection screen is displayed when the user performs a selection operation on the settings menu S via the operating means 37.

[0075] For example, it is possible to perform an operation to display only the mark M corresponding to a specific site type on the display 36.

[0076] Specifically, by selecting the settings menu S, the selection operation screen that appears allows the user to choose the type of site to display from options such as "belt conveyor," "hopper," and "chute." For example, if the user selects "belt conveyor" as the type of site to display, the request information generation unit 311 generates initial request information requesting map information and location information associated with the "belt conveyor" site type, and the generated initial request information is sent to the server 4.

[0077] Then, the information extraction unit 412 of the server 4, upon receiving the initial request information, extracts map information and location information associated with the site type "belt conveyor" from the storage means 45 according to the initial request information. At the same time, the display information generation unit 413 generates initial display information that displays Mark M only at the locations corresponding to the location information corresponding to the site type "belt conveyor" and transmits it to the second terminal 3. As a result, only Mark M, for which the site type is "belt conveyor," is displayed on the display 36 of the second terminal 3.

[0078] Furthermore, it is possible to display only the Mark M corresponding to a specific level of need on the display 36.

[0079] Specifically, on the selection screen displayed after selecting the settings menu S, the user can choose the level of response required to display from "A", "B", "C", and "D". For example, if the user selects "A" as the level of response required to display, the request information generation unit 311 generates initial request information requesting map information and location information associated with the level of response required "A", and the generated initial request information is sent to the server 4.

[0080] Then, the information extraction unit 412 of the server 4, which has received the initial request information, extracts map information and location information associated with response necessity level "A" from the storage means 45 according to the initial request information, and the display information generation unit 413 generates initial display information that displays Mark M only at locations corresponding to the location information associated with response necessity level "A", and transmits it to the second terminal 3. As a result, only Mark M with a response necessity level of "A" will be displayed on the display 36 of the second terminal 3.

[0081] Furthermore, it is possible to display only the Mark M corresponding to a specific shooting period on the display 36.

[0082] Specifically, by selecting the settings menu S, the user can select the shooting period to display on the selection screen. For example, if the user selects "April 2022 to June 2022" as the shooting period, the request information generation unit 311 generates initial request information requesting map information and location information associated with the shooting date and time information included in the shooting period "April 2022 to June 2022," and the generated initial request information is sent to the server 4.

[0083] Then, the information extraction unit 412 of the server 4, upon receiving the initial request information, extracts map information and location information associated with the shooting date and time information included in the shooting period "April 2022 to June 2022" from the storage means 45 according to the initial request information. At the same time, the display information generation unit 413 generates initial display information that displays Mark M only at the locations corresponding to the location information associated with the shooting date and time information included in the shooting period "April 2022 to June 2022", and transmits it to the second terminal 3. As a result, only Mark M corresponding to the shooting date and time information included in "April 2022 to June 2022" is displayed on the display 36 of the second terminal 3.

[0084] Furthermore, if work progress information is associated with and stored in relation to the image information and location information of the on-site image P, it is also possible to display only the marker M corresponding to a specific work progress on the display 36.

[0085] Specifically, on the selection screen displayed after selecting the settings menu S, the user can choose the work progress to display from "Not yet processed," "In progress," and "Completed." For example, if the user selects "Not yet processed" as the work progress to display, the request information generation unit 311 generates initial request information requesting map information and location information associated with the work progress "Not yet processed," and the generated initial request information is sent to the server 4.

[0086] Then, the information extraction unit 412 of the server 4, which has received the request information, extracts map information and location information associated with the "unaddressed" work progress from the storage means 45 according to the initial request information. At the same time, the display information generation unit 413 generates initial display information that displays the mark M only at the location information associated with the "unaddressed" work progress, and transmits it to the second terminal 3. As a result, only the mark M indicating the "unaddressed" work progress is displayed on the display 36 of the second terminal 3.

[0087] In this way, it is possible to extract and display only the Mark M related to specific response requirements, shooting periods, and / or work progress, making it possible to extract and view only specific inspection results, thus making it easier for users to plan inspections and repairs.

[0088] Figure 13 is a schematic diagram showing the state where a selection operation has been performed on mark M displayed on the display 36 in Figure 12, and the on-site image P is displayed on the enlarged map EA. In the example shown in Figure 13, it is assumed that mark M3 (indicating a response requirement level A) displayed in Figure 12, located in the upper right corner of the page and on the belt conveyor BE, has been selected.

[0089] When mark M3 on the belt conveyor BE is selected via the operating means 37 of the second terminal 3, the request information generation unit 311 of the second terminal 3 generates detailed request information that requests detailed display information from the server 4 based on the selection operation of mark M3. The generated detailed request information is transmitted to the server 4 via the communication means 38.

[0090] Upon receiving the detailed request information, the display information generation unit 413 of the server 4 displays an enlarged map EA, which is an enlarged version of the map AT surrounding the selected upper right mark M3, on the display 36, and generates detailed display information that displays the on-site image associated with the shooting location corresponding to mark M3 on the enlarged map EA. The generated detailed display information is transmitted to the second terminal 3 via the communication means 46.

[0091] The display unit 312 of the second terminal 3, having received the detailed display information, displays the enlarged map EA on the display 36 as shown in Figure 13, and also displays the field image P on the enlarged map EA. The magnification ratio of the enlarged map EA relative to the original map AT may be set in advance, or it may be arbitrarily changed via the setting menu S of the second terminal 3.

[0092] Furthermore, as shown in Figure 13, in addition to the enlarged map EA and the site image P, an information window W may be displayed that shows information such as the date and time of shooting, site type, level of need for action, work progress, and the photographer of the site image. In this case, the detailed display information transmitted from the server 4 includes information indicating that the information window W will display information such as the date and time of shooting, site type, level of need for action, work progress, and photographer information indicating the photographer.

[0093] The operation to return from the display state shown in Figure 13 to the display state shown in Figure 12 can also be performed via the operation means 37.

[0094] Next, the operation of the inspection support system 1 having the above-described configuration and functions will be explained using a flowchart. Figure 14 is a flowchart showing the processing in the inspection support method for large-scale outdoor facilities according to an embodiment of the present invention.

[0095] As shown in Figure 14, first, when a field image P is captured using the camera 27 of the first terminal 2, the image information generation unit 211 generates image information representing the field image P (step S1).

[0096] Next, the location information generation unit 212 of the first terminal 2 generates location information indicating the location where the field image was taken using GNSS, which is a location information generation means provided in the first terminal 2 (step S2).

[0097] Next, image information and location information are transmitted from the first terminal 2 to the server 4 (step S3). At this time, as described above, various other information such as the date and time of shooting may also be transmitted.

[0098] Next, the server 4 receives image information and location information transmitted from the first terminal 2 (step S4), and the response necessity determination unit 414 determines the response necessity at the site based on the received image information and generates response necessity information (step S5). Then, the storage unit 411 of the server 4 stores the image information, location information, and response necessity information in the storage means 45, associating them with the site map information that has been previously stored in the storage means 45 (step S6). At this time, as described above, various information such as shooting date and time information may also be received from the first terminal 2 along with the image information and location information, and may be stored in association with the storage means 45.

[0099] Next, the request information generation unit 311 of the second terminal 3 generates initial request information requesting initial display information, and sends it to the server 4 (step S7).

[0100] Then, the information extraction unit 412 of the server 4, which has received the initial request information, extracts map information of the facility in which the second terminal 3 is used, based on the identification information of the second terminal 3, and also extracts location information stored in the storage means 45 in association with the map information, and response necessity information associated with the location information. Furthermore, the display information generation unit 413 displays the map AT on the display 36 of the second terminal 3 based on the map information extracted by the information extraction unit 412, and generates initial display information that displays different marks M according to the response necessity, which can be selected via the operation means 37 at the location information on the map AT. Once the initial display information is generated, it is transmitted to the second terminal 3 via the communication means 46 (step S8).

[0101] Then, the display unit 312 of the second terminal 3 displays the map AT on the display 36 based on the received initial display information, and also displays different marks M on the map AT according to the degree of need for correspondence (step S9, see Figure 12).

[0102] Then, when the user selects a mark M via the operating means 37 (step S10: YES), the request information generation unit 311 of the second terminal 3 generates request information requesting detailed display information related to the selected mark M and sends it to the server 4 (step S11).

[0103] Upon receiving the detailed request information, the display information generation unit 413 of the server 4 generates detailed display information related to the selected mark M, and the generated detailed display information is transmitted to the second terminal 3 (step S12).

[0104] Then, the display unit 312 of the second terminal 3, having received the detailed display information, displays the enlarged map EA on the display 36 based on the detailed display information, and also displays the field image P associated with the location information corresponding to the mark M selected in step S8 on the enlarged map EA (step S13), and ends the series of processes related to the display of the field image P.

[0105] On the other hand, if mark M is not selected in step S8 (step S10: NO), the second terminal 3 continues to display information based on the initial display information (step S9) until mark M is selected.

[0106] According to the inspection support system 1 of this embodiment described above, any user possessing a first terminal 2, such as a smartphone, can easily take on-site images P of the facility. Since taking such images constitutes the facility inspection work, even users unfamiliar with inspection work can easily perform inspection work and record the inspection results. Furthermore, the on-site images P stored on the server 4 can be easily viewed using a second terminal 3, such as a notebook computer, simply by selecting a mark M on the map AT, thus making it easy to view the inspection results.

[0107] Furthermore, by displaying different marks (M) depending on the level of need for action, users can easily select and view inspection results that are of high importance and require priority attention.

[0108] Furthermore, when a mark M is selected on the map AT, the surrounding map of that mark M is enlarged, and the site image P is displayed on the enlarged map EA. This allows the user to grasp the detailed location of the site on the enlarged map EA while checking the site conditions, making it possible to understand the site conditions more accurately.

[0109] Furthermore, the present invention is not limited to the embodiments and modifications described above, and various modifications can be adopted.

[0110] For example, the response requirement level generated by server 4 may be modified via terminal 2.

[0111] Specifically, when the server 4's response necessity determination unit 414 determines the response necessity based on the on-site image and generates response necessity information, the server 4 generates and transmits to the first terminal 2 that transmitted the on-site image information the response necessity information, along with display information including the display mode of the operation screen for changing the response necessity information generated by the display information generation unit 413.

[0112] Upon receiving the response requirement information and display information, the first terminal 2 displays a change operation screen on the touch display 26 based on the display information. When a change operation is performed on the response requirement via the change operation screen on the touch display 26, the operation information generation unit 214 generates and transmits change operation information indicating the change operation to the server 4.

[0113] Then, the storage unit 411 of the server 4, which has received the change operation information, updates the response necessity information stored in the storage means 45 based on the change operation information and saves it.

[0114] In this way, by making it possible to change the response necessity level via the first terminal 2, if the response necessity level automatically generated by the response necessity level determination unit 414 is deemed inaccurate, the user who took the on-site image can correct the response necessity level themselves, thereby making the response necessity level more accurate in reflecting the actual on-site conditions.

[0115] Alternatively, the server 4 may not perform a determination of the need for response, and instead, the user may set the need for response at the site via the first terminal 2. In this case, the server 4 is not provided with a need for response determination unit 414. When the user takes a picture of the site using the first terminal 2, they may further perform an operation to set the need for response at the site via the first terminal 2, and the need for response information indicating the set need for response may be sent to the server 4 and stored in association with the map information, image information, and location information. By adopting such a configuration, it is possible to simplify the configuration of the server 4 and reduce the processing load on the server 4.

[0116] Furthermore, the inspection support system 1 may be configured to generate charts and graphs based on the degree of need for action and the date and time of photography information. Figure 15 is a schematic diagram showing chart D created in an inspection support method for large-scale outdoor facilities according to an embodiment of the present invention.

[0117] Specifically, the display information generation unit 413 of server 4 generates display information, including the display mode of the chart creation button that creates chart D based on the response necessity information and shooting period selected by the user via the second terminal 3, and transmits it to the second terminal 3.

[0118] Then, the second terminal 3 displays a chart creation button on the display 36 based on the displayed information, and when the request information generation unit 311 receives an operation of the chart creation button via the operation means 37, it generates and transmits chart request information to the server 4 requesting a chart based on the selected response necessity and shooting period.

[0119] Upon receiving the chart request information, the information extraction unit 412 of the server 4 extracts information on the degree of need for action and the date and time of shooting within the shooting period based on the chart request information. The display information generation unit 413 generates chart display information, including the display method of the chart, based on the extracted degree of need for action and date and time of shooting information. The generated chart display information is then transmitted from the server 4 to the second terminal.

[0120] Then, the display unit 312 of the second terminal 3, which has received the chart display information, displays Chart D on the display 36 based on the chart display information. In the example shown in Figure 15, the number of responses required for response level A is shown as an "x", the number of responses required for response level B is shown as a triangle, and the number of responses required for response level C is shown as a circle, with each period being displayed accordingly.

[0121] In this way, by enabling the creation of Figure D based on the required response level and the date and time of shooting information, the user of the second terminal can easily grasp the changes in the required response level from time to time, easily predict changes in the required response level, and easily plan inspection and repair work.

[0122] Furthermore, in the embodiment described above, when Mark M is selected, the surrounding map of Mark M is displayed in an enlarged view, and the site image P is displayed on the enlarged map EA. However, this enlarged view may be omitted. That is, when Mark M is selected, the site image P may be displayed on map A without enlarging map A. By adopting such a configuration, the processing burden on server 4 to generate the drawing information for the enlarged map EA can be reduced, and detailed display information can be generated quickly. However, as mentioned above, using the enlarged map EA has the advantage of allowing the detailed location of the site to be grasped using the enlarged map EA, while also allowing the site conditions to be confirmed, thus enabling a more accurate understanding of the site conditions.

[0123] Furthermore, while only one second terminal 3 was used in the inspection support system 1 according to the above embodiment, two or more second terminals 3 may be used in the present invention. This further facilitates the inspection work of facilities.

[0124] Furthermore, a program may be provided that causes a computer to execute each of the processes performed by server 4. The program may be recorded on a computer-readable medium. Using a computer-readable medium, it is possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, but may be a recording medium such as a CD-ROM or DVD-ROM. [Explanation of Symbols]

[0125] 1. Inspection support system 2, 2A, 2B, 2N 1st terminal 3. Second Terminal 4 servers 20. Main unit 21 Control Unit 22 CPU 23 ROM 24 RAM 25 Memory means 26 Touch displays 27 Cameras 28. Means of communication 30. Main unit 31 Control Unit 32 CPU 33 ROM 34 RAM 35 Memory means 36 displays 37 Operating means 38. Means of communication 41 Control Unit 42 CPU 43 ROM 44 RAM 45 Memory means 46. ​​Means of communication 211 Image Information Generation Unit 212 Location information generation unit 213 Shooting Date and Time Information Generation Unit 214 Operation information generation section 215 Display section 311 Request information generation section 312 Display section 411 Preservation Department 412 Information extraction part 413 Display information generation section 414 Unit for determining the degree of need for action AT Map BE Belt Conveyor CE facility EA Enlarged Map F Mine fall G gap H Height M, M1, M2, M3 marks N Network P On-site image S Settings Menu W Information Window

Claims

1. The first terminal takes a picture of the site, generates image information of the site and location information indicating the shooting location, and transmits them to the server. The server performs the steps of determining the degree of need for action at the site based on the image information and generating information on the degree of need for action, The server performs the steps of associating the image information, location information, and correspondence necessity information with the map information of the map including the shooting location and storing it, The second terminal generates initial request information requesting initial display information related to a predetermined site and transmits it to the server. The server extracts the map information, location information, and response necessity information in accordance with the initial request information, generates initial display information that displays a user-selectable mark at the shooting location on the map, which differs according to the response necessity information, along with the map, and transmits it to the second terminal. The second terminal performs the step of displaying information on the display means based on the initial display information, When the user performs the selection operation on the mark displayed on the second terminal, the second terminal generates detailed request information requesting detailed display information about the selected mark and sends it to the server. The server generates detailed display information which, in response to the detailed request information, displays an enlarged map of the map surrounding the selected mark, extracts the image information associated with the location information corresponding to the selected mark, and overlays the on-site image onto the enlarged map, and transmits this information to the second terminal. The second terminal performs the step of displaying information on the display means based on the detailed display information, A method for supporting inspections of large-scale outdoor facilities, including [specific details omitted].

2. The first terminal is capable of modifying the response necessity information generated by the server, and when such modification is performed, it generates modification information and sends it to the server. The inspection support method for large-scale outdoor facilities according to claim 1, wherein the server updates and stores the response necessity information based on the change operation information.

3. The first terminal takes a picture of the site at the site, generates information on the need for action at the site based on the image information of the site, location information indicating the shooting location, and user settings, and transmits this information to the server. The server performs the steps of associating the image information, location information, and correspondence necessity information with the map information of the map including the shooting location and storing it, The second terminal generates initial request information requesting initial display information related to a predetermined site and transmits it to the server. The server extracts the map information, location information, and response necessity information in accordance with the initial request information, generates initial display information that displays a user-selectable mark at the shooting location on the map, which differs according to the response necessity information, along with the map, and transmits it to the second terminal. The second terminal performs the step of displaying information on the display means based on the initial display information, When the user performs the selection operation on the mark displayed on the second terminal, the second terminal generates detailed request information requesting detailed display information about the selected mark and sends it to the server. The server generates detailed display information which, in response to the detailed request information, displays an enlarged map of the map surrounding the selected mark, extracts the image information associated with the location information corresponding to the selected mark, and overlays the on-site image onto the enlarged map, and transmits this information to the second terminal. The second terminal performs the step of displaying information on the display means based on the detailed display information, A method for supporting inspections of large-scale outdoor facilities, including [specific details omitted].

4. The first terminal further generates shooting date and time information indicating the date and time the on-site image was taken, and transmits it to the server along with the image information and the location information. The server stores the shooting date and time information in association with the map information, the image information, the location information, and the correspondence necessity information. The second terminal is capable of generating chart request information that requests a chart based on a predetermined level of response required and shooting period, and transmitting it to the server. The server extracts predetermined response necessity information and shooting date and time information within the shooting period based on the chart request information, generates chart display information including the display mode of the chart, and transmits it to the second terminal. The second terminal displays the chart on the display means based on the chart display information. The inspection support method for large-scale outdoor facilities according to claim 1 or 3.

5. A step of storing image information of a site image, which is an image of the site transmitted from the first terminal, and location information indicating the shooting location, in association with map information of a map that includes the shooting location. A step of determining the degree of need for action at the site based on the aforementioned image information and generating information on the degree of need for action, The process of storing the aforementioned response necessity information in association with the aforementioned map information, the aforementioned image information, and the aforementioned location information, The process includes extracting the map information, location information, and response necessity information in response to initial request information regarding a predetermined site transmitted from the second terminal, generating initial display information that displays a user-selectable mark on the map at the shooting location, which varies according to the response necessity information, along with the map, and transmitting it to the second terminal; When the second terminal receives detailed request information requesting detailed display information generated in response to the selection operation by the user, the process of generating detailed display information that displays an enlarged map of the map surrounding the selected mark, extracts the image information associated with the location information corresponding to the selected mark, and overlays the on-site image onto the enlarged map, and transmits it to the second terminal, is as follows: A program that causes a computer to execute something.

6. A step of storing image information of a site image, which is an image of the site transmitted from the first terminal, location information indicating the shooting location, and response necessity information indicating the degree of need for response at the site, in association with map information of a map including the shooting location. The process includes extracting the map information, location information, and response necessity information in response to initial request information regarding a predetermined site transmitted from the second terminal, generating initial display information that displays a user-selectable mark on the map at the shooting location, which varies according to the response necessity information, along with the map, and transmitting it to the second terminal; When the second terminal receives detailed request information requesting detailed display information generated in response to the selection operation by the user, the process of generating detailed display information that displays an enlarged map of the map surrounding the selected mark, extracts the image information associated with the location information corresponding to the selected mark, and overlays the on-site image onto the enlarged map, and transmits it to the second terminal, is as follows: A program that causes a computer to execute something.

Images