Safety patrol indication management support system and method

The system automates safety patrol reporting and management by using mobile terminals with imaging and voice-to-text capabilities, reducing manual effort and ensuring accurate, location-based tracking of safety issues.

JP2024168152APending Publication Date: 2024-12-05KK TOSHIBA +1
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2023084589
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-23
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Conventional safety patrols at power plants require significant manual effort for recording and editing safety issues, lack location records, and depend heavily on personnel familiar with the site, making timely information sharing and management challenging.

Method used

A system that uses a mobile terminal with imaging and voice-to-text capabilities to capture and register safety issues, integrates a platform for managing and classifying these issues by location, and automates form creation, reducing reliance on individual knowledge and workload.

Benefits of technology

Enables reliable reporting and management of safety issues without depending on site-specific knowledge, streamlining the process and ensuring accurate, area-specific tracking and corrective actions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024168152000001_ABST
    Figure 2024168152000001_ABST
Patent Text Reader

Abstract

To certainly perform report and management of an indication without depending on a person in charge who knows a field while reducing workloads about safety.SOLUTION: A patrol indication management support system comprises: a portable terminal 11 which inputs an indication about safety; a photographing device 12 which acquires video during safety patrol; and a platform 13 which registers and stores information regarding the indication to be managed. The platform comprises: a registration function unit 16 which registers information regarding the indication; and a slip automatic preparation function unit 17 which prepares a slip from information regarding the indication. The platform further comprises at least one of: a voice character conversion function unit 18; a position information calculation function unit 19 which calculates a position of the photographing device from the video of the photographing device; an indication position coordination function unit 20 which coordinates the position of the indication specified on the basis of the position of the photographing device with content of the indication; a classification function unit by area 21 which classifies the information regarding the indication for every management area to be stored; and a remaining case extraction function unit 22 which extracts the indication requiring treatment.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] An embodiment of the present invention relates to a safety patrol findings management support system and a safety patrol findings management support method. [Background technology]

[0002] In conventional safety patrols at power plants, issues are extracted, the object (subject of the issue) is photographed with a camera, the details are noted, and then the worker returns to the office to create a report. The number of issues extracted per safety patrol is just under 100, and because there are many issues extracted and the amount of work involved in recording and editing is large, there is an issue of time required to report. In addition, no records are kept of the location of issues, and corrective measures are confirmed based on the information in the photograph. For this reason, the extraction and management of issues during safety patrols at power plants must be handled by a person familiar with the site. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2009-15669 A [Patent Document 2] Patent Publication No. 2021-47638 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, there have been proposals for an automatic report creation function that classifies inspection items by ranking them (Patent Document 1), as well as a system that consolidates reports by predefined items to address the issue of relying on the person in charge to describe the details, making the classification and consolidation of findings cumbersome (Patent Document 2).

[0005] Actual work sites change over time as construction progresses, and the routes that safety patrols take, the locations they point out, and the details they point out are not fixed. For this reason, in order to perform appropriate safety management, people need to know the details of measures and the locations they point out, which creates the issue of being dependent on the person in charge. In addition, as the work site changes, it is necessary to reflect and share that information in a timely manner, and to manage the history of measures using fixed values ​​such as plant coordinates.

[0006] The embodiments of the present invention have been made in consideration of the above-mentioned circumstances, and aim to provide a safety patrol issue management support system and a safety patrol issue management support method that can reliably report and manage on-site safety issues and the like without relying on staff who are familiar with the site and while reducing the workload. [Means for solving the problem]

[0007] A safety patrol issue management support system according to an embodiment of the present invention includes a mobile terminal carried by a person in charge of safety patrols at the site and used to input safety issues at the site, a camera carried by the person in charge of safety patrols and used to capture images of the surrounding area during the safety patrol, a platform that registers, stores, and manages information related to the issues, including the issues, and a viewing terminal for viewing information related to the issues managed by the platform. The platform includes a registration function unit that registers and stores information related to the issues in a database, and a viewing terminal that creates and outputs information related to the issues as a form. In addition to having a report creation function unit, the system is further configured to have at least one of the following: a voice-to-text conversion function unit that converts voice input into text on the mobile terminal; a position information calculation function unit that calculates the position of the camera from the image captured by the camera; a problem position linking function unit that identifies the position of the problem based on the position of the camera and links the position of the problem with the content of the problem; an area-based classification function unit that classifies information about the problems by management area and stores it in the database; and a remaining item extraction function unit that extracts the problems requiring action from the database and displays them.

[0008] In an embodiment of the present invention, a method for supporting management of safety patrol issues includes a mobile terminal carried by a person in charge of safety patrols at the site, supporting the input of safety issues at the site, a camera carried by the person in charge of safety patrols, acquiring video of the surrounding area during the safety patrol, a platform registering, storing and managing information related to the issues including the issues, and a viewing terminal supporting viewing of the information related to the issues managed by the platform, and the platform performs a registration step of registering and storing information related to the issues in a database and a form creation step of creating and outputting the information related to the issues as a form, as well as at least one of a voice-to-text conversion step of converting voice input by the mobile terminal into text, a position information calculation step of calculating the position of the camera from the video acquired by the camera, an issues position linking step of identifying the position of the issues based on the position of the camera and linking the position of the issues with the content of the issues, an area-based classification step of classifying information related to the issues by management area and storing it in the database, and a remaining extraction step of extracting the issues requiring action from the database and displaying them. Effect of the Invention

[0009] According to an embodiment of the present invention, reporting and management of on-site safety issues and the like can be performed reliably without relying on staff who are familiar with the site and while reducing the workload. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of a safety patrol indication management support system according to a first embodiment. [Diagram 2] FIG. 2 is an explanatory diagram for explaining the function of a pointed out item location linking function unit in FIG. 1; [Diagram 3] (A) is a layout diagram on which the locations of points 1 to 4 are plotted, and (B) is a layout diagram on which the locations of points 5 to 9 are plotted. [Figure 4]FIG. 2 is a schematic diagram showing the hierarchical structure of the database in FIG. 1. [Diagram 5] 2 is a flowchart showing the operation of the safety patrol indication management support system of FIG. 1; [Figure 6] FIG. 2 is an explanatory diagram for explaining the function of a specified item position selection function unit in FIG. 1; [Figure 7] FIG. 11 is a block diagram showing the configuration of a safety patrol indication management support system according to a second embodiment. [Figure 8] 8 is an explanatory diagram for explaining the function of an indication target position estimation function unit in FIG. 7; [Figure 9] 8 shows point cloud data and shape data acquired by the LiDAR function of the mobile device in FIG. 7, where (A) and (B) are diagrams showing the shape data and point cloud data, respectively, before corrective action is taken, and (C) and (D) are diagrams showing the shape data and point cloud data, respectively, after corrective action is taken. [Figure 10] 9 is a diagram for explaining a state in which an indicated object position is estimated by the indicated object position estimation function unit in FIG. 7 and FIG. 8 . DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [A] First embodiment (FIGS. 1 to 6) Fig. 1 is a block diagram showing the configuration of a safety patrol findings management support system according to the first embodiment. The safety patrol findings management support system 10 shown in Fig. 1 supports reporting and management of findings and the like related to concerns about safety findings (subjects of awareness) discovered during a safety patrol at a work site, and is configured to include a mobile terminal 11 and an image capture device 12 carried by a person in charge of safety patrol, a platform 13 connected to the mobile terminal 11 and the image capture device 12 via a communication network 14, and a viewing terminal 15 connected to the platform 13 via the communication network 14 and installed in, for example, an office.

[0012] Of these, the platform 13 has a registration function unit 16 and an automatic report creation function unit 17, as well as at least one of a voice-to-text conversion function unit 18, a location information calculation function unit 19, a pointed out item location linking function unit 20, an area-based classification function unit 21, a remaining item extraction function unit 22, and a pointed out item location selection function unit 23.

[0013] The mobile terminal 11 is used to input safety issues at the site, and has general smartphone functions such as a browser display function, an input function, a voice collection function, and a photographing function. The browser display function displays a menu screen such as a search menu. The input function allows the content of safety issues at the site (described later) to be input as text. The voice collection function collects the content of the issues by voice when it is difficult to input text using the input function. The photographing function allows the photographing of the issues. The content of the issues input by text or voice and the photograph of the issues are uploaded to the platform 13 from the mobile terminal 11 by the person in charge. The mobile terminal 11 may be a smartphone, a tablet terminal, a pocket PC, or the like, and may be not only one but multiple terminals.

[0014] The viewing terminal 15 has a browser display function and a form download function, and the browser display function enables viewing of information (described later) related to the issues managed by the platform 13. The form download function enables forms created as described later by the automatic form creation function unit 17 of the platform 13 to be downloaded via the communication network 14. This viewing terminal 15 is a smartphone, tablet terminal, pocket PC, etc., and may be not only one terminal but also multiple terminals. Furthermore, this viewing terminal 15 may be provided in an office or the like that is remote from the site and the platform 13.

[0015] The photographing device 12 is attached to, for example, a helmet of a person in charge of safety patrol, and captures images (video) of the surroundings during the safety patrol. The video captured by this photographing device 12 is uploaded to the platform 13 via a communication network 14. For this photographing device 12, a camera capable of capturing general video, a 360-degree camera, or LiDAR (Laser Imaging Detection and Ranging) is used. This photographing device 12 starts capturing images at the start of a safety patrol and ends capturing images at the end of the safety patrol. It is assumed that the photographing device 12 uploads the captured video to the platform 13 when the capturing ends, but the video may be uploaded to the platform 13 at any time during the capturing.

[0016] The platform 13 registers, saves, and manages information related to the findings, including the findings. Here, the information related to the findings includes the contents of the findings (contents of the findings, classification of the findings, and whether or not corrective action is required), as well as the location of the findings, photos of the findings taken by the mobile terminal 11, and the movement trajectory of the photographing device 12 during safety patrols (the movement trajectory of the person in charge). The classification of the findings includes, for example, good, guidance, criticism, concern, reference, and quality.

[0017] The voice-to-text conversion function unit 18 in the platform 13 described above executes the function of converting voice memos relating to the contents of the pointed out matters collected and input by the voice collection function of the mobile terminal 11 into text data.

[0018] The position information calculation function unit 19 in the platform 13 extracts feature points from the surrounding image captured by the camera device 12, and calculates the position of the camera device 12 (the position of the person in charge) in three-dimensional coordinates in time series using a self-location estimation technology that extracts feature points from the surrounding image captured by the camera device 12 and estimates the position of the camera device 12 from changes in the feature points accompanying the movement of the camera device 12. The three-dimensional coordinates of the position of the camera device 12 calculated by the position information calculation function unit 19 are converted into values ​​in the plant coordinate system by the coordinate conversion function unit 24. When the camera device 12 moves outdoors, the position information calculation function unit 19 may use a global positioning system (GPS) in combination with the position information calculation function unit 19 to calculate the position of the camera device 12. In addition, it is also possible to detect changes in the arrangement of equipment from the image (video) captured by the camera device 12 by utilizing the above-mentioned feature point processing and using a difference detection technology.

[0019] The registration function unit 16 in the platform 13 registers and stores, in the database 25 of the platform 13, the information regarding the issues raised, including the content of the issues raised (content of the issues, classification of the issues, and summary of corrective measures), photos taken by the mobile terminal 11 of the issues raised, and the movement trajectory of the camera 12 during the safety patrol (movement trajectory of the person in charge), along with time information.

[0020] The indication position linking function unit 20 on the platform 13 identifies the position of the indication based on the position of the imaging device 12 (position of the person in charge) and links the position of the indication with the contents of the indication. That is, as shown in Fig. 2, the indication position linking function unit 20 first obtains a time-series three-dimensional coordinate (plant coordinate) representing the position of the imaging device 12 before or after the registration based on the time information when the contents of the indication are registered in the database 25 by the registration function unit 16, and calculates the stop position within a predetermined distance (for example, 1 meter) in the three-dimensional coordinate as the position of the indication. Next, the indication position linking function unit 20 links the calculated position of the indication with the contents of the indication (the indication, the indication classification, and the gist of the corrective action) based on the time information when the indication is registered.

[0021] The positions of the pointed out items calculated by the pointed out item position linking function unit 20 are plotted on a layout diagram (plan) as shown in Figures 3(A) and (B) by the pointed out item position plotting function unit 26 (Figure 1) in the platform 13. In Figure 3(A), the positions of pointed out items 1 to 4 are shown on the layout diagram, and in Figure 3(B), the positions of pointed out items 5 to 9 are shown on the layout diagram.

[0022] 1 and 2, the area-based classification function unit 21 in the platform 13 classifies information related to the findings for each management area based on the location of the findings, and stores the information in the database 25. As shown in Fig. 4, the database 25 is divided into hierarchical levels corresponding to the management areas, and the area-based classification function unit 21 stores information related to the findings in each of these hierarchical levels.

[0023] After the safety patrol is completed, the automatic report creation function unit 17 in the platform 13 shown in Figure 1 creates and outputs information regarding the issues raised as a report, together with a layout diagram showing the locations of the issues raised, created by the issue location plot function unit 26.

[0024] The remaining item extraction function unit 22 in the platform 13 extracts information regarding issues that are stored in each hierarchical level of the database 25, including issues that require corrective action, from the database 25 for each management area, and displays it on the mobile terminal 11 and the viewing terminal 15.

[0025] Next, the operation of the safety patrol indication management support system 10 will be described with reference to FIG. When a safety patrol by a person in charge starts (S1), the person in charge connects mobile terminal 11 to platform 13 via communication network 14, and a menu is displayed on mobile terminal 11 by the browser display function of mobile terminal 11 (S2). At this time, if information on issues requiring corrective action is stored in database 25 of platform 13, the information on these issues is extracted for each management area by remaining item extraction function unit 22 of platform 13 (S3) and displayed on mobile terminal 11. In addition, when the safety patrol starts, the camera device 12 also starts capturing video of the surroundings (S4).

[0026] During a safety patrol, if a safety concern is found (S5), the person in charge inputs the concerns using the input function or voice input function of the mobile terminal 11 (S6), takes a photograph of the concern using the photographing function of the mobile terminal 11 (S7), and these are uploaded to the platform 13. If the concerns are input by voice, the concerns are converted into text by the voice-to-text conversion function unit 18 of the platform 13 (S8), and then registered in the database 25 together with the photograph taken, with time information added by the registration function unit 16 (S9).

[0027] When a new problem is found during the safety patrol by the moving person in charge (S10), the above-mentioned steps S5 to S9 are repeated as steps S11 to S15, respectively. During the safety patrol by the moving person in charge (S16), usually about 100 problem items are found and the problem items are registered.

[0028] When the safety patrol by the person in charge ends (S17), the image capture by the camera 12 also ends (S18), and this image is uploaded from the camera 12 to the platform 13 (S19). Then, the position of the camera 12 is calculated from the image of the camera 12 by the position information calculation function unit 19 of the platform 13 (S20), and this position is converted into a value in the plant coordinate system by the coordinate conversion function unit 24. The position of the camera 12 converted into the plant coordinate system is registered in the database 25 by the registration function unit 16 as the movement trajectory of the camera 12 (movement trajectory of the person in charge) with time information attached (S21).

[0029] In the platform 13, the position of the pointed out matter is first specified (calculated) by the pointed out matter position linking function unit 20 based on the time information when the pointed out matter was registered in the database 25 by the registration function unit 16 and the movement trajectory of the imaging device 12 with the time information attached thereto, and then the position of the pointed out matter is linked to the content of the pointed out matter (S22). In addition, the position of the pointed out matter specified by the pointed out matter position linking function unit 20 is plotted on the layout drawing by the pointed out matter position plotting function unit 26.

[0030] Furthermore, on the platform 13, after the safety patrol is completed, information on the findings, including the content and location of the findings and a photograph of the findings, together with a layout diagram on which the findings are plotted, is created as a report by the automatic report creation function unit 17 (S23). The report created by the automatic report creation function unit 17 is checked by the person in charge (S24) and then reported (S25).

[0031] As configured as above, the first embodiment provides the following advantages (1) to (4). (1) As shown in FIG. 1, in the platform 13 where on-site safety issues, etc. are captured and uploaded using a mobile terminal 11, the registration function unit 16 and the area classification function unit 21 register, store and manage information regarding the issues, including the content and location of the issues, in a database 25, so that information regarding the issues can be reliably managed without relying on the person in charge of on-site safety patrols.

[0032] (2) In the platform 13, the automatic report creation function unit 17 creates and outputs information regarding the issues raised, including the content and location of the issues, as a report, so that information regarding the issues raised can be reported reliably while reducing the workload.

[0033] (3) When it is difficult to input the details of the findings as text using the input function of the mobile terminal 11, the details can be collected by voice using the voice collection function, and the voice memo is converted into text by the voice-to-text conversion function unit 18 of the platform 13 to produce text data. This makes it possible to streamline the on-site input work of inputting the details of the findings using the mobile terminal 11 during safety patrols.

[0034] (4) Information on the findings, including the content (content of the findings, classification of findings, and whether or not corrective action is required) and location, is classified by management area by the area classification function unit 21 of the platform 13 and stored in each layer of the database 25. Therefore, even if the movement route of the safety patrol is different, corrective action for the findings can be appropriately taken, and even if the content of the findings is different, the findings can be optimally managed.

[0035] However, depending on the route of the safety patrol, there may be cases where it is not possible to bring in the imaging device 12. Even in such cases, the indication position selection function unit 23 on the platform 13 assigns a position in the plant coordinate system to the indication.

[0036] That is, a layout plan for carrying out a safety patrol (see Figs. 3(A) and (B)) is displayed on the display screen of the mobile terminal 11. When a problem is found during a safety patrol, the position of the person in charge of the safety patrol at that time is selected by the person in charge on the layout plan on the display screen of the mobile terminal 11 as the location where the problem occurred (for example, by touching the display screen of the mobile terminal 11). Meanwhile, the problem location selection function unit 23 of the platform 13 has a coordinate system [Point A(x a , y a ), point B(x b , y b )] and the plant coordinate system [Point A(X a , Y a ), point B(X b , Y b )] is manually input by a person in charge or extracted from the database 25 and recorded in advance.

[0037] Therefore, when the position where an indication is generated [e.g., position (x1, y1)] is selected on the layout plan displayed on the mobile terminal 11, the indication position selection function unit 23 converts the position where the indication is generated into a position in the plant coordinate system [e.g., position (X1, Y1)] based on the above-mentioned correspondence relationship M, thereby assigning a position in the plant coordinate system to the indication. In this way, even if the camera 12 cannot be brought in during a safety patrol and the position information calculation function unit 19, the coordinate conversion function unit 24, and the indication position linking function unit 20 cannot calculate the position of the indication based on the image of the camera 12, the indication position selection function unit 23 can assign a position in the plant coordinate system to the indication.

[0038] The coordinate conversion by the indication location selection function unit 23 may be performed each time the location where an indication has occurred is selected by the mobile terminal 11, or may be performed all at once after all the locations where indications have occurred have been selected and stored in the database 25.

[0039] [B] Second embodiment (Figs. 7 to 10) 7 is a block diagram showing the configuration of a safety patrol indication management support system according to the second embodiment. In the second embodiment, the same parts as those in the first embodiment are denoted by the same reference numerals as those in the first embodiment, and the description thereof will be simplified or omitted.

[0040] The difference between the safety patrol issue management support system 30 of the second embodiment and the first embodiment is that the mobile terminal 31 has a LiDAR function capable of acquiring point cloud data in addition to the functions of the mobile terminal 11 of the first embodiment, and the platform 32 has an issue target position estimation function unit 33 that calculates and estimates the position of the issue target P (Figure 10), which is the subject of the issue, instead of the position information calculation function unit 19 of the first embodiment.

[0041] By using the LiDAR function of the mobile terminal 31, it is possible to obtain point cloud data of the pointed out object P and its surroundings. Fig. 9 shows the point cloud data of the pointed out object P and its surroundings obtained by the LiDAR function of the mobile terminal 31, and shape data obtained from this point cloud data. Fig. 9(A) and (B) respectively show the shape data and point cloud data before corrective action, and Fig. 9(C) and (D) respectively show the shape data and point cloud data after corrective action.

[0042] As shown in FIG. 8, the pointed out object position estimation function unit 33 estimates the position of the mobile terminal 31 (the position of the person in charge) by self-position estimation technology from the surrounding image acquired by the photographing device 12 and the surrounding point cloud data acquired by the LiDAR function of the mobile terminal 31 based on the time information attached to each. The reference R in FIG. 10 indicates the movement route of the mobile terminal 31 and the photographing device 12 (the movement route of the person in charge) in the safety patrol. Also, as shown in FIG. 8, the pointed out object position estimation function unit 33 estimates the direction of the mobile terminal 31 toward the pointed out object P from the point cloud data of the pointed out object P acquired by the LiDAR function of the mobile terminal 31 with time information attached. The reference S in FIG. 9 indicates the direction of the mobile terminal 31 toward the pointed out object P. As shown in FIG. 8, the pointed out object position estimation function unit 33 calculates and estimates the position of the pointed out object P in three-dimensional coordinates from the position (movement route R) of the mobile terminal 31 and the direction S of the mobile terminal 31 based on the time information.

[0043] The position of the pointed out object P estimated by the pointed out object position estimation function unit 33 is converted into a value of the plant coordinate system by the coordinate conversion function unit 24 (FIG. 7), and then, as shown in FIG. 8, the photo of the pointed out object P taken by the photographing function of the mobile terminal 31, the video of the pointed out object P taken by the photographing device 12, and the shape data obtained from the point cloud data of the pointed out object P acquired by the LiDAR function of the mobile terminal 31 are registered in the database 25 by the registration function unit 16 with time information. In addition, the position of the pointed out object P estimated by the pointed out object position estimation function unit 33 is linked to the contents of the pointed out object based on the time information by the pointed out object position linking function unit 34 shown in FIG. 7, and is plotted on the layout diagram by the pointed out object position plotting function unit 35.

[0044] As configured as above, the second embodiment also provides the following effect (5) in addition to the effects (1) to (4) of the first embodiment.

[0045] (5) The position of the pointed out object P is calculated and estimated by the pointed out object position estimation function unit 33 from the point cloud data acquired by the LiDAR function of the mobile terminal 31 and the image acquired by the photographing device 12, and the position of the pointed out object P is linked to the contents of the pointed out object (pointed out contents, pointed out classification, whether or not corrective measures are required) by the pointed out object position linking function unit 34. Then, the position of the pointed out object and the contents of the pointed out object are stored in the database 25 for each management area by the area classification function unit 21 together with the photograph of the pointed out object P taken by the mobile terminal 31, the image of the pointed out object P taken by the photographing device 12, and the shape data of the pointed out object P obtained from the point cloud data acquired by the mobile terminal 31. This makes it possible to reliably specify the position and shape of the pointed out object P and manage the corrective measures for the pointed out object P. Therefore, in the next safety patrol, it is possible to accurately determine whether or not corrective measures have been taken for the pointed out object P from the image of the photographing device 12 and the photograph of the mobile terminal 31.

[0046] Although several embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, changes, and combinations can be made without departing from the spirit of the invention. Furthermore, these substitutions, changes, and combinations are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0047] Reference Signs List 10...Safety patrol findings management support system, 11...Mobile terminal, 12...Photographing device, 13...Platform, 15...Viewing terminal, 16...Registration function, 17...Automatic form creation function, 18...Speech to text conversion function, 19...Location information calculation function, 20...Results position linking function, 21...Area classification function, 22...Remaining item extraction function, 23...Results position selection function, 25...Database, 26...Results position plot function, 30...Safety patrol findings management support system, 31...Mobile terminal, 32...Platform, 33...Results object position estimation function, M...Correspondence, P...Results object, R...Movement route, S...Direction

Claims

1. A mobile device carried by a person in charge of safety patrols at the site to input safety observations at the site; A photographing device carried by the person in charge to capture images of the surroundings during the safety patrol; A platform for registering, storing, and managing information regarding the findings, including the findings; a viewing terminal for viewing information regarding the indicated matters managed by the platform, The platform includes a registration function unit that registers and stores information about the identified issues in a database, and a form creation function unit that creates and outputs information about the identified issues as a form. A safety patrol issue management support system further comprising at least one of a voice-to-text conversion function unit that converts voice input into the mobile terminal into text; a position information calculation function unit that calculates the position of the camera from the image captured by the camera; an issue position linking function unit that identifies the position of the issue based on the position of the camera and links the position of the issue with the content of the issue; an area-based classification function unit that classifies information about the issues by management area and stores it in the database; and a remaining issue extraction function unit that extracts the issues that require action from the database and displays them.

2. The safety patrol issue management support system of claim 1, characterized in that the location information calculation function unit is configured to calculate the location of the camera in three-dimensional coordinates in time series using self-location estimation technology from the surrounding image captured by the camera.

3. The safety patrol issue management support system of claim 2, characterized in that the issue position linking function unit is configured to calculate the position of the issue as a position within a specified distance in a time-series three-dimensional coordinate system representing the position of the imaging device before and after registration based on the time information when the issue was registered on the platform, and to further link this position of the issue with the content of the issue.

4. The safety patrol issue management support system of claim 3, characterized in that the area classification function unit is configured to classify information regarding issues in which the content and location of the issues are linked, by management area based on the location of the issues, and store it in the corresponding hierarchy of the database.

5. The safety patrol issue management support system as described in claim 4, characterized in that the remaining extraction function unit is configured to extract and display information regarding issues stored in each hierarchy of the database, including issues that require action, for each management area.

6. The platform further comprises an indication location selection function; The safety patrol issue management support system of claim 1, characterized in that this issue location selection function unit is configured to, when the location where an issue has occurred is selected on the layout plan of the mobile terminal, assign a position in the plant coordinate system to the issue by converting the location where the issue has occurred into the plant coordinate system based on the correspondence between the layout plan coordinate system and the plant coordinate system that is stored in advance.

7. The mobile terminal is configured to be able to acquire point cloud data, and the platform further includes a pointed out object position estimation function unit that estimates a position of a pointed out object that is a subject of a pointed out item, The safety patrol issue management support system of claim 1, characterized in that the pointed out object position estimation function unit is configured to estimate the position of the mobile terminal based on the time information attached to the surrounding image captured by the imaging device and the surrounding point cloud data captured by the mobile terminal, and to estimate the orientation of the mobile terminal from the point cloud data of the pointed out object acquired by the mobile terminal, and to calculate the position of the pointed out object in three-dimensional coordinates from the position and orientation of the mobile terminal.

8. The safety patrol issue management support system of claim 7, characterized in that the database is configured to register the location of the pointed out object together with a photograph of the pointed out object taken by a mobile terminal, an image of the pointed out object taken by a camera, and shape data obtained from point cloud data of the pointed out object obtained by the mobile terminal.

9. The mobile devices carried by the on-site safety patrol personnel assist in inputting safety-related points at the site, The camera carried by the person in charge captures images of the surroundings during the safety patrol, The platform registers, stores and manages information regarding the findings, including the findings; A viewing terminal supports viewing of information related to the indicated matters managed by the platform; The platform performs a registration step of registering and storing information about the identified matters in a database, and a form creation step of creating and outputting information about the identified matters as a form, A safety patrol issue management support method, further comprising at least one of the following steps: a voice-to-text conversion step for converting voice input into text at the mobile terminal; a position information calculation step for calculating the position of the camera from the image captured by the camera; a notice position linking step for identifying the position of the notice based on the position of the camera and linking the position of the notice with the content of the notice; an area-by-area classification step for classifying information regarding the notices by management area and storing it in the database; and a remaining item extraction step for extracting the notices requiring action from the database and displaying them.

Citation Information

Patent Citations

  • Information managing method and device, program for making computer implement information managing method, recording medium recording the program, safety patrol support device and system

    JP2004005380A

  • Information processing system, terminal device, server, information processing method and program

    JP2017084050A

  • Site video management system and site video management method

    JP2022021009A

  • Business management support device, business management support system, business management support method, and business management support program

    JP2022155553A

  • Inspection system and related methods

    US20160055602A1