Underground work scoring device and scoring method

Abstract

To provide a scoring device and a method that cannot only determine safety of tunnel pit labor in real time, but allow a worker himself or herself to objectively grasp afterward whether the details and process of his or her work are correctly executed.SOLUTION: A pit labor scoring device comprises basic information registration means 2, measuring means 3, image analysis means 4, evaluation means 5, output means 6, and storage means 7. The measuring means 3 is composed of imaging means 31, environment measuring means 32, and timing means 33, and the environment measuring means 32 is composed of noise measuring means 32a, dust measuring means 32b, and vibration measuring means 32c. The image analysis means 4 comprises object detection means 41, bounding box application means 42, and a learning model 43. The evaluation means 5 comprises basic score calculation means 51, dangerous action extraction means 52, danger degree determination means 53, difficulty determination means 54, and scoring means 55.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technique for evaluating whether heavy machinery and workers can work safely in a mine such as a tunnel.

Background Art

[0002] At a work site such as a tunnel construction site, a large number of heavy machines such as excavators, wheel loaders, dump trucks, wheel jumbo drills, and spraying machines are used, and generally a large number of workers work around the heavy machines at the same time. It is necessary to prevent contact accidents between the heavy machines and the workers. For this purpose, it is beneficial to prevent accidents by issuing an alarm when it is determined that the risk of an accident is high during work. In addition, in order to prevent contact accidents between heavy machines and workers, it is also important to prevent the occurrence of such risks themselves. However, it is difficult for workers to objectively recognize how safely they are working during work. Therefore, a technique that can objectively and retrospectively grasp whether the worker himself / herself is performing his / her work content and work process correctly is desired.

[0003] As a technique for managing workers, a system for managing the working locations of workers using wearable devices is known (see Patent Document 1). This is a system for managing workers by judging the working state, working condition, health condition or environmental safety risk of the worker using information obtained from wearable devices worn on the worker such as helmets. However, the working location management system of Patent Document 1 has a problem that it is necessary for workers to wear dedicated wearable devices, and when a worker who is not wearing the wearable device approaches a heavy machine or the like, the risk cannot be detected. In addition, the working location management system of Patent Document 1 has a problem that an effect of preventing future accidents cannot be obtained by objectively and retrospectively grasping whether the worker himself / herself is performing his / her work content and work process correctly. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2019-102044 [Overview of the project] [Problems that the invention aims to solve]

[0005] In view of these circumstances, the present invention aims to provide a scoring device and method that not only allows for real-time determination of the safety of tunnel work, but also enables workers to retrospectively and objectively understand whether their work content and work process are being performed correctly. [Means for solving the problem]

[0006] To solve the above problems, the underground work scoring device of the present invention is a device for evaluating the safety of underground work in a tunnel, and comprises the following means. 1) Imaging means for photographing the inside of a tunnel. 2) A timing device for measuring the time from the start of shooting to the end of shooting. 3) Object detection means that uses a learning model that has learned the shape and rotation range of heavy machinery to detect at least heavy machinery and workers as target objects from images captured by an imaging means. 4) Bounding box assignment means for assigning a bounding box to a detected target object. 5) An evaluation means that extracts dangerous activities based on the presence and degree of contact or overlap between the bounding boxes of heavy machinery or between heavy machinery and workers, obtained by comparing the presence and degree of contact or overlap between the bounding boxes of heavy machinery or between heavy machinery and workers with the captured images before and after the activity, and scores the safety of tunnel work based on these dangerous activities. 6) A storage means for storing images captured by an imaging means in relation to time. 7) Output means for outputting captured images and scores from which dangerous behavior has been extracted.

[0007] By evaluating work safety based on the analysis results of captured images, safety can be assessed even if workers do not have access to specialized terminals. Furthermore, the quantitative evaluation as a score makes it easier for workers to understand the degree of safety and risk. 1) A wide range of known cameras can be used as the imaging means, and are not limited to monocular RGB cameras; compound RGB cameras may also be used. Using a compound RGB camera makes it easier to calculate the 3D position of heavy machinery or workers, enabling high-precision determination. Imaging may be performed by capturing moving images or by capturing still images at predetermined intervals. 2) In the timing means, the time to be measured includes the time of day. The timing means is used to measure the total shooting time, and is also used to store images captured by the imaging means in the storage means in association with the time. The timing means can evaluate all the time from the start to the end of shooting, but alternatively, it may be possible to set a specific point in time between the start and end of shooting as the start or end of measurement. This allows for accurate reflection of the actual work time during the total shooting time. Therefore, in this specification, shooting time, measurement time, and work time are basically synonymous, but if a start and end of measurement are set, the measurement time becomes the work time, and the measurement time and work time become different from the shooting time.

[0008] 3) The object detection means uses a learning model that has learned the shape and rotation range of heavy machinery to detect heavy machinery and workers as target objects from images captured by the imaging means. Here, the rotation range of heavy machinery does not mean only rotation, but also includes movements related to forward, backward, left, and right movement. The learning model is a computer program that performs machine learning, such as a deep neural network model (DNN) or a convolutional neural network (CNN). In the case of the learning model in this specification, the shape and rotation range of heavy machinery have been learned, but it may also include the type and size of heavy machinery, and movements other than rotation or movement. Furthermore, it may not be limited to heavy machinery, but may also be able to distinguish workers, safety equipment described later, or the state of dust being stirred up in the tunnel from the background image. It is possible to determine whether or not someone is a worker based on their average body type, and for safety equipment, it is possible to determine whether it is dangerous or safe based on the type of safety equipment, size, and connection status with other safety equipment. Furthermore, regarding the construction site environment, it is possible to determine the presence and degree of dust from the background roughness of the image, and the presence and degree of vibration from the image blur. 4) In the bounding box application means, the method of applying the bounding box to the heavy machinery varies depending on the type and condition of the heavy machinery. For example, in the case of heavy machinery that frequently performs turning movements, the bounding box is provided wider on the left and right sides. Also, in the case of heavy machinery that frequently moves in a straight line, the bounding box is provided wider in the direction of movement. In the bounding box application means, the application of the bounding box may be done in real time or intermittently.

[0009] 5) In the evaluation method, contact of bounding boxes refers to a situation where the edges of the bounding boxes are touching each other but not overlapping, and the degree of contact is determined by the number of points of contact with each other. Furthermore, the degree of overlap is determined by the size of the overlapping area, but it may also be determined by including the degree of importance of the overlapping parts. For example, if the bounding box attached to heavy machinery and the bounding box attached to a worker overlap, the degree of overlap can be determined to be large if the part of the bounding box attached to the heavy machinery where the movable parts, such as the tip of the heavy machinery's arm, are located overlaps. Furthermore, the bounding boxes of the workers were not used in the evaluation because the risk was considered low even if there was overlap or other similarities.

[0010] Changes in the presence and degree of contact or overlap of bounding boxes are obtained by comparing them with preceding and succeeding images. However, "preceding and succeeding images" here does not mean only the image immediately before or after, but rather includes multiple images that precede or follow even further. Therefore, changes here are detected by comparing the presence and degree of contact or overlap of bounding boxes across multiple consecutive images.

[0011] 6) By storing the captured images and scores in relation to time, the memory device can objectively determine the degree of danger in the actions shown in the captured images. 7) The output means may be a system that outputs the evaluation results to a display or the like, or a system that transmits data to another terminal. When displaying the evaluation results on a display, the scores may include the overall score for the entire task, scores for each task item, scores for each worker or work group, a graph showing the change in each score over time, and scores associated with captured images at a specific point in time within the graph. Furthermore, captured images that are judged to be particularly dangerous may be highlighted.

[0012] In the underground work scoring device of the present invention, the object detection means further detects safety equipment as target objects from images captured by the imaging means, and the evaluation means further extracts dangerous acts based on the presence and degree of contact or overlap of the bounding boxes of heavy machinery and safety equipment, or worker and safety equipment, and the change in the presence and degree of contact or overlap between heavy machinery and safety equipment, or worker and safety equipment, obtained by comparing the captured images before and after, and scores the safety of underground work in the tunnel based on the dangerous acts. Since safety equipment can be detected and used in scoring, dangerous acts such as workers entering restricted areas can also be detected, enabling a more accurate evaluation. Safety equipment refers to devices installed to define restricted areas or to alert workers, and includes pylons, safety bars attached to pylons, safety lights, and signs.

[0013] In the underground work scoring device of the present invention, the evaluation means preferably comprises: a risk determination means for determining the degree of risk for extracted dangerous activities; a difficulty determination means for determining the difficulty of the work during the time period in which the dangerous activity occurred; and a scoring means for calculating a final score based on information regarding the dangerous activities for which the degree of risk and difficulty have been determined. By providing the risk determination means and the difficulty determination means, it becomes possible to determine the degree of risk according to the difficulty, enabling a more accurate evaluation. Here, dangerous activities refer to activities that endanger the life and physical safety of workers, including heavy equipment operators, due to contact between heavy equipment or activities with a high risk of such contact, contact between heavy equipment and workers or activities with a high risk of such contact, or activities in which heavy equipment or workers enter a restricted area. The risk assessment method determines the degree of risk associated with dangerous activities. The assessment is based on factors such as the number of heavy machines and workers in the captured image, the type of dangerous activity, the number of heavy machines and workers performing the dangerous activity, the severity of the dangerous activity, and the proportion of time the dangerous activity took during the measurement period. If safety equipment is detected, the presence, number, and size of restricted areas are also used in the assessment. The difficulty level determination means makes a determination based on captured images and other data analyzed by the image analysis means.

[0014] The underground work scoring device of the present invention further includes environmental measuring means for measuring environmental levels inside a tunnel, the environmental measuring means including at least one of noise measuring means for measuring sound levels inside a tunnel, dust measuring means for measuring suspended particulate matter inside a tunnel, and vibration measuring means for measuring the magnitude of vibrations inside a tunnel, and preferably the storage means stores the measurement results from the environmental measuring means in association with the time measured by the timing means. Depending on the work site environment, such as high noise levels, excessive dust, or significant vibration, the difficulty of the work may increase. Therefore, by equipping the work site with environmental measurement devices and incorporating such changes in the environment into the evaluation, accurate evaluation tailored to the site conditions becomes possible. The levels of noise, dust, or vibration in the tunnel measured by the environmental measurement devices can be used in the difficulty determination by the difficulty determination device. Furthermore, the underground work scoring device of the present invention may be configured without environmental measurement means, making it possible to evaluate work safety without measuring the environment of the work site.

[0015] The underground work scoring device of the present invention further comprises a basic information registration means for receiving registration of basic information relating to at least one of workers, heavy machinery, or the construction site environment, a storage means for further storing the basic information received by the basic information registration means, and an evaluation means further comprises a basic score calculation means for estimating the difficulty level of the entire work and calculating a basic score based on the basic information received by the basic information registration means, and preferably the scoring means for applying information on dangerous activities for which the degree of danger and difficulty level has been determined to the basic score and calculating a final score. By incorporating a basic information registration mechanism, a more accurate safety assessment becomes possible, taking into account the conditions of workers, heavy machinery, and the construction site environment. The basic information received by the basic information registration mechanism can be used not only for calculating the basic score but also for determining the difficulty level in the difficulty level determination mechanism. Note that the underground work scoring device of the present invention may not include a basic information registration means, and it is also possible to evaluate the work safety without registering the basic information.

[0016] In the evaluation means of the underground work scoring device of the present invention, for the extracted dangerous acts, within the measurement time, the time associated with the captured image in which the dangerous act was first detected is taken as the start time of the dangerous act, and the time associated with the captured image immediately before the captured image in which the dangerous act was no longer detected is taken as the end time of the dangerous act. This makes it possible to accurately determine the time when the dangerous act was performed.

[0017] The underground work scoring method of the present invention is a method for evaluating the safety of tunnel underground work, and includes the following steps. A) An imaging step of photographing the inside of the tunnel. B) An object detection step of detecting at least the heavy machine and the worker as target objects from the image captured in the imaging step using a learning model that has learned the shape and turning operation range of the heavy machine. C) A bounding box assigning step of assigning a bounding box to the detected target objects. D) An evaluation step of extracting dangerous acts based on the presence or absence and degree of contact or overlap between the bounding boxes of the heavy machines or between the bounding box of the heavy machine and the worker, and the change in the presence or absence and degree of contact or overlap between the bounding boxes of the heavy machines or between the bounding box of the heavy machine and the worker obtained by comparing with the previous and subsequent captured images, and scoring the safety of the tunnel underground work based on the dangerous acts. E) An output step of outputting the captured image and the score in which the dangerous act was extracted.

[0018] In the underground work scoring method of the present invention, it is preferable that the object detection step further detects safety equipment as target objects from the image captured in the imaging step, and the evaluation step further extracts dangerous acts based on the presence and degree of contact or overlap of the bounding boxes of heavy machinery and safety equipment, or worker and safety equipment, and the change in the presence and degree of contact or overlap of heavy machinery and safety equipment, or worker and safety equipment, obtained by comparing the captured images before and after, and scores the safety of underground work in the tunnel based on the dangerous acts.

[0019] In the underground work scoring method of the present invention, the evaluation step preferably comprises a risk determination step of determining the degree of risk for the extracted hazardous activity, a difficulty determination step of determining the difficulty of the work during the time period in which the hazardous activity occurred, and a scoring step of calculating a final score based on the information regarding the hazardous activity for which the degree of risk and difficulty have been determined.

[0020] The underground work scoring method of the present invention further includes an environmental measurement step for measuring environmental levels inside a tunnel, wherein the environmental measurement step preferably includes at least one of the following: a noise measurement step for measuring sound levels inside the tunnel; a dust measurement step for measuring suspended particulate matter inside the tunnel; and a vibration measurement step for measuring the magnitude of vibrations inside the tunnel.

[0021] The underground work scoring method of the present invention further comprises a basic information registration step for receiving registration of basic information relating to at least one of workers, heavy machinery, or the construction site environment; the evaluation step further comprises a basic score calculation step for estimating the difficulty level of the entire work and calculating a basic score based on the basic information received in the basic information registration step; and the scoring step preferably applies information regarding dangerous activities for which the degree of danger and difficulty level has been determined to the basic score and calculates a final score.

[0022] In the evaluation step of the underground work scoring method of the present invention, it is preferable that the time associated with the first image detected for an extracted hazardous activity within the measurement time is considered the start of the hazardous activity, and the time associated with the image immediately preceding the image that is no longer detected is considered the end of the hazardous activity. [Effects of the Invention]

[0023] The scoring device and method of the present invention not only allow for real-time assessment of the safety of tunnel work, but also enable workers to objectively and retrospectively understand whether their work content and process are being performed correctly, thereby preventing future accidents. [Brief explanation of the drawing]

[0024] [Figure 1] Functional block diagram of the underground work scoring device of Example 1 [Figure 2] Image illustrating the use of the underground work scoring device in Example 1. [Figure 3] Schematic flowchart of the underground work scoring method in Example 1 [Figure 4] Image diagram illustrating the analysis of captured images inside a tunnel. [Figure 5] Work Safety Evaluation Flowchart [Figure 6] Output diagram (1) of the underground work scoring device of Example 1 [Figure 7] Output diagram (2) of the underground work scoring device of Example 1 [Figure 8] Image diagram illustrating the extraction of dangerous behaviors. [Modes for carrying out the invention]

[0025] Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. It should be noted that the scope of the present invention is not limited to the following embodiments or illustrated examples, and numerous modifications and variations are possible. [Examples]

[0026] Figure 1 shows a functional block diagram of the underground work scoring device of Example 1. As shown in Figure 1, the underground work scoring device 1 consists of a basic information registration means 2, a measurement means 3, an image analysis means 4, an evaluation means 5, an output means 6, and a storage means 7. Basic information registration method 2 accepts registration of basic information relating to at least one of the following: workers, heavy machinery, or the construction site environment. Basic information about workers includes the number of workers scheduled to work at the construction site, the experience level of individual workers and the entire work group, and information about the planned work content. Here, the planned work content includes not only specific tasks such as excavation and soil transport, but also attribute information such as whether the work is performed at high altitudes or low altitudes. Basic information about heavy machinery includes not only information about the type of heavy machinery (such as slewing or straight-driving types) and the number of units, but also information about the workers who operate the heavy machinery. The type of heavy machinery includes not only the specific name of the heavy machinery, but also attribute information related to its mobility characteristics, such as whether it is slewing or straight-driving. Basic information regarding the construction site environment includes information such as the size of the tunnel, temperature, presence or absence of dust, and the presence, number, and size of restricted areas.

[0027] The measurement means 3 consists of an imaging means 31, an environmental measurement means 32, and a timing means 33. The environmental measurement means 32 consists of a noise measurement means 32a, a dust measurement means 32b, and a vibration measurement means 32c. The imaging means 31 is used to photograph the inside of the tunnel. The imaging means 31 is not limited to a monocular RGB camera; a compound RGB camera may also be used. Using a compound RGB camera makes it easier to calculate the three-dimensional position of heavy machinery or workers, enabling highly accurate determination. The environmental measurement means 32 measures the degree of noise, dust, or vibration inside the tunnel. The noise measurement means 32a measures the sound level inside the tunnel, and a known sound level meter is preferably used. The dust measurement means 32b measures the amount of suspended particulate matter (dust) inside the tunnel, and a known dust meter is preferably used. The vibration measurement means 32c measures the magnitude of vibration inside the tunnel, and a known vibration meter is preferably used. Measurements by the environmental measurement means 32 may be performed in real time or intermittently, such as every second. Furthermore, the environmental measurement means 32 may also include means for measuring the temperature inside the tunnel. The timing means 33 measures the time from the start of shooting to the end of shooting. The timing means 33 is used to measure the total shooting time, and is also used to store images captured by the imaging means 31 and numerical values ​​measured by the environmental measurement means 32 in the storage means 7 in association with the time.

[0028] The image analysis means 4 analyzes the images captured by the imaging means 31, and the analysis results are used to evaluate the safety of the workers. The image analysis means 4 includes an object detection means 41, a bounding box assignment means 42, and a learning model 43. The object detection means 41 uses a learning model 43 to detect heavy machinery, workers, or safety equipment as target objects from images captured by the imaging means 31. The bounding box application means 42 applies a bounding box to the detected target object. The method of applying the bounding box to heavy machinery in the bounding box application means 42 varies depending on the type and condition of the heavy machinery. For example, in the case of heavy machinery that performs many turning movements, the bounding box is provided wider on the left and right. Also, in the case of heavy machinery that moves mostly in a straight line, the bounding box is provided wider in the direction of movement. In the bounding box application means 42, the application of the bounding box may be performed in real time or intermittently, such as every second. The learning model 43 has learned the shape and rotational range of heavy machinery. The rotational range does not only refer to rotation, but also includes movements related to forward, backward, left, and right movement.

[0029] The evaluation means 5 scores the safety of the work based on the basic information received by the basic information registration means 2, the results of the image analysis means 4, specifically the presence and degree of contact or overlap between bounding boxes relating to heavy machinery and heavy machinery, heavy machinery and workers, heavy machinery and safety equipment, or workers and safety equipment, the changes in the presence and degree of contact or overlap between bounding boxes relating to heavy machinery and heavy machinery, heavy machinery and workers, heavy machinery and safety equipment, or workers and safety equipment, obtained by comparing the captured images before and after, and the environmental measurement results from the environmental measurement means 32. The evaluation means 5 comprises a basic score calculation means 51, a dangerous activity extraction means 52, a risk level determination means 53, a difficulty level determination means 54, and a scoring means 55. The base score calculation means 51 estimates the difficulty level of the entire task and calculates a base score based on the basic information received by the basic information registration means 2. The hazardous activity extraction means 52 extracts hazardous activities during work based on the captured images after analysis. Specifically, as described above, it extracts hazardous activities based on the presence and degree of contact or overlap between bounding boxes relating to heavy machinery and heavy machinery, heavy machinery and workers, heavy machinery and safety equipment, or workers and safety equipment, and the changes in the presence and degree of contact or overlap between bounding boxes relating to heavy machinery and heavy machinery, heavy machinery and workers, heavy machinery and safety equipment, or workers and safety equipment, obtained by comparing these with the captured images before and after the image. In contrast to this embodiment, the degree of contact or overlap between bounding boxes and their changes may not be used as a criterion for extracting hazardous activities in the hazardous activity extraction means 52, but only as a criterion for judgment in the hazard determination means 53, in order to simplify the processing in the hazardous activity extraction means 52.

[0030] The risk assessment means 53 determines the degree of risk for the dangerous activities extracted by the dangerous activity extraction means 52, and makes the determination based on the number of workers and heavy machinery in the captured image, the presence, number, and size of restricted areas, the type of dangerous activity, the number of people who performed the dangerous activity, the degree of the dangerous activity, the proportion of time the dangerous activity took during the measurement period, etc. The difficulty level determination means 54 determines the difficulty level of the work during the time period in which the dangerous activity occurs, and makes the determination based on the basic information received by the basic information registration means 2, the captured images analyzed by the image analysis means 4, the degree of noise, dust, or vibration inside the tunnel measured by the environmental measurement means 32, etc. The scoring means 55 applies information about the dangerous activity, whose degree of danger and difficulty has been determined, to the base score to calculate the final score.

[0031] Output means 6 outputs the evaluation results regarding work safety performed by evaluation means 5, specifically outputting captured images and scores from which dangerous activities were extracted. The storage means 7 stores the basic information received by the basic information registration means 2, the images captured by the imaging means 31, the analysis results of the captured images by the image analysis means 4, the environmental measurement results by the environmental measurement means 32, and the safety score of the work by the evaluation means 5. The images captured by the imaging means 31, the environmental measurement results by the environmental measurement means 32, and the score by the evaluation means 5 are stored by the timing means 33 in association with the time, such as the number of minutes since the start of shooting. Therefore, when evaluating safety, the evaluation means 5 can determine the degree of danger and difficulty by referring to the environmental measurement results stored in association with the time, and the evaluation result score is also stored in association with the time or date.

[0032] Figure 2 shows an illustrative diagram of the underground work scoring device of Example 1 in use. As shown in Figure 2, the underground work scoring device 1 is used at work sites such as tunnel construction sites. Inside tunnel 16, heavy machinery (11a, 11b) and workers (9a, 9b) are performing their respective tasks. Furthermore, pylons (12a-12c) are installed inside tunnel 16, with safety bars 13a attached to pylons 12a and 12b, and safety bars 13b attached to pylons 12b and 12c, forming a safety device 14 as a whole, thereby establishing a restricted area. Although only two pieces of heavy machinery (11a, 11b) and two workers (9a, 9b) are shown here, in reality, on-site work involves a much larger number of pieces of heavy machinery and workers, and they frequently enter and exit the site. Furthermore, regarding safety equipment 14, more pylons and safety bars may be installed, or other safety equipment may be used.

[0033] The imaging means 31 captures images from the opening side towards the tunnel face 16a side inside the tunnel 16. The images captured by the imaging means 31 are transmitted to the underground work scoring device main unit 10, where image analysis is performed and used for safety evaluation. The safety evaluation results are displayed on the display 60. In this example, the imaging means 31, the underground work scoring device main unit 10, and the display 60 are all connected by wires, but they may also be connected wirelessly. The underground work scoring device body 10 is equipped with noise measuring means 32a, dust measuring means 32b, and vibration measuring means 32c (not shown in the figure) to measure the underground environment of the tunnel 16.

[0034] Figure 3 shows a schematic flowchart of the underground work scoring method in Example 1. As shown in Figure 3, first, before measurement, the basic information registration means 2 accepts the registration of basic information (step S01). The basic information accepted here is used for both image analysis by the image analysis means 4 and safety evaluation of the work by the evaluation means 5.

[0035] Next, images of the inside of the tunnel are taken using the imaging means 31 (step S02). The images may be taken in real time or intermittently, such as every second. Furthermore, the environment inside the tunnel is measured using an environmental measurement means 32 consisting of a noise measurement means 32a, a dust measurement means 32b, and a vibration measurement means 32c (step S03). It is preferable that steps S02 and S03 are performed simultaneously.

[0036] Based on the learning model 43, the target object is detected from the image captured by the imaging means 31 (step S04). A bounding box is assigned to the target object (step S05). Figure 4 shows an image diagram of the analysis of captured images inside the tunnel. As shown in Figure 4, the captured image 8 shows the inside of the tunnel 16, and heavy machinery (11a, 11b), workers (9a-9c), and safety equipment 14 are detected by the object detection means 41. Then, the bounding box assignment means 42 assigns a bounding box 15d to heavy machinery 11a and a bounding box 15e to heavy machinery 11b. In addition, a bounding box 15a is assigned to worker 9a, a bounding box 15b to worker 9b, and a bounding box 15c to worker 9c. The bounding boxes assigned to the heavy machinery (11a, 11b) are assigned taking into account the possibility of rotation and movement of the heavy machinery. For the sake of explanation, bounding boxes assigned to heavy machinery are shown as solid lines, while bounding boxes assigned to workers are shown as dashed lines. Furthermore, bounding boxes assigned to workers deemed to pose a hazard are shown as bold dashed lines.

[0037] Next, the safety of the work performed by the workers is scored using the evaluation means 5 (step S06). Figure 5 shows the safety evaluation flow section for the work. As shown in Figure 5, the safety of the work performed by the workers is scored by first using the basic score calculation means 51 to calculate a basic score based on the basic information about the workers, heavy machinery, and construction site environment received by the basic information registration means 2 (step S61). The basic score may be set only for the overall score, or it may be set for each worker, each work group, or each work content. Note that the calculation of the basic score in step S61 may be performed after the acceptance of the registration of basic information (step S01), for example, before the start of filming.

[0038] Regarding the calculation method for the base score, if the difficulty level is determined to be high, the base score will be set higher, and conversely, if the difficulty level is determined to be low, the base score will be set relatively lower. For example, regarding basic information about workers, a higher base score is assigned if the difficulty of the planned work is high, such as if it involves working at heights. Similarly, if there are many workers or heavy machines scheduled to work at the construction site, or if the experience level of individual workers or the entire work group is low, the difficulty will be judged as high, and a higher base score will be assigned. It should be noted that experience level is not used in calculating the base score, and it is also possible to perform an absolute evaluation that does not take experience level into consideration. For basic information regarding heavy machinery, the difficulty level is determined and a base score is set according to the type and number of machines. For example, if there are many machines, the difficulty level is determined to be high, and a higher base score is set. Regarding basic information about the construction site environment, if the tunnel is narrow, the temperature is high, there is a lot of dust, or there are restricted areas, the difficulty level will be judged as high and a higher base score will be set. Regarding restricted areas, if there are many restricted areas or if the restricted areas are large, the difficulty level will be judged as higher.

[0039] Next, using the risky behavior extraction means 52, high-risk behaviors during measurement, i.e., risky behaviors, are extracted based on the analyzed captured images (step S62). Dangerous behaviors are identified based on the presence and degree of contact or overlap between bounding boxes attached to heavy machinery, between bounding boxes attached to heavy machinery and bounding boxes attached to workers, or between bounding boxes attached to safety equipment and bounding boxes attached to heavy machinery or workers, and changes in the presence and degree of contact or overlap between bounding boxes attached to heavy machinery, between bounding boxes attached to heavy machinery and bounding boxes attached to workers, or between bounding boxes attached to safety equipment and bounding boxes attached to heavy machinery or workers, obtained by comparing these with the captured images before and after the event. As shown in Figure 4, first, in the captured image 8, the bounding box 15d attached to the heavy machinery 11a and the bounding box 15a attached to worker 9a are not in contact and are therefore not detected as a dangerous act. However, the bounding box 15f attached to the safety device 14 and the bounding box 15c attached to worker 9c overlap and are therefore detected as a dangerous act, and are shown in bold with a dashed line. Note that such captured image analysis images may be displayed in real time on a screen for construction managers, etc., or they may be processed internally without being displayed on a screen.

[0040] If the same dangerous act is detected in multiple consecutive captured images, the dangerous act extracted from those multiple images will be treated as a single dangerous act. If a single dangerous act is detected in a single captured image, the dangerous act detected in that image will be the extracted dangerous act. Furthermore, if a single dangerous act is detected in multiple consecutive captured images, the dangerous act treated as a single entity will be the extracted dangerous act.

[0041] Furthermore, if a dangerous act is detected in multiple consecutive captured images and treated as a single entity, the time associated with the first captured image in which the dangerous act was detected within the measurement period is considered the start time of the dangerous act, and the time associated with the image immediately preceding the image in which the dangerous act was no longer detected is considered the end time of the dangerous act. Figure 8 shows an illustrative diagram of the extraction of dangerous behaviors. In the example in Figure 8, the horizontal axis represents the passage of time during the recording process, and the case where dangerous behaviors A1 and A2 are extracted from all the captured images in the measurement is explained. Dangerous behaviors A1 and A2 in the figure represent the time when the dangerous behaviors actually occurred. Note that the captured images (8a to 8h) show only a portion of the total captured images, and although not shown, images were actually captured between images 8b and 8c, between images 8c and 8d, between images 8d and 8e, and between images 8f and 8g. Furthermore, images were continuously captured before image 8a and after image 8h.

[0042] As shown in Figure 8, for example, only dangerous behavior A1 is detected in captured image 8b, and only dangerous behavior A2 is detected in captured image 8f. In contrast, both dangerous behavior A1 and dangerous behavior A2 are detected in captured image 8d. For dangerous behavior A1, the time associated with the first image 8b in which dangerous behavior A1 was detected, when it was not yet detected in image 8a within the measurement period, is considered the start of dangerous behavior A1. The time associated with the image 8e immediately preceding the image 8f in which dangerous behavior A1 was no longer detected is considered the end of dangerous behavior A1. Similarly, for dangerous behavior A2, the time associated with the first image 8c in which dangerous behavior A2 was detected, when it was first detected within the measurement period, is considered the start of dangerous behavior A2. The time associated with the image 8g immediately preceding the image 8h in which dangerous behavior A2 was no longer detected is considered the end of dangerous behavior A2. Thus, even when multiple dangerous actions are detected in a single image, it is possible to extract the dangerous actions in a way that links them to temporal data by combining multiple images. In the case of dangerous act A2, the timing of the start and end coincides precisely with the start and end of the dangerous act, whereas in the case of dangerous act A1, they do not strictly coincide. However, by using the method of this embodiment, the time when the dangerous act was performed can be determined with high accuracy. Furthermore, if an identified hazardous act and a subsequently identified hazardous act are related to the same heavy machinery, worker, or safety equipment and occur in close proximity in time, they may be considered as a single hazardous act. Proximity in time means, for example, that one hazardous act begins within a few seconds (e.g., within 10 seconds) after the completion of the other hazardous act. In such cases, it can be judged that only a temporary interruption of the hazardous act occurred.

[0043] Furthermore, the risky behavior extraction means 52 detects whether or not there is movement of the heavy machinery within a predetermined time period based on the presence or absence of changes in the image of the heavy machinery. If there is heavy machinery for which no movement is detected within the predetermined time period, that heavy machinery is determined to be stationary and is excluded from the extraction target. As shown in Figure 4, the bounding box 15e assigned to the heavy machinery 11b and the bounding box 15b assigned to the worker 9b overlap, but since no movement is detected in the heavy machinery 11b within the predetermined time period, it is determined to be stationary and is excluded from the extraction target.

[0044] For the identified dangerous activities, the degree of danger is determined using the danger determination means 53 (step S63). The degree of danger is determined based on the type of dangerous activity, the number of heavy machinery and workers who performed the dangerous activity, the severity of the dangerous activity, the duration of the dangerous activity, etc. Here, the types of dangerous acts include the risk of contact between heavy machinery, the risk of contact between heavy machinery and workers, or the intrusion of heavy machinery or workers into restricted areas. Dangerous acts of a type that are considered to pose a high risk to the workers' bodies are more likely to be judged as high-risk. The number of heavy machinery and workers who engaged in dangerous activities refers to the number of heavy machinery and workers involved in those dangerous activities. For example, if two workers operate one piece of heavy machinery, and three workers are in close proximity to the machinery to the extent deemed dangerous, the number of heavy machinery is 1 and the number of workers is 3. If a dangerous activity continues for a certain period of time, all workers who are in close proximity to the machinery to the extent deemed dangerous during that period are counted. In contrast, even if multiple workers are captured in the same image, workers who are not involved in the dangerous activity are not counted. Also, if multiple dangerous activities are performed in the same image, the number of heavy machinery and workers will be counted for each dangerous activity. Therefore, if one worker is involved in multiple dangerous activities simultaneously, they may be counted multiple times. Furthermore, while it is possible to use the number of heavy machinery and workers who engaged in dangerous activities absolutely in the risk assessment, it is preferable to evaluate them relatively in relation to the total number of heavy machinery and the total number of workers. In other words, for example, if a large number of workers engage in dangerous behavior, the risk is more likely to be judged as high, but if the total number of workers is large, the risk is relatively more likely to be judged as low.

[0045] Furthermore, by analyzing the captured images using the image analysis means 4, heavy machinery and workers who are determined to have no or very low responsibility for a particular dangerous act may be extracted and excluded from the number of heavy machinery and workers who performed the dangerous act. For example, if one worker is operating heavy machinery and two workers are in close proximity to the heavy machinery to the extent that it is deemed dangerous, and the heavy machinery is determined to have no responsibility, then the number of heavy machinery and workers involved in that dangerous act will be 0.

[0046] The degree of danger in an action is determined based on the degree of contact or overlap between bounding boxes attached to heavy machinery, between bounding boxes attached to heavy machinery and workers, or between bounding boxes attached to safety equipment and bounding boxes attached to heavy machinery or workers, as well as changes in these conditions. Specifically, a greater degree of contact or overlap between bounding boxes is more likely to be judged as a high-risk activity. The time during which a dangerous act occurred includes not only the duration of the dangerous act but also the proportion of time the dangerous act spent within the total measured time. While the duration of a dangerous act can be used absolutely for risk assessment, it is preferable to evaluate it relatively in relation to the total measured time. In other words, a longer duration of a dangerous act tends to be judged as high risk, while a longer total measured time tends to be judged as relatively low risk.

[0047] Next, the difficulty level of the dangerous act is determined (step S64). The difficulty level is determined based on the movement of the heavy machinery, the movement of the workers, changes in the environment, and the elapsed time since the start of filming, etc. The measurement means 3 is equipped with a timing means 33, and all images captured by the imaging means 31 are associated with the imaging time and date and stored in the storage means 7. Therefore, all images captured after analysis by the image analysis means 4, and also all dangerous acts extracted using the dangerous act extraction means 52, are stored with their imaging time and date associated. In addition, the measurement results from the environmental measurement means 32 are also stored in the storage means 7 with their measurement time and date associated with the timing means 33. Therefore, the difficulty level of the dangerous act in question is determined based on the movement of the heavy machinery, the movements of the workers, changes in the environment, and the elapsed time since the start of filming, etc., at the time the dangerous act occurred.

[0048] Regarding the movement of heavy machinery or workers, the movement of heavy machinery or workers is detected from the captured images after analysis by the image analysis means 4. If the amount of movement of heavy machinery or workers per unit of measurement time is large, the risk of collisions and other accidents increases, and the difficulty level is likely to be judged as high. Environmental changes include not only the measurement results from the environmental measurement means 32, but also changes that appear in the captured images, such as the state of sediment accumulation. For example, if the passage for workers becomes narrower due to the accumulation of sediment in a certain area, it is more likely to be judged as having a high difficulty level. Regarding the environmental measurement results, the measurement results from the environmental measurement device 32 at the time the dangerous activity occurred are referred to. If the sound level inside the tunnel is high, the amount of dust is large, or the vibration is strong, it is more likely to be judged as difficult because visibility is easily obstructed and communication with the surroundings becomes difficult. Regarding the elapsed time after filming begins, the longer the elapsed time, the more fatigue accumulates among the workers, making it more likely that the task will be judged as difficult.

[0049] Using the scoring means 55, information regarding the hazardous activity whose degree of danger and difficulty has been determined is applied to the base score to calculate the final score (step S65). The final score may be calculated as the total score for all workers in the entire work, or it may be calculated for each worker or work group. Alternatively, the final score may be calculated for each work item.

[0050] Finally, the score is output to a display or the like (step S07). Figures 6 and 7 show an output image of the underground work scoring device of Example 1. The display on the display 60 shown in Figure 6 does not relate to the final score, but rather allows for retrospective confirmation of images in the captured image group that were determined to be dangerous activities. The captured images may be displayed in their entirety or only in part. As shown in Figure 6, 24 thumbnail images 80 are displayed on the display 60. Although not shown here, more thumbnail images 80 can be viewed by scrolling the screen up and down. For example, as shown in thumbnail image 80a, an image description section 81 is provided at the bottom of each image, displaying information such as the time the image was taken. Also, unlike thumbnail images (80a, 80b), thumbnail images (80c to 80e) are highlighted with a thick border. This highlighting is applied to make it easier to retrospectively identify images that have been determined to represent high-risk behavior.

[0051] The display on the display 60 shown in Figure 7 is an output image of the final score. As shown in Figure 7, the display 60 is equipped with a heavy machinery slewing range work score display unit 60a, a restricted area score display unit 60b, a high-altitude work score display unit 60c, a total score display unit 60d, and a hazardous work display unit 60e. The displayed items are not limited to the above items. The heavy machinery slewing range work score is a score that evaluates the safety of work performed by heavy machinery that involves slewing. The restricted area score is a score that evaluates safety regarding whether or not entry into restricted areas occurred. The high-altitude work score is a score that evaluates the safety of workers performing high-altitude work. The total score is a comprehensive score for all heavy machinery and all workers in the measurement. In this embodiment, the heavy machinery slewing range work score display unit 60a displays 72 points as the final score, the restricted area score display unit 60b displays 82 points, and the high-altitude work score display unit 60c displays 78 points. Furthermore, the overall score display section 60d is displayed as an arched pie chart and also shows the score as a numerical value of 78 points. The hazardous work display unit 60e displays images related to the most dangerous hazardous activity in the work in question. The selection of such images is determined based on the correlation between the degree of danger and the difficulty level of each hazardous activity. In this case, the reduced images (80c~80e) that were highlighted in Figure 6 are extracted and displayed. In this way, not only is the final score provided, but dangerous tasks can also be visually identified, making it an effective tool for post-work education and training. [Industrial applicability]

[0052] This invention is useful as a technology to improve safety in tunnel work. [Explanation of Symbols]

[0053] 1. Underground work scoring device 2. Basic Information Registration Method 3. Measurement means 4. Image analysis means 5. Evaluation methods 6 Output means 7 Memory means 8,8a~8h Acquired Images 9a~9c worker 10. Underground work scoring device main body 11a,11b Heavy equipment 12a~12c Pylon 13a,13b Safety rod 14 Safety equipment 15a~15f Bounding Box 16 Tunnel 16a Face 31 Imaging means 32 Environmental measurement means 32a Noise measurement means 32b Dust measurement means 32c Vibration measurement means 33 Timekeeping means 41 Object detection means 42 Bounding box application means 43 Learning Models 51. Basic score calculation method 52. Means for Identifying Risky Behavior 53 Risk Assessment Methods 54 Difficulty level determination method 55 Scoring Methods 60 displays 60a Heavy equipment swing range work score display unit 60b Restricted Entry Score Display Unit 60c High-altitude work score display unit 60d Overall score display section 60e Hazardous Work Indicator 80, 80a~80e Reduced Images 81 Image Description Section A1,A2 Dangerous acts

Claims

1. A device for evaluating the safety of work inside a tunnel, An imaging device for photographing the inside of a tunnel, A timing device to measure the time from the start of shooting to the end of shooting, An object detection means that uses a learning model that has learned the shape and rotation range of heavy machinery to detect at least heavy machinery and workers as target objects from images captured by an imaging means, A bounding box assignment means for assigning a bounding box to a detected target object, The presence and extent of contact or overlap between the bounding boxes of heavy machinery, or between the bounding boxes of heavy machinery and workers, By comparing the captured images before and after, we can determine the presence and degree of contact or overlap between the bounding boxes of heavy machinery, or between the bounding boxes of heavy machinery and workers. An evaluation means that extracts dangerous activities based on the criteria and scores the safety of tunnel work based on the said dangerous activities, A storage means for storing the captured images from the imaging means in relation to time, An output means that outputs the captured image from which the aforementioned dangerous behavior was extracted and the score, An underground work scoring device characterized by comprising the following features.

2. The object detection means further detects the safety device as the target object from the image captured by the imaging means. The evaluation means further, The presence and extent of contact or overlap between the bounding boxes of heavy machinery and safety equipment, or between workers and safety equipment, By comparing the captured images before and after, we can determine the presence and degree of contact or overlap between heavy machinery and safety equipment, or between workers and safety equipment. The underground work scoring device according to claim 1, characterized in that it extracts dangerous activities based on the criteria and scores the safety of underground work in the tunnel based on the said dangerous activities.

3. The evaluation means is A risk determination means for determining the degree of risk for the extracted risky behaviors, A difficulty determination means for determining the difficulty of the work during the time period in which the dangerous activity occurred, The underground work scoring device according to claim 1 or 2, further comprising a scoring means for calculating a final score based on information relating to the dangerous activity for which the degree of danger and the difficulty level have been determined.

4. The underground work scoring device further includes environmental measuring means for measuring environmental levels inside the tunnel, The aforementioned environmental measurement means includes at least one of the following: noise measurement means for measuring the sound level inside the tunnel; dust measurement means for measuring suspended particulate matter inside the tunnel; and vibration measurement means for measuring the magnitude of vibrations inside the tunnel. The underground work scoring device according to claim 1 or 2, characterized in that the storage means stores the measurement results by the environmental measurement means in association with the time measured by the timing means.

5. The underground work scoring device further comprises a basic information registration means for receiving registration of basic information relating to at least one of the following: workers, heavy machinery, or the construction site environment. The storage means further stores the basic information received by the basic information registration means, The evaluation means further comprises a basic score calculation means that estimates the difficulty level of the entire task and calculates a basic score based on the basic information received by the basic information registration means. The underground work scoring device according to claim 3, characterized in that the scoring means applies information regarding the dangerous activity, for which the degree of danger and difficulty has been determined, to the base score to calculate a final score.

6. In the evaluation means, the dangerous behavior extracted within the measurement time, The time associated with the first detected image is considered to be the start of the dangerous act. The time associated with the image immediately preceding the image that is no longer detected is considered to be the end of the dangerous act. The underground work scoring device according to claim 1 or 2.

7. A method for evaluating the safety of work inside a tunnel, The imaging step involves taking pictures inside the tunnel, An object detection step that uses a learning model that has learned the shape and rotation range of heavy machinery to detect at least the heavy machinery and workers as target objects from the image captured in the imaging step, A bounding box assignment step in which a bounding box is assigned to the detected target object, The presence and extent of contact or overlap between the bounding boxes of heavy machinery, or between the bounding boxes of heavy machinery and workers, By comparing the captured images before and after, we can determine the presence and degree of contact or overlap between the bounding boxes of heavy machinery, or between the bounding boxes of heavy machinery and workers. An evaluation step which involves extracting dangerous activities based on the criteria and scoring the safety of tunnel work based on the said dangerous activities, An output step which outputs the captured image from which the aforementioned dangerous behavior was extracted and the score, A method for scoring underground work, characterized by comprising the following:

8. The object detection step further detects the safety device as the target object from the image captured in the imaging step. The aforementioned evaluation step further, The presence and extent of contact or overlap of the bounding boxes between heavy machinery and safety equipment, or between workers and safety equipment, By comparing the captured images before and after, we can determine the presence and degree of contact or overlap between heavy machinery and safety equipment, or between workers and safety equipment. The underground work scoring method according to claim 7, characterized in that dangerous acts are extracted based on the criteria and the safety of underground work in the tunnel is scored based on the said dangerous acts.

9. The aforementioned evaluation step is, A risk assessment step is performed to determine the degree of risk for the identified risky behaviors, A difficulty determination step for determining the difficulty of the work during the time period in which the aforementioned dangerous activity occurred, The underground work scoring method according to claim 7 or 8, further comprising: a scoring step of calculating a final score based on information regarding the dangerous activity for which the degree of danger and the difficulty level have been determined.

10. The above underground work scoring method further includes an environmental measurement step of measuring environmental levels inside the tunnel, The underground work scoring method according to claim 7 or 8, characterized in that the environmental measurement step includes at least one of: a noise measurement step for measuring the sound level inside the tunnel; a dust measurement step for measuring suspended particulate matter inside the tunnel; and a vibration measurement step for measuring the magnitude of vibrations inside the tunnel.

11. The aforementioned underground work scoring method further comprises a basic information registration step that accepts registration of basic information relating to at least one of the following: workers, heavy machinery, or the construction site environment. The evaluation step further comprises a base score calculation step which estimates the difficulty level of the entire task and calculates a base score based on the basic information received in the basic information registration step. The underground work scoring method according to claim 9, characterized in that the scoring step applies information regarding the dangerous activity for which the degree of danger and difficulty have been determined to the base score, and calculates a final score.

12. In the evaluation step described above, the identified dangerous behaviors were, within the measurement time, The time associated with the first detected image is considered to be the start of the dangerous act. The time associated with the image immediately preceding the image in question, which is no longer detected, is considered to be the end of the dangerous act. The underground work scoring method according to claim 7 or 8, characterized by the features described above.

Images