Information processing device and information processing method

Abstract

To provide an information processing apparatus capable of detecting a plurality of risk factors at a work site.SOLUTION: The information processing apparatus comprises: an acquisition unit which acquires an image of a work site provided with a prescribed facility; and a processing unit which inputs the acquired image to a learning model trained so as to, in response to input of the image of the work site, output a first item corresponding to a risk factor of the prescribed facility in the input image, a second item corresponding to a risk factor of an environment of the work site in the input image, and a third item corresponding to a risk factor of movement of an observation object at the work site in the input image out of a plurality of items and causes the learning model to output items corresponding to risk factors in the acquired image out of the first to third items.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present invention relates to an information processing device and an information processing device. [Background technology]

[0002] Patent Document 1 discloses a learning model that outputs an event corresponding to a dangerous location when an image including the dangerous location at a work site is input. Specifically, for example, when an image including a "crane" installed at a work site as a dangerous location is input, the learning model outputs an event of "hanging load falling." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-81549 A Summary of the Invention [Problem to be solved by the invention]

[0004] The device in which the learning model disclosed in Patent Document 1 is implemented can mechanically detect dangerous locations at a work site. However, even if dangerous locations at a work site can be identified, accidents may occur at the work site, for example, if the equipment used by workers is not maintained or if workers behave dangerously at the work site. Therefore, in order to determine whether the situation at the work site is dangerous or not, it is necessary to consider not only the dangerous locations at the work site but also other risk factors.

[0005] The present invention has been made in consideration of the above-mentioned problems in the conventional art, and has an object to provide an information processing device capable of detecting a plurality of risk factors in a work site. [Means for solving the problem]

[0006] The information processing device of the present invention for solving the above-mentioned problems includes an acquisition unit for acquiring an image of a work site where a predetermined facility is installed, and a processing unit for processing an image of the work site when the image of the work site is input. , enter In the image At the work site The first item according to the hazards of the given equipment group and a second item according to environmental hazards at the work site in the input image. group and a third item according to a risk factor of the observed action at the work site in the input image. group The acquired image is input to a learning model trained to output the first to third items. group of From inside A processing unit that outputs items corresponding to risk factors in the acquired image; an output unit that outputs a risk level for an item corresponding to a risk factor in the acquired image using data including a risk level for each of the items included in the first to third item groups; an addition unit that adds up, among the risk levels for items corresponding to risk factors in the acquired image, a first risk level for each of the items included in the first item group, a second risk level for each of the items included in the second item group, and a third risk level for each of the items included in the third item group, which is output together with the first and second risk levels and held for a predetermined period; and a determination unit that determines that the situation of the work site in the acquired image is dangerous when a result of the addition by the addition unit is greater than a predetermined value. Equipped with.

[0007] The information processing method of the present invention further includes a step of acquiring an image of a work site where a predetermined facility is installed, and when the image of the work site is input, , enter In the image At the work site The first item according to the hazards of the given equipment group and a second item according to environmental hazards at the work site in the input image. group and a third item according to a risk factor of the observed action at the work site in the input image. group The acquired image is input to a learning model trained to output the first to third items. group of From inside outputting items corresponding to risk factors in the acquired image; a step of outputting a risk level for an item corresponding to a risk factor in the acquired image using data including a risk level for each of the items included in the first to third item groups; a step of adding up, among the risk levels for the items corresponding to the risk factors in the acquired image, a first risk level for each of the items included in the first item group, a second risk level for each of the items included in the second item group, and a third risk level for each of the items included in the third item group, which is output together with the first and second risk levels and held for a predetermined period; and a step of determining that the situation at the work site in the acquired image is dangerous if the addition result is greater than a predetermined value. Includes. Effect of the Invention

[0008] According to the present invention, it is possible to provide an information processing device capable of detecting a plurality of risk factors in a work site. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an example of an information processing system 10. [Diagram 2] FIG. 2 illustrates an example of a learning device 21. [Diagram 3] FIG. 2 is a diagram showing an example of a functional block realized in a learning device 21. [Figure 4] FIG. 13 is a diagram illustrating an example of teacher data. [Diagram 5] FIG. 11 is a diagram illustrating an example of teacher data. [Figure 6] FIG. 11 is a diagram illustrating an example of teacher data. [Figure 7] FIG. 13 is a diagram illustrating an example of teacher data. [Figure 8] 13 is a flowchart showing an example of a generation process of the learning model 41. [Figure 9] FIG. 2 is a diagram showing an example of items output from the learning model 41. [Figure 10] FIG. 2 is a diagram illustrating an example of a determination device 22. [Figure 11] FIG. 2 is a diagram showing an example of risk level data 70. [Figure 12] FIG. 2 is a diagram showing an example of a functional block realized in a determination device 22. [Figure 13] 13 is a flowchart showing an example of a determination process S30. [Figure 14] 13 is a flowchart showing an example of a risk score calculation process S44. [Figure 15] FIG. 13 is a diagram showing an example of a transition of a risk score. [Figure 16] FIG. 13 is a diagram illustrating an example of teacher data. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] At least the following points will become apparent from the description of this specification and the accompanying drawings.

[0011] =====This embodiment===== 1 is a diagram showing the configuration of an information processing system 10 according to one embodiment of the present invention. The information processing system 10 is a system that detects a plurality of risk factors in a work site 15 and judges whether the situation in the work site 15 is dangerous or not, and is configured to include a camera 20, a learning device 21, a judgment device 22, and a terminal 23.

[0012] <<<Worksite 15>>> The work site 15 is, for example, a room in an office building under construction, and is provided with a door 200 in the wall and an opening 201 for accessing the space under the floor. The work site 15 is also provided with a portable work platform 210 (hereinafter referred to as "work platform 210").

[0013] The workbench 210 is a facility used when a worker installs a ceiling board or ceiling piping on, for example, a ceiling (not shown), and has a top platform 220, support legs 221, and handrails 222. The top platform 220 is a part that constitutes a foothold for a worker to perform work, and is disposed at a higher position (above) than the work floor when the workbench 210 is installed on the work floor.

[0014] The support legs 221 are ladders that support the platform 220 while allowing the worker to ascend and descend to the platform 220, and are provided at both ends of the platform 220 in the longitudinal direction. The handrails 222 are members that prevent the worker from falling from the platform 220, and a pair of handrails are provided on both sides of the platform 220 in the width direction. The work platform 210 corresponds to the "predetermined equipment." In this embodiment, the "predetermined equipment" refers to equipment used at a work site, and is therefore not limited to the work platform 210, and may be, for example, a ladder, a stepladder, a hand lift, a light base, a high-altitude work vehicle, etc.

[0015] <<<Overview of Information Processing System 10>>> Camera 20 is a small, portable device used to capture images of the conditions at work site 15. In this embodiment, camera 20 is attached to, for example, a tripod (not shown) and is placed at a position in work site 15 where it can capture images of the area where workers are working. Camera 20 may also be attached using a hook or the like to, for example, a curtain rail (not shown) attached to a wall surface at the work site.

[0016] The learning device 21 is an information processing device that constructs a learning model for detecting risk factors in the work site 15, and the determination device 22 is an information processing device that determines whether the situation in the work site 15 is dangerous or not based on an image from the camera 20 and the learning model constructed by the learning device 21. In this embodiment, a server is used as the information processing device, but this is not limited to this. Specifically, all or a part of the information processing device may be realized using virtual information processing resources provided using virtualization technology, process space separation technology, or the like, such as a virtual server provided by a cloud system. In addition, although details will be described later, all or a part of the functions provided by the information processing device may be realized by a service provided by a cloud system via an API (Application Programming Interface), for example.

[0017] The terminal 23 is a tablet on which the site supervisor performs a predetermined operation when the determination device 22 detects a risk factor. The details of the terminal 23 will be described later, but when it is determined that the work site 15 is in a dangerous situation, an alarm is displayed on the display screen of the terminal 23. The camera 20, the learning device 21, the determination device 22, and the terminal 23 are each connected via a network 24.

[0018] <<<Details of Learning Device 21>>> 2 is a diagram showing an example of a hardware configuration of the learning device 21. The learning device 21 is a computer including a CPU (Central Processing Unit) 30, a memory 31, a storage device 32, an input device 33, a display device , and a communication device .

[0019] The CPU 30 executes programs stored in the memory 31 and the storage device 32 to realize various functions of the learning device 21 .

[0020] The memory 31 is, for example, a RAM (Random-Access Memory) and is used as a temporary storage area for programs, data, and the like.

[0021] The storage device 32 is a non-volatile storage device that stores the learning program 40 and the learning model 41. When the learning device 21 is realized using a cloud server, the storage area of ​​the storage device 32 corresponds to the storage area of ​​the cloud server.

[0022] The learning program 40 is a program that is executed when constructing the learning model 41, and the learning model 41 is a model for detecting risk factors at the work site 15 from an image of the work site 15 captured by the camera 20. The details of the learning model 41 will be described later.

[0023] The input device 33 is a device that accepts commands and data input by a user, and includes an input interface such as a keyboard and a touch sensor that detects a touch position on a touch panel display.

[0024] The display device 34 is, for example, a device such as a display, and the communication device 35 exchanges various programs and data with the determination device 22 and other devices via the network 24.

[0025] <<<Functional blocks of the learning device 21>>> 3 is a diagram showing an example of functional blocks realized in the learning device 21 when the CPU 30 of the learning device 21 executes the learning program 40. In the learning device 21, an acquisition unit 50 and a learning unit 51 are realized.

[0026] The acquiring unit 50 acquires the teacher data D1 to D3 for constructing the learning model 41, which are prepared in advance.

[0027] <<Teacher data D1>> The training data D1 in Fig. 3 is data in which a predetermined correct answer label (or correct answer data) is attached to an image including equipment used at a work site. Fig. 4(a) is an example of training data D1 including a workbench that does not pose a safety problem, and Fig. 4(b) is an example of training data D1 including a workbench that poses a safety problem.

[0028] The target location in the training data D1 in FIG. 4(a) is labeled with a correct answer label 300 of “work platform” indicating the target equipment, a correct answer label 301 of “ladder” used to estimate the height of the equipment, and a correct answer label 302 of “handrail safety” indicating a safety handrail.

[0029] In this embodiment, six correct labels 301 are attached to each of the first to third steps of the left and right support legs of the workbench, but for example, the correct labels may be attached only to any of the parts. However, by attaching the correct labels 301 to each of the first to third steps of the support legs, the learning model can be trained with higher accuracy. Although details are omitted, the correct labels in this embodiment include the coordinates of the range (i.e., the bounding box) of the target location.

[0030] The training data D1 in FIG. 4(b) is data for learning about an unsafe work platform that does not have a handrail along the longitudinal direction of the platform. The target location of the training data D1 in FIG. 4(b) is labeled with the above-mentioned correct answer label 300 of "work platform", the correct answer label 301 of "ladder", and the correct answer label 303 of "handrail unsafe" indicating an unsafe handrail (unsafe handrail). Note that FIG. 4(b) is an example of training data for learning about unsafe work platforms, and various images, such as images with a bent handrail, can be used as training data.

[0031] <<Teacher data D2>> The teacher data D2 in Fig. 3 is data in which an image including factors that are dangerous to workers in the environment of a work site is associated with the contents of the factors. In addition, factors that are dangerous to workers in the environment of a work site are, for example, a door through which workers enter and exit, and an opening in the floor. For this reason, although not shown here for convenience, in the image of the teacher data D2, the door area is given the correct answer label of "door", and the opening area is given the correct answer label of "opening".

[0032] In this embodiment, the two factors of danger to workers are doors and openings, but the present invention is not limited to these, and other factors may also be dangerous, such as steps, stacked blocks, ducts, etc. Therefore, images including these dangerous factors may be used as training data D2.

[0033] <<Teacher data D3>> The teacher data D3 in Fig. 3 is data in which a predetermined correct answer label is attached to an image including the motion of a worker to be observed. Fig. 5(a) is an example of teacher data D3 including a worker performing appropriate lifting and descending motion, and Fig. 5(b) is an example of teacher data D3 including a worker performing dangerous (inappropriate) lifting and descending motion.

[0034] In the training data D3 in Fig. 5(a), the worker's region is labeled with a correct answer label 310 of "appropriate ascent and descent." In the training data D3 in Fig. 5(b), the worker's region is labeled with a correct answer label 311 of "dangerous ascent and descent."

[0035] Moreover, Figure 6(a) is an example of teacher data D3 including a worker working on the platform in an appropriate posture, and Figure 6(b) is an example of teacher data D3 including a worker working on the platform in an inappropriate posture (with the upper body protruding from the handrail).

[0036] In the training data D3 in Fig. 6(a), the worker region is labeled with a correct answer label 312 of "appropriate posture." In the training data D3 in Fig. 5(b), the worker region is labeled with a correct answer label 313 of "upper body protruding."

[0037] Each of Figures 7(a) and 7(b) is an example of training data D3 that includes a worker working on a platform in an inappropriate posture (feet overhanging the handrail). In the training data D3 of Figures 7(a) and 7(b), the worker's region is labeled with a correct answer label 314 of "feet overhanging".

[0038] In the present embodiment, for the sake of convenience, only the correct answer level of the worker is added to the image of the teacher data D3, but the above-mentioned correct answer labels 300 to 303 of the equipment may be added to the workbench included in the image. Such an image is an image including the teacher data D1 and D3.

[0039] <<Learning model generation process S10>> 8 is a flowchart showing an example of a generation process S10 for generating a learning model. The learning unit 51 of this embodiment uses teacher data D1 to D3 to detect an object with a correct answer label when an image is input, and generates a learning model that outputs a risk factor item according to the detection result. In this embodiment, a model using a convolutional network (CNN: Convolutional Neural Network) (not shown) is used as the learning model.

[0040] 3 acquires training data D1 to D3 (S20). Then, the learning unit 51 tunes the parameters of the CNN using the training data D1 to D3 so that an object corresponding to a correct label can be detected from an image input to the CNN (S11).

[0041] As described above, the teacher data D1 is assigned with four correct labels 300-303 ("work platform", "ladder", "handrail safe", and "handrail unsafe"). The teacher data D2 is assigned with two correct labels ("door" and "opening"), and the teacher data D3 is assigned with five correct labels 310-314 ("proper ascent", "dangerous ascent", "proper posture", "upper body protrusion", and "feet protrusion"). As a result, the CNN can detect objects corresponding to the 11 correct labels from the input image.

[0042] Then, the learning unit 51 adjusts, for example, the output layer of the CNN so that the CNN can output a plurality of items according to risk factors as shown in Fig. 9 from the detection results of targets corresponding to the 11 correct labels (S12). Specifically, the learning unit 51 adjusts the CNN so that item 1A is output when a five-step support leg (i.e., a "work platform" having five "ladders") is detected, and item 2A is output when "handrail unsafe" is detected.

[0043] In this embodiment, the height of one stage of the support legs of the workbench is about 30 cm, so the height of the top platform of a workbench with five stages of support legs is about 1.5 m from the floor. When using such a workbench, there is a high risk of injury if a worker falls from the top platform. For this reason, when the CNN of this embodiment detects a workbench with five or more stages of support legs, it outputs item 1A indicating that the equipment is dangerous.

[0044] In this embodiment, when the CNN detects five support legs, it outputs item 1A. However, this is not limited to this, and item 1A may also be output based on other factors that pose a danger to workers (e.g., the support leg is broken).

[0045] Also, the learning unit 51 adjusts the CNN so that if a "door" is detected, item 2A is output, and if an "opening" is detected, item 2B is output. Furthermore, the learning unit 51 adjusts the CNN so that if a "dangerous ascent or descent" is detected, item 3A is output, if an "upper body protruding" is detected, item 3A is output, and if a "leg protruding" is detected, item 3C is output.

[0046] After the process S12 is executed, the learning unit 51 stores the adjusted CNN as the learning model 41 in the storage device 32 (S13). Therefore, the learning model 41 of this embodiment is a model that outputs items 1A and 1B corresponding to risk factors of the equipment in the image, items 2A and 2B corresponding to risk factors of the environment of the work site in the image, and items 3A to 3C corresponding to risk factors of the actions of the worker who is the object of observation in the image. Therefore, by using such a learning model 41, various risk factors at the work site can be detected.

[0047] Items 1A and 1B correspond to the "first item" corresponding to risk factors of the equipment, and the two items 2A and 2B correspond to the "second item" corresponding to risk factors in the work site environment. The three items 3A to 3C correspond to the "third item" corresponding to risk factors in the actions of the worker. In this embodiment, the number of "first items" and "second items" is two, and the number of "second items" is three, but this is not limited to these and may be at least one.

[0048] In addition, the CNN of this embodiment outputs two items, items 1A and 1B, as items indicating that the equipment is dangerous, but it may output an item indicating that the equipment is dangerous only when, for example, there are five support legs (i.e., the height of the top platform is about 1.5 m or more from the floor) and "handrail unsafe" is detected. In such a case, the CNN can appropriately detect equipment with a higher degree of danger.

[0049] <<<Details of the Determination Device 22>>> Fig. 10 is a diagram showing an example of the hardware configuration of the determination device 22 in Fig. 1. The determination device 22 is a computer including a CPU 60, a memory 61, a storage device 62, an input device 63, a display device 64, and a communication device 65. Note that each component of the CPU 60 and the like of the determination device 22 is similar to the CPU 30 and the like of the learning device 21, so detailed explanations are omitted here and the following description will focus on the information stored in the storage device 62.

[0050] The storage device 62 stores the learning model 41 generated by the learning device 21, risk level data 70, and a judgment program 71. As shown in FIG. 11, the risk level data 70 is data indicating the risk level for each of seven items 1A to 3C. In this embodiment, for example, the item 1A indicating "five or more support legs" is associated with a risk level of "1". The judgment program 71 is a program executed when judging whether the situation at the work site is dangerous or not.

[0051] Incidentally, the degree of danger of items 1A to 3C generally varies depending on the work site, the workers, etc. For example, in a work site, if the distance between the work platform and the door is very far, the worker working on the work platform is hardly affected by the door or the workers entering and exiting through the door. In such a situation, the degree of danger of the door in the work site is low.

[0052] Furthermore, if the worker assigned to the work site is an experienced worker and has a high safety consciousness, the worker is less likely to take dangerous actions. On the other hand, if an inexperienced worker is assigned to the work site, the worker is more likely to cause an accident. In this way, it is preferable to determine the value of the risk data 70 taking into consideration the environment of the work site, the experience of the worker, and the like. As will be described in detail later, the risk data 70 is set, for example, by the site supervisor operating the terminal 23. For this reason, in this embodiment, the risk levels of items 1A to 3C can be set to appropriate values ​​according to the actual degree of risk in accordance with the work site, etc. In this embodiment, the risk level data 70 is set by the site supervisor, but this is not limited to this. For example, risk level data 70 of a predetermined value may be stored in the storage device 62 in advance.

[0053] <<<Functional blocks of the determination device 22>>> 12 is a diagram showing an example of functional blocks realized in the determination device 22 when the CPU 60 of the determination device 22 executes the determination program 71. The determination device 22 realizes a receiving unit 80, an acquiring unit 81, a processing unit 82, an output unit 83, an adding unit 84, a determination unit 85, and an alarm unit 86. In the following description, an image of the work site 15 captured by the camera 20 is referred to as an image 400.

[0054] The receiving unit 80 receives the risk level data 70 set by the terminal 23 and stores it in the storage device 62. In this embodiment, the storage device 62 in which the risk level data 70 is stored corresponds to the "data storage unit." Note that the receiving unit 80 stores the risk level data 70 in the storage device 62, but the risk level data 70 may also be stored in the memory 61 or a storage area of ​​a cloud server. In this case, the portion in which the risk level data 70 is stored corresponds to the "data storage unit."

[0055] The acquisition unit 81 acquires the image 400 transmitted from the camera 20. The processing unit 82 inputs the image 400 to the learning model 41, and causes the learning model 41 to output an item corresponding to the risk factor in the image 400, from among the seven items 1A to 3C.

[0056] The output unit 83 uses the risk data 70 stored in the storage device 62 to output the risk for an item corresponding to the risk factor in the image 400. In this embodiment, the risk of an item x (x is 1A to 3C) is referred to as risk R(x). The addition unit 84 adds the risk R(x) output from the output unit 83 to calculate a risk score.

[0057] Incidentally, among the risk level R(x), the risk levels R(3A) to R(3C) caused by the worker's behavior occur only when the worker commits problematic behavior. Therefore, if the worker repeatedly commits problematic behavior within a given period (e.g., 30 minutes), the risk score decreases each time the problematic behavior subsides.

[0058] However, in an actual work site, if a worker repeatedly performs problematic behavior, the possibility of an accident occurring increases, and so it is necessary to reflect such a state in the risk score. Therefore, the addition unit 84 of this embodiment holds and adds the risk levels R(3A) to R(3C) attributable to the worker's behavior that are output during a predetermined period (e.g., 30 minutes) out of the seven risk levels R(1A) to R(3C). By performing such calculations, the addition unit 84 can output a risk score that corresponds to the degree of risk at the work site with higher accuracy.

[0059] When the danger score, which is the result of the addition by the addition unit 84, is greater than a predetermined value, the determination unit 85 determines that the situation at the work site 15 is dangerous. In addition, the warning unit 86 transmits a warning to the terminal 23 when it is determined that the situation at the work site 15 is dangerous.

[0060] <<<An example of the danger determination process S30>>> Fig. 13 is a flowchart showing an example of the risk determination process S30 mainly executed by the determination device 22. Fig. 14 is a flowchart showing an example of the risk score calculation process 44. Fig. 15 is a diagram showing an example of the transition of the risk score at the work site 15. Fig. 15 illustrates the transition of the risk score at the work site 15 between 9:00 and 16:00.

[0061] In the following, in this embodiment, it is assumed that a worker climbs onto, for example, the work platform 210 at the work site 15 shown in FIG. 1 and performs piping work on a ceiling (not shown).

[0062] First, the site supervisor of the work site 15 operates the terminal 23 to input the risk level data 70 before starting work (for example, at 8:50). The terminal 23 accepts the input risk level data 70 (S40). In this embodiment, it is assumed that the risk level data 70 shown in FIG. 11 is input by the site supervisor. As described above, the site supervisor can freely set the value of the risk level data 70 in consideration of the environment of the work site 15, the skill level of the workers, and the like. For example, in the work site 15 of FIG. 1, the distance between the door 200 and the workbench 210 is short, and the area of ​​the opening 201 is small. For this reason, the site supervisor sets the risk level of the door item 2A to "2" and the risk level of the opening item 2B to "1", which is lower than the risk level of the door.

[0063] Then, terminal 23 transmits the received risk level data 70 to determination device 22 (S41). Receiving unit 80 of determination device 22 receives risk level data 70 (S42) and stores the risk level data 70 in storage device 62 (S43). Thereafter, the worker turns on camera 20 at 9:00, which is the work start time, and begins work at work site 15. When camera 20 is turned on, determination device 22 executes a process (FIG. 14) of calculating a risk score for work site 15 based on image 400 of work site 15 transmitted from camera 20 (S44).

[0064] First, the acquisition unit 81 acquires the image 400 transmitted from the camera 20 (S50). Then, when the image 400 is acquired, the processing unit 82 inputs the image 400 to the learning model 41, and causes the learning model 41 to output items according to risk factors of the work site 15 (S51). Here, for example, at 9 o'clock, the worker has not yet performed piping work. In addition, in this embodiment, the number of steps of the support legs of the work platform 210 is three, the handrail 222 is not defective, and it is in a safe state. Therefore, at this timing, the learning model 41 outputs only the item 2A for "door" and the item 2B for "opening" among the seven items 1A to 3C in FIG. 11.

[0065] Next, the output unit 83 outputs the risk levels of the output items (S52). For example, at the time of 9 o'clock, the risk levels R(2A) and R(2B) of the two items 2A and 2B are output. The addition unit 84 then adds up the output risk levels to calculate a risk score (S53). Here, the risk level R(2A) is "2" and the risk level R(2A) is "1", so for example, the risk score at 9 o'clock is "3". The addition unit 84 then stores the calculated risk score in the storage device 62 (S54).

[0066] 13, the determination unit 85 compares the danger score with a predetermined value indicating that the work site 15 is dangerous (S45). In this embodiment, the "predetermined value" is a value ("7") set in advance by the site supervisor and stored in the storage device 26. For this reason, for example, at the time of 9 o'clock, the danger score of "3" is smaller than the predetermined value of "7" (S45: No), and the above-mentioned danger score calculation process S44 is repeated.

[0067] Then, for example, at 13:00, the worker extends the upper half of his body far out from the handrail 222 of the work platform 210. In such a case, the learning model 41 outputs item 2A for "door" and item 2B for "opening", as well as item 3B for "upper body protruding" (S51). Therefore, the output unit 83 outputs the risk levels R(2A), R(2B), and R(3B) of the three items 2A, 2B, and 3B (S52).

[0068] In this case, the adding unit 84 adds up the three output risk levels (S53), so the risk score becomes "6." In this example, the adding unit 84 holds the risk level caused by the worker's actions for 30 minutes. Therefore, of the risk score, the value based on the risk level R(3B) (here, "3") is held until 1:30 p.m.

[0069] Thereafter, for example, at 13:10, the worker puts his / her feet far out from the handrail 222 of the work platform 210. In such a case, the learning model 41 outputs the item 2A for the "door" and the item 2B for the "opening", as well as the item 3C for the "feet sticking out" (S51). Therefore, the output unit 83 outputs the risk levels R(2A), R(2B), and R(3C) of the three items 2A, 2B, and 3C (S52). At this time, the adding unit 84 adds the risk level R(3B) related to the behavior among the risk levels output at 13:00 to the three risk levels output this time (S53), so the risk score becomes "8".

[0070] When the danger score becomes "8", the judgment unit 85 judges that the danger score is greater than a predetermined value ("7") (S45: Yes). Therefore, the alarm unit 86 transmits an alarm to the terminal 23 (S46). When the terminal 23 receives the alarm, it displays the alarm on a display screen (not shown) of the terminal 23. As a result, the site supervisor can understand that the work site 15 is in a dangerous state. After "Yes" is selected in process S45, process S44 is executed. Therefore, the judgment device 22 continues to calculate the danger score of the work site 15.

[0071] In this embodiment, when the predetermined period of 30 minutes has passed since 13:00, when the worker puts his / her upper body widely out, the risk level of item 3C is cleared, and the risk score is reduced from "8" to "5." Furthermore, when the predetermined period of 30 minutes has passed since 13:10, when the worker puts his / her legs widely out, the risk level of item 3B is cleared, and the risk score is reduced from "5" to "3."

[0072] In this way, the determination device 22 of this embodiment determines the work site based on the image 400 from the camera 20. It can be objectively determined whether the situation at the work site 15 is dangerous or not. Note that the support legs 221 of the work platform 210 at the work site 15 have three steps, which is less than five steps, and therefore the height of the work platform 210 is lower than, for example, 1.5 m. For this reason, item 1A is not output here. Also, the handrails 222 of the work platform 210 are safe, and therefore item 1B is not output.

[0073] However, if the support legs 221 of the work platform 210 have five or more steps and the handrails 222 are in an unsafe state, items 1A and 1B are output. In such a case, the risk level for each of items 1A and 1B corresponds to the "first risk level." The risk level for each of items 2A and 2B corresponds to the "second risk level," and the risk level for each of items 3A to 3C corresponds to the "third risk level."

[0074] <<Another example of training data D1>> Fig. 16 is a diagram showing an example of teacher data D1 including a work platform that has no safety issues. The target location of teacher data D1 in Fig. 16 is labeled with a correct answer label 300 of "work platform" indicating the target equipment, a correct answer label 302 of "handrail safety" indicating a safe handrail, and a correct answer label 320 of "ladder" used to estimate the height of the equipment.

[0075] Since the objects with the same reference numerals are the same in FIG. 16 and FIG. 4 described above, the correct label 320 will be described here. Here, the correct label 320 "ladder" is attached to a pair of objects, which are two steps of the left and right support legs of a work platform. When the teacher data D1 with such a correct label 320 is used, the determination device 22 can, for example, more accurately distinguish between rectangular steel materials and support legs, which are common in general construction sites.

[0076] ===Summary=== The information processing system 10 according to this embodiment has been described above. The determination device 22 uses the learning model 41 that is trained using the teacher data D1 to D3. Therefore, the determination device 22 can detect a plurality of risk factors in a work site.

[0077] Furthermore, the output unit 83 uses the risk level data 70 to output the risk level of an item corresponding to the risk factor in the image 400 (for example, S52 in FIG. 14). Therefore, by referring to such risk levels, the site supervisor can more objectively determine whether or not the work site 15 is in a dangerous situation.

[0078] Furthermore, the determination unit 85 determines whether or not the work site 15 is dangerous based on the degree of danger of the item corresponding to the risk factor in the image 400 (for example, S45 in FIG. 13). Therefore, the site supervisor can immediately determine whether or not the work site 15 is in a dangerous situation based on the determination result.

[0079] As described above, the adding unit 84 holds and adds the risk level corresponding to the behavior of the worker among the items corresponding to risk factors in the image 400 for 30 minutes. In general, if the number of times a worker performs a dangerous behavior increases in a predetermined period, the possibility of the worker being injured increases. The determination device 22 of this embodiment can reflect such behavior that is likely to lead to injury of the worker in the risk score, and therefore can more accurately detect that the workplace 15 is in a dangerous state.

[0080] Moreover, the risk levels according to items 1A to 3C vary greatly depending on where in the work site the equipment used by the worker is located, as well as the experience of the worker, etc. In this embodiment, the site supervisor can set the risk level before starting work, so that the determination device 22 can evaluate the risk of the work site 15 based on a risk level that is more suited to the actual situation of the work site.

[0081] In the present embodiment, the determination unit 85 determines that the work site 15 is dangerous when the danger level score is greater than a predetermined value, but this is not limited to the above. For example, the determination unit may immediately determine that the work site 15 is dangerous when a predetermined item (e.g., item 3B) is output from among the seven items 1A to 3C. Even with this configuration, the site supervisor can grasp the danger of the work site 15.

[0082] In addition, in this embodiment, the determination device 22 assumes that the "observation target" is a worker, and outputs items 3A to 3C corresponding to risk factors based on the worker's actions, but is not limited to this. For example, the "observation target" may be equipment or tools (e.g., drills, gas burners) that are installed and operated at the work site 15. For such equipment, etc., by using teacher data that includes safe actions and unsafe actions as correct answer labels and generating a learning model, the determination device can detect risk factors caused by the operation of the equipment. Even when such equipment, etc. is the observation target, the determination device can detect risk factors at the work site.

[0083] In this embodiment, the work site is a room in a building, but is not limited thereto and may be outdoors. In such a case, for example, the "equipment" at the work site may be a crane, a backhoe, a concrete mixer, etc. Furthermore, when the work site is outdoors, items of risk factors according to the "environment" of the work site may be, for example, utility poles, high-voltage lines, trees, roads, etc. Therefore, when a determination device similar to that of this embodiment is used at an outdoor work site, risk factors at the outdoor work site can be detected.

[0084] The above-mentioned embodiment is for the purpose of facilitating understanding of the present invention, and is not intended to limit the present invention. Furthermore, the present invention can be modified or improved without departing from the spirit of the present invention, and it goes without saying that the present invention includes equivalents thereof. [Explanation of symbols]

[0085] 10. Information Processing Systems 15 Work Site 20 Camera 21 Learning Device 22 Judgment device 23 Terminals 30,60 CPU 31,61 Memory 32,62 Storage device 33,63 Input Devices 34,64 Display device 35,65 Communication equipment 40 Study Programs 41 Learning Model 50 Acquisition Department 51 Learning Department 70 Risk Data 71 Judgment Program 80 Receiving section 81 Acquisition Department 82 Processing section 83 Output section 84 Addition section 85 Judgment section 86 Alarm section 200 Doors 201 Opening 210 Portable workbench (workbench) 220 Tendai 221 Support leg 222 Handrail 300~303,310~315,320 Correct label D1~D3 Teacher data

Claims

1. An acquisition unit that acquires an image of a work site where the predetermined equipment is installed; a processing unit which inputs an acquired image to a learning model which has been trained to output, when an image of a work site is input, a first group of items corresponding to risk factors of a specific piece of equipment at the work site in the input image, a second group of items corresponding to risk factors of the environment at the work site in the input image, and a third group of items corresponding to risk factors of the action of an observed object at the work site in the input image, and causes the learning model to output an item corresponding to the risk factor in the acquired image from the first to third groups of items; an output unit that uses data including a risk level for each of the items included in the first to third item groups to output a risk level for each of the items corresponding to a risk factor in the acquired image; an adder that adds together a first risk level for each of the items included in the first group of items, a second risk level for each of the items included in the second group of items, and a third risk level for each of the items included in the third group of items, among the risk levels for items corresponding to risk factors in the acquired image, the third risk level being output together with the first and second risk levels and held for a predetermined period of time; a determination unit that determines that a situation of the work site in the acquired image is dangerous when a result of the addition by the addition unit is greater than a predetermined value; An information processing device comprising:

2. 2. The information processing device according to claim 1, A receiving unit that receives the data transmitted from the terminal and stores the data in a data storage unit. Information processing device.

3. 3. The information processing device according to claim 1, The observation target is a worker performing work at the work site. Information processing device.

4. acquiring an image of a work site where a predetermined facility is installed; a step of inputting the acquired image to a learning model that has been trained to output, when an image of a work site is input, a first group of items corresponding to risk factors of a specific piece of equipment at the work site in the input image, a second group of items corresponding to risk factors of the environment at the work site in the input image, and a third group of items corresponding to risk factors of the action of an observed object at the work site in the input image, and outputting an item corresponding to the risk factor in the acquired image from among the first to third groups of items; outputting a degree of risk for each item corresponding to a risk factor in the acquired image by using data including a degree of risk for each item included in the first to third item groups; a step of adding up a first risk level for each of the items included in the first group of items, a second risk level for each of the items included in the second group of items, and a third risk level for each of the items included in the third group of items, among the risk levels for items corresponding to risk factors in the acquired image, the third risk level being output together with the first and second risk levels and held for a predetermined period of time; determining that the situation of the work site in the acquired image is dangerous when the sum is greater than a predetermined value; An information processing method comprising:

Images