Working machine and detection method

Abstract

To easily detect the presence or absence of a dangerous action and issue a warning in order to secure the safety of an operator at a work site.SOLUTION: A detection system comprises: an acquisition unit for acquiring imaging data from an imaging device that images a site; and a dangerous action determination unit for determining whether or not a person who is performing a dangerous action is present at the site on the basis of the imaging data. The dangerous action determination unit issues a warning corresponding to a type of the dangerous action abd stores data related to the dangerous action in association with image data.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present disclosure relates to Work Machine and detection methods. [Background technology]

[0002] Patent Document 1 discloses a technique related to a perimeter monitoring system that detects people around a work machine. According to the technique described in Patent Document 1, the perimeter monitoring system detects surrounding obstacles. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-035791 A Summary of the Invention [Problem to be solved by the invention]

[0004] For the safety of workers at work sites where work machines are in operation, it is important to avoid dangerous behavior such as not wearing protective gear such as helmets and safety vests, and walking while operating a smartphone. Therefore, supervisors must constantly pay attention to whether dangerous behavior is occurring at the work site, which places a heavy burden on supervisors. The objective of the present disclosure is to easily detect the presence or absence of risky behavior. Work Machine and a detection method. [Means for solving the problem]

[0005] According to a first aspect, the detection system includes an acquisition unit that acquires imaging data from an imaging device that images the scene, and a risky behavior determination unit that determines whether or not there is a person performing a risky behavior at the scene based on the imaging data. Effect of the Invention

[0006] According to the above aspect, the presence or absence of a dangerous act can be easily detected by using the detection system. [Brief description of the drawings]

[0007] [Figure 1] 1 is a schematic diagram showing the configuration of a work machine according to a first embodiment. [Diagram 2] 2 is a diagram showing the imaging ranges of a plurality of cameras equipped on the work machine according to the first embodiment. FIG. [Diagram 3] FIG. 2 is a diagram showing the internal configuration of a driver's cab according to the first embodiment. [Figure 4] 1 is a schematic block diagram showing a configuration of a control device according to a first embodiment. [Diagram 5] FIG. 4 is a diagram showing an example of information stored in risky behavior dictionary data according to the first embodiment. [Figure 6] 5 is a flowchart showing the operation of the control device according to the first embodiment. [Figure 7] FIG. 2 is a diagram showing an example of an image captured by a camera according to the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] First embodiment Hereinafter, the embodiments will be described in detail with reference to the drawings. The detection system according to the first embodiment is realized by a work machine 100 that is deployed at a site.

[0009] Configuration of the work machine 100 FIG. 1 is a schematic diagram showing the configuration of a work machine 100 according to a first embodiment. The work machine 100 operates at a construction site and works on a construction target such as earth and sand. The work machine 100 according to the first embodiment is, for example, a hydraulic excavator. The work machine 100 includes a traveling body 110, a rotating body 120, a work implement 130, and a cab 140. Note that the work machine 100 may be a work machine for mining, such as a mining excavator that operates in a mine, with the work site being a mine. The running body 110 supports the working machine 100 so that the working machine 100 can run. The running body 110 is, for example, a pair of left and right caterpillar tracks. The rotating body 120 is supported by the running body 110 so as to be capable of rotating about a rotation center. The work machine 130 is hydraulically driven. The work machine 130 is supported on the front part of the rotating body 120 so as to be drivable in the vertical direction. The operator's cab 140 is a space where an operator sits and operates the work machine 100. The operator's cab 140 is provided on the left front part of the rotating body 120. Here, the part of the rotating body 120 to which the work machine 130 is attached is referred to as the front part. Furthermore, with respect to the rotating body 120, the part opposite the front part is referred to as the rear part, the left part as the left part, and the right part as the right part.

[0010] Configuration of the rotating body 120 The revolving body 120 is provided with multiple cameras 121 that capture images of the surroundings of the work machine 100 and speakers 122 that output sound toward the outside of the work machine 100. An example of the speaker 122 is a horn speaker. Figure 2 is a diagram showing the imaging ranges of the multiple cameras 121 equipped on the work machine 100 according to the first embodiment.

[0011] Specifically, the rotating body 120 is provided with a left rear camera 121A that images a left rear area Ra of the periphery of the rotating body 120, a rear camera 121B that images a rear area Rb of the periphery of the rotating body 120, a right rear camera 121C that images a right rear area Rc of the periphery of the rotating body 120, and a right front camera 121D that images a right front area Rd of the periphery of the rotating body 120. Note that the imaging ranges of the multiple cameras 121 may partially overlap each other. The imaging ranges of the multiple cameras 121 cover the entire circumference of the work machine 100 excluding the left front area Re visible from the cab 140. Note that, although the cameras 121 according to the first embodiment capture images of the left rear, rear, right rear, and right front of the revolving body 120, other embodiments are not limited to this. For example, the number and imaging ranges of the cameras 121 according to other embodiments may be different from the examples shown in Figs. 1 and 2.

[0012] The left rear camera 121A is for imaging the left side area and the left rear area of ​​the revolving body 120 as shown in the rear range Rb in FIG. 2, but may image either one of the areas. Similarly, the right rear camera 121C is for imaging the right side area and the right rear area of ​​the revolving body 120 as shown in the right rear range Rc in FIG. 2, but may image either one of the areas. Similarly, the right front camera 121D is for imaging the right front area and the right side area of ​​the revolving body 120 as shown in the right front range Rd in FIG. 2, but may image either one of the areas. In another embodiment, the entire periphery of the work machine 100 may be the imaging range using multiple cameras 121. For example, a left front camera that images the left front range Re may be provided, and the entire periphery of the work machine 100 may be the imaging range.

[0013] Configuration of the work machine 130 The work machine 130 includes a boom 131, an arm 132, a bucket 133, a boom cylinder 131C, an arm cylinder 132C, and a bucket cylinder 133C.

[0014] The base end of the boom 131 is attached to the revolving body 120 via a boom pin 131P. The arm 132 connects the boom 131 and the bucket 133. The base end of the arm 132 is attached to the tip of the boom 131 via an arm pin 132P. The bucket 133 includes a blade for excavating earth and sand, and a storage portion for storing the excavated earth and sand. A base end portion of the bucket 133 is attached to a tip portion of the arm 132 via a bucket pin 133P.

[0015] The boom cylinder 131C is a hydraulic cylinder for operating the boom 131. A base end of the boom cylinder 131C is attached to the rotating body 120. A tip end of the boom cylinder 131C is attached to the boom 131. The arm cylinder 132C is a hydraulic cylinder for driving the arm 132. A base end of the arm cylinder 132C is attached to the boom 131. A tip end of the arm cylinder 132C is attached to the arm 132. The bucket cylinder 133C is a hydraulic cylinder for driving the bucket 133. A base end of the bucket cylinder 133C is attached to the arm 132. A tip end of the bucket cylinder 133C is attached to a link member connected to the bucket 133.

[0016] Configuration of the cab 140 FIG. 3 is a diagram showing the internal configuration of the operator's cab 140 according to the first embodiment. A driver's seat 141, an operating device 142, and a control device 143 are provided in the driver's cab 140.

[0017] The operation device 142 is a device for driving the traveling body 110, the rotating body 120, and the work machine 130 by manual operation by an operator. The operation device 142 includes a left operation lever 142LO, a right operation lever 142RO, a left foot pedal 142LF, a right foot pedal 142RF, a left traveling lever 142LT, and a right traveling lever 142RT.

[0018] The left operating lever 142LO is provided on the left side of the driver's seat 141. The right operating lever 142RO is provided on the right side of the driver's seat 141.

[0019] The left operation lever 142LO is an operation mechanism for performing the rotation operation of the revolving body 120 and the pulling / pushing operation of the arm 132. Specifically, when the operator of the work machine 100 tilts the left operation lever 142LO forward, the arm 132 performs a pushing operation. When the operator of the work machine 100 tilts the left operation lever 142LO rearward, the arm 132 performs a pulling operation. When the operator of the work machine 100 tilts the left operation lever 142LO to the right, the revolving body 120 rotates to the right. When the operator of the work machine 100 tilts the left operation lever 142LO to the left, the revolving body 120 rotates to the left. Note that in other embodiments, when the left operation lever 142LO is tilted in the front-rear direction, the revolving body 120 rotates to the right or left, and when the left operation lever 142LO is tilted in the left-right direction, the arm 132 performs a pulling operation or a pushing operation.

[0020] The right operation lever 142RO is an operation mechanism for performing an excavation / dump operation of the bucket 133 and a raising / lowering operation of the boom 131. Specifically, when the operator of the work machine 100 tilts the right operation lever 142RO forward, a lowering operation of the boom 131 is performed. When the operator of the work machine 100 tilts the right operation lever 142RO rearward, a raising operation of the boom 131 is performed. When the operator of the work machine 100 tilts the right operation lever 142RO to the right, a dump operation of the bucket 133 is performed. When the operator of the work machine 100 tilts the right operation lever 142RO to the left, an excavation operation of the bucket 133 is performed. Note that in other embodiments, when the right operation lever 142RO is tilted in the front-rear direction, the bucket 133 may perform a dump operation or an excavation operation, and when the right operation lever 142RO is tilted in the left-right direction, the boom 131 may perform a raising operation or a lowering operation.

[0021] The left foot pedal 142LF is located on the left side of the floor surface in front of the driver's seat 141. The right foot pedal 142RF is located on the right side of the floor surface in front of the driver's seat 141. The left travel lever 142LT is pivoted to the left foot pedal 142LF and configured so that the tilt of the left travel lever 142LT and the depression of the left foot pedal 142LF are linked. The right travel lever 142RT is pivoted to the right foot pedal 142RF and configured so that the tilt of the right travel lever 142RT and the depression of the right foot pedal 142RF are linked.

[0022] The left foot pedal 142LF and the left traveling lever 142LT correspond to the rotational drive of the left track of the traveling body 110. Specifically, when the operator of the work machine 100 pushes the left foot pedal 142LF or the left traveling lever 142LT forward, the left track rotates in the forward direction. Conversely, when the operator of the work machine 100 pushes the left foot pedal 142LF or the left traveling lever 142LT backward, the left track rotates in the reverse direction.

[0023] The right foot pedal 142RF and the right traveling lever 142RT correspond to the rotational drive of the right crawler belt of the traveling body 110. Specifically, when the operator of the work machine 100 pushes the right foot pedal 142RF or the right traveling lever 142RT forward, the right crawler belt rotates in the forward direction. Conversely, when the operator of the work machine 100 pushes the right foot pedal 142RF or the right traveling lever 142RT backward, the right crawler belt rotates in the reverse direction.

[0024] The control device 143 includes a display 143D that displays information related to a plurality of functions possessed by the work machine 100. The control device 143 is an example of a display system. The display 143D is an example of a display unit. The input means of the control device 143 according to the first embodiment is a hard key. Note that, in other embodiments, a touch panel, a mouse, a keyboard, or the like may be used as the input means. Also, the control device 143 according to the first embodiment is provided integrally with the display 143D, but in other embodiments, the display 143D may be provided separately from the control device 143. Note that, when the display 143D and the control device 143 are provided separately, the display 143D may be provided outside the cab 140. In this case, the display 143D may be a mobile display. Also, when the work machine 100 is driven by remote operation, the display 143D may be provided in a remote operation room provided remotely from the work machine 100. The control device 143 may be configured by a single computer, or the configuration of the control device 143 may be divided and arranged among multiple computers, and the multiple computers may function as a detection system by working together. In other words, the work machine 100 may be equipped with multiple computers that function as control devices 143. The above-mentioned single control device 143 is also an example of a detection system.

[0025] Configuration of the control device 143 FIG. 4 is a schematic block diagram showing the configuration of the control device 143 according to the first embodiment. The control device 143 is a computer that includes a processor 210, a main memory 230, a storage 250, and an interface 270.

[0026] The camera 121 and the speaker 122 are connected to the processor 210 via an interface 270 . Examples of the storage 250 include an optical disk, a magnetic disk, a magneto-optical disk, a semiconductor memory, etc. The storage 250 may be an internal medium directly connected to the bus of the control device 143, or may be an external medium connected to the control device 143 via the interface 270 or a communication line. The storage 250 stores a program for implementing monitoring of the surroundings of the work machine 100. Furthermore, a plurality of images including icons to be displayed on the display 143D are stored in advance in the storage 250.

[0027] The program may be for realizing a part of the functions to be performed by the control device 143. For example, the program may be for realizing the functions by combining with other programs already stored in the storage 250 or by combining with other programs implemented in other devices. In another embodiment, the control device 143 may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions to be realized by the processor 210 may be realized by the integrated circuit.

[0028] The storage 250 also stores person dictionary data D1 for detecting a person, and risky behavior dictionary data D2 for detecting a risky behavior. The person dictionary data D1 may be, for example, dictionary data of feature amounts extracted from each of a plurality of known images that include a person. Examples of feature amounts include Histograms of Oriented Gradients (HOG) and Co-occurrence HOG (CoHOG).

[0029] FIG. 5 is a diagram showing an example of information stored in the risky behavior dictionary data D2 according to the first embodiment. The dangerous behavior dictionary data D2 stores characteristic data and warning data indicating characteristics of a person performing a dangerous behavior according to the type of dangerous behavior to be detected. Examples of dangerous behavior include not wearing a helmet, not wearing a safety vest, not using three-point support when ascending or descending the work machine 100, walking while operating a smartphone, running, and walking with hands in pockets. Note that three-point support means supporting the body by placing one foot on the footrest of the work machine 100 and holding the handrest of the work machine 100 with both hands. The characteristic data of a person performing a dangerous behavior may be represented, for example, by a feature amount of an image, or may be represented by skeleton data indicating the skeletal posture of a person. Note that a safety vest is a work garment for safety purposes such as increasing visibility to prevent contact accidents, and is, for example, a work garment with a reflector attached. Also, safety pants with a reflector attached may be used instead of a safety vest.

[0030] The risky behavior dictionary data D2 may further store additional conditions in association with the feature data of the person performing the risky behavior. When the additional conditions are associated with the feature data, it is determined that a person performing the risky behavior exists when a feature matching the feature data is detected and the additional conditions are satisfied. When the additional conditions are not associated with the feature data, it is determined that a person performing the risky behavior exists when a feature matching the feature data is detected.

[0031] For example, data indicating characteristics related to not wearing protective equipment such as a helmet, a safety vest, or safety pants may be represented by a feature amount of an image of a person not wearing the protective equipment. Note that the feature amount may be represented by a feature amount of an image of the part where the protective equipment is worn, rather than an image of the whole body of the person. For example, data indicating characteristics related to not wearing a helmet may be represented by a feature amount of an image of the person's head, and data indicating characteristics related to not wearing a safety vest may be represented by a feature amount of an image of the person's torso.

[0032] Also, for example, feature data related to non-implementation of three-point support may be represented by a feature amount of an image of a person's head facing outward from the work machine 100. The feature data is associated with an additional condition indicating that a person is present in the vicinity of the cab 140 of the work machine 100. This is for the purpose of detecting a person jumping off the work machine 100.

[0033] Also, for example, the data indicating characteristics related to walking while operating a smartphone may be represented by skeleton data representing a posture of a person operating the smartphone. That is, the data may be represented by skeleton data representing a posture with the head lowered and the hands positioned in front of the chest. The characteristic data is associated with an additional condition indicating that the person is moving at a speed equal to or greater than a first speed.

[0034] For example, the data indicating a characteristic related to running may be represented by skeleton data representing a posture of a person running. The characteristic data is associated with an additional condition indicating that the person is moving at a speed equal to or greater than a second speed, the second speed being greater than the first speed.

[0035] For example, the data indicating the characteristic related to walking with hands in pockets may be represented by skeleton data representing the posture of a person walking with hands in pockets, i.e., the posture with the arms fixed near the waist. The characteristic data is associated with an additional condition indicating that the person is moving at a speed equal to or greater than a first speed.

[0036] In other embodiments, data indicating characteristics related to walking while operating a smartphone, data indicating characteristics related to running, and data indicating characteristics related to walking with hands in pockets may not be represented by skeleton data. For example, when detecting walking while operating a smartphone, running, walking with hands in pockets, etc. using an image and a classifier based on pattern matching or machine learning, the feature amount of the image may be stored in the risky behavior dictionary data D2 as data indicating the characteristics.

[0037] The warning data differs depending on the type of dangerous act. The warning data may be audio data in which a predetermined sound is recorded in advance. The warning data may also be image or text data indicating a warning to be displayed on a display. The warning data may also be light such as a warning light. In the example shown in FIG. 5, the warning data differs depending on the type of dangerous act, but is not limited to this. In other embodiments, the warning data may be common warning data. For example, the warning data may be common warning data regardless of the type of dangerous act, such as "Do not perform dangerous acts", "Please follow safety rules", an icon indicating caution, an icon indicating danger, a blinking display on the screen, etc.

[0038] The types of dangerous actions, characteristic data, and additional conditions stored in the dangerous action dictionary data D2 may differ depending on the site. For example, if walking while operating a smartphone is prohibited at one site but stopping to operate a smartphone is not prohibited, the operation of a smartphone itself may be prohibited at another site. Dangerous actions may be defined as rules at the site.

[0039] By executing the program, the processor 210 is provided with an acquisition unit 211, an extraction unit 212, a risky act determination unit 213, a warning unit 214, a recording unit 215, and a transmission unit 216.

[0040] The acquisition unit 211 acquires captured images from the multiple cameras 121 . Based on the person dictionary data D1, the extraction unit 212 extracts a partial image in which a person appears from the captured image acquired by the acquisition unit 211. Examples of a method for detecting a person include an object detection process based on pattern matching and machine learning. In the first embodiment, the extraction unit 212 extracts a person using a feature amount of an image, but is not limited to this. For example, in other embodiments, the extraction unit 212 may extract a person based on a measurement value of LiDAR (Light Detection and Ranging) or the like.

[0041] The risky act determination unit 213 determines whether or not the person extracted by the extraction unit 212 is performing a risky act, based on the risky act dictionary data D2, which is imaging data, and the partial image extracted by the extraction unit 212. When the risky act determination unit 213 determines that the person is performing a risky act, it specifies the type of the risky act. The warning unit 214 causes the speaker 122 to output a warning sound when the risky behavior determination unit 213 determines that a person is performing a risky behavior.

[0042] When the risky act determination section 213 determines that a person is performing a risky act, the recording section 215 stores in the storage 250 risky act history data in which the captured image, the image capture time, the image capture position, and the type of risky act acquired by the acquisition section 211 are associated with each other. The risky act history data does not necessarily have to associate all of the captured image, the image capture time, the image capture position, and the type of risky act, but may associate the captured image with at least one of the image capture time, the image capture position, the type of risky act, and other data related to risky acts. At least one of the image capture time, the image capture position, the type of risky act, and other data related to risky acts may be stored. The number of risky acts may also be stored, and for example, the number of risky acts may be stored for each type of risky act. The transmission unit 216 transmits the risky behavior history data stored in the recording unit 215 to a server device (not shown). The transmission unit 216 does not necessarily need to transmit all of the captured images, image capture times, image capture positions, and types of risky behavior, and may transmit only a part of the risky behavior history data. For example, the transmission unit 216 may transmit information indicating the number of risky behaviors or the number of times for each type of risky behavior.

[0043] <Method for detecting dangerous behavior> FIG. 6 is a flowchart showing the operation of the control device 143 according to the first embodiment. When the control device 143 starts the surroundings monitoring process, the process shown in FIG. 6 is repeatedly executed.

[0044] The acquisition unit 211 acquires captured images from the multiple cameras 121 (step S1). Next, the extraction unit 212 executes an extraction process of partial images in which people appear using the person dictionary data D1 for each captured image acquired in step S1, and determines whether one or more partial images are extracted (step S2). If a partial image in which a person appears is not extracted (step S2: NO), there is no person performing a risky act, and the control device 143 ends the process.

[0045] If a partial image containing a person is extracted (step S2: YES), the risky action determination unit 213 selects one or more partial images extracted in step S2 one by one, and executes the processing of steps S4 to S14 below (step S3). The risky act determination unit 213 selects the types of risky acts stored in the risky act dictionary data D2 one by one, and executes the processes from step S5 to step S14 below (step S4).

[0046] The risky act determination unit 213 identifies feature data associated with the type selected in step S4, and generates feature data of the same type from the partial image selected in step S3 (step S5). The risky act determination unit 213 calculates the similarity between the feature data associated with the type selected in step S4 and the feature data of the partial image generated in step S5 (step S6). The risky act determination unit 213 determines whether the similarity of the feature data is equal to or greater than a predetermined threshold (step S7).

[0047] The risky act determination unit 213 determines whether the similarity of the feature data is equal to or greater than a predetermined threshold (step S7). If the similarity of the feature data is equal to or greater than the threshold (step S7: YES), the risky act determination unit 213 determines whether there is an additional condition associated with the feature data (step S8). If there is an additional condition (step S8: YES), the risky act determination unit 213 determines whether the partial image selected in step S3 satisfies the additional condition (step S9). If the additional condition is a condition related to speed, the risky act determination unit 213 determines, for example, whether the distance between the position of the partial image extracted in the previous captured image and the position of the partial image selected in step S3 is equal to or greater than a predetermined distance.

[0048] If the additional condition is satisfied (step S9: YES), or if the matching feature data does not have the additional condition (step S8: NO), the risky behavior determination unit 213 determines that the person associated with the partial image selected in step S3 is engaging in the type of risky behavior selected in step S4 (step S10). The warning unit 214 outputs a warning sound from the speaker 122 based on the warning data associated with the type of risky behavior selected in step S4 (step S11).

[0049] The recording unit 215 starts recording the moving image (step S12). That is, the recording unit 215 generates a moving image by recording the captured image of the camera 121 for a certain time after it is determined in step S10 that the dangerous behavior is being performed. The person performing the dangerous behavior is captured in the moving image. Then, the recording unit 215 causes the storage 250 to store dangerous behavior history data that associates the moving image generated in step S11, the image capture time, the image capture position, and the type of dangerous behavior identified in step S10 (step S13). The dangerous behavior history data recorded in the storage 250 is later transmitted to the server device by the transmission unit 216. The image capture position is represented by, for example, position data acquired by a GNSS positioning device (not shown) of the work machine 100 at the time of image capture.

[0050] On the other hand, if the additional condition is not met (step S9: NO) or the similarity of the feature data is less than the threshold value (step S7: NO), the risky behavior determination unit 213 determines that the person related to the partial image selected in step S3 is not engaging in the type of risky behavior selected in step S4 (step S14).

[0051] The process shown in FIG. 6 is merely an example, and in other embodiments, the control device 143 may perform a process different from that shown in FIG. 6. For example, the risky behavior dictionary data D2 according to other embodiments may not have an additional condition. In this case, the control device 143 may not make the judgments in steps S8 and S9. In other embodiments, the control device 143 may not output the warning sound in step S11 or record the moving image in step S13. In other embodiments, the warning may not be by sound, such as by displaying on the display 143D or illuminating a warning light. In addition, the loop process in at least one of steps S3 and S4 may be realized by parallel processing.

[0052] Actions and Effects In this way, when a person appearing in a captured image is performing a dangerous act, the control device 143 can determine that the person is performing the dangerous act. This allows the control device 143 to reduce the burden on the supervisor of paying attention to dangerous acts at the work site. Furthermore, when the control device 143 determines that a dangerous act has occurred, it can output a warning. For example, a warning sound can be output to the outside of the work machine 100. This allows a worker performing a dangerous act in the vicinity of the work machine 100 to become aware of his or her own dangerous act, and also allows other workers to be alerted to the dangerous act. For example, a warning image can be displayed on the display 143D. This allows the operator of the work machine 100 to be alerted.

[0053] Furthermore, the control device 143 can record data related to the dangerous act in the storage 250. For example, a video showing the scene of the dangerous act can be recorded in the storage 250. This makes it possible to store the scene where the dangerous act occurred. Furthermore, the control device 143 can transmit data related to the dangerous act to the server device. For example, a video showing the scene of the dangerous act can be transmitted to the server device. This makes it possible for an on-site supervisor to later check the video and confirm whether or not a dangerous act actually occurred.

[0054] Example of operation FIG. 7 is a diagram showing an example of an image captured by the camera 121 according to the first embodiment. When the camera 121 obtains a captured image as shown in FIG. 6, the extraction unit 212 of the control device 143 extracts two partial images G1 and G2 in step S2. First, the risky behavior determination unit 213 executes the process of steps S5 to S14 for each type of risky behavior for the partial image G1. At this time, the risky behavior determination unit 213 compares the feature data related to "not wearing a safety vest" with the feature data of the torso G11 of the partial image G1 in steps S5 to S7 and determines that the similarity is high. Since the feature data related to "not wearing a safety vest" is not associated with an additional condition, the risky behavior determination unit 213 determines in step S10 that the person related to the partial image G1 is performing a risky behavior. Therefore, the warning unit 214 outputs a warning sound from the speaker 122 saying "Please wear a safety vest." In addition, the recording unit 215 records the moving image including the image shown in FIG. 6 in the storage 250.

[0055] Furthermore, the risky behavior determination unit 213 executes the processes from step S5 to step S14 for each type of risky behavior for partial image G2. The risky behavior determination unit 213 compares the feature data related to "not wearing a safety vest" with the feature data of the torso G21 of partial image G1 and determines that the similarity is low. Similarly, the processes from step S5 to step S14 are executed for the other types of risky behavior, and since the similarity is low for all of the risky behaviors and the additional condition is not satisfied, the risky behavior determination unit 213 determines in step S14 that the person related to partial image G2 is not performing a risky behavior.

[0056] Other Embodiments Although one embodiment has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes are possible. That is, in other embodiments, the order of the above-mentioned processes may be changed as appropriate. Also, some of the processes may be executed in parallel.

[0057] In the embodiment described above, the work machine 100 is equipped with a plurality of cameras 121 and speakers 122, but this is not limited to the above. For example, in other embodiments, the cameras and speakers may be provided externally to the work machine 100. Examples of speakers and cameras provided externally include speakers and cameras installed at the work site, speakers and cameras provided on other work machines 100, etc.

[0058] The detection system according to the above-described embodiments may be provided external to the work machine 100 . In another embodiment, some of the components constituting the detection system may be mounted inside the work machine 100, and other components may be provided outside the work machine 100. For example, the display 143D may be provided in a remote control room provided remotely from the work machine 100 in the detection system. In another embodiment, the above-mentioned multiple computers or the above-mentioned single computer may all be provided outside the work machine 100. For example, the detection system may include a combination of a fixed camera installed on-site and one or more computers provided in a management room or the like, instead of or in addition to the control device 143. In this case, the computer provided outside the work machine 100 has the same configuration as part or all of the control device 143 shown in FIG. 4. The computer provided outside the work machine 100 may perform the processing shown in FIG. 6 based on the captured image obtained from the fixed camera.

[0059] Furthermore, the detection system according to the embodiment described above outputs an audio warning from the speaker 122 of the work machine 100 to alert a worker or a supervisor who is performing a dangerous act near the work machine 100. However, this is not limited to the above. For example, the detection system according to another embodiment may be one in which a speaker is provided inside the cab 140 to alert the operator. The speaker may be a buzzer provided in the cab, or a speaker integrated into the display 143D in the cab, and the speaker may be used to alert the operator. Furthermore, the detection system according to the embodiment described above outputs an audio warning from the speaker 122 of the work machine 100, but is not limited to this. For example, a detection system according to another embodiment may output an audio warning from a fixed speaker provided at the work site. Furthermore, according to another embodiment, the detection system of the work machine 100 may output an audio warning from the speaker 122 of another work machine 100 through vehicle-to-vehicle communication.

[0060] Furthermore, in the above-described embodiment, an example has been described in which a person performing a dangerous act outside the work machine 100 is determined, but the present invention is not limited to this. For example, in other embodiments, a dangerous act performed by a person inside the work machine 100 may be determined. For example, a camera may be provided inside the cab 140 so as to capture an image of the operator, and the detection system may determine the operator's dangerous act. In this case, the operator's attention can be drawn by outputting a warning sound from a speaker provided in the cab, or by displaying an image or text indicating a warning on the display 143D.

[0061] In addition, the detection system according to the embodiment described above extracts a person and then determines whether or not the person is engaging in a dangerous behavior, but is not limited to this. For example, in another embodiment, the presence or absence of a person engaging in a dangerous behavior may be estimated directly from the captured image using a trained model that estimates the presence or absence of a person engaging in a dangerous behavior from the entire captured image.

[0062] In addition, the detection system according to the embodiment described above records a moving image showing the scene of the dangerous act in step S12, but is not limited to this. For example, the detection system according to another embodiment may record the captured image, i.e., a still image, acquired in step S1.

[0063] In addition, the detection system according to the above embodiment outputs a warning sound according to the type of the dangerous behavior, but is not limited to this. For example, a detection system according to another embodiment may output a horn sound regardless of the type of the dangerous behavior.

[0064] In another embodiment, the detection system may calculate the degree of danger of a person performing a dangerous act, and determine whether or not to output the warning based on the degree of danger. For example, the degree of danger may be calculated based on the person performing the dangerous act and the distance from the rotation center of the work machine 130, and a warning may be output when the degree of danger exceeds a threshold. In another embodiment, the detection system may be configured to include only the display 143D among the display 143D and the speaker 122. In this case, the operator of the work machine 130 or a supervisor at the site can be alerted based on the image or text indicating a warning displayed on the display 143D.

[0065] Furthermore, the work machine 100 according to the embodiment described above is a hydraulic excavator, but is not limited to this. For example, the work machine 100 according to other embodiments may be another work machine, such as a dump truck, a bulldozer, or a wheel loader. [Explanation of symbols]

[0066] REFERENCE SIGNS LIST 100 work machine 110 travelling body 120 revolving body 121 camera 130 work machine 143 control device 211 acquisition unit 212 extraction unit 213 dangerous action determination unit 214 warning unit 215 recording unit 216 transmission unit

Claims

1. A work machine, comprising: A running body, A rotating body rotatably supported on the traveling body; A plurality of cameras provided on the rotating body for capturing images of the surroundings of the work machine; an acquisition unit that acquires imaging data from the plurality of cameras; a dangerous behavior determination unit that determines whether or not there is a person performing a dangerous behavior around the work machine by comparing the imaging data with feature data that represents features of the dangerous behavior; A work machine comprising:

2. an extraction unit that extracts a portion corresponding to a person from the imaging data; The risky behavior determination unit determines whether or not a person appearing in the extracted portion is engaging in the risky behavior.

2. The work machine of claim 1.

3. a speaker provided on the rotating body for outputting sound to the outside of the work machine; a warning unit that outputs a warning from the speaker when it is determined that a person is performing the dangerous behavior; 2. The work machine of claim 1 .

4. The warning unit outputs a sound corresponding to the type of the dangerous act as the warning.

4. A work machine according to claim 3.

5. A work machine, comprising: A running body, A rotating body rotatably supported on the traveling body; A plurality of cameras provided on the rotating body for capturing images of the surroundings of the work machine; an acquisition unit that acquires imaging data from the plurality of cameras; a dangerous behavior determination unit that determines whether or not there is a person performing a dangerous behavior around the work machine based on the imaging data; a recording unit that records the imaging data related to the determination when it is determined that a person performing the dangerous behavior is present; A work machine comprising:

6. The work machine according to claim 5 , wherein the recording unit records data relating to the dangerous act in association with the imaging data.

7. The dangerous acts include not wearing protective equipment. A work machine according to any one of claims 1 to 6.

8. A detection method for a work machine including a traveling body and a rotating body rotatably supported on the traveling body, comprising: acquiring imaging data from a plurality of cameras provided on the rotating body for imaging an area around the work machine; a step of determining whether or not there is a person engaging in a dangerous act around the work machine by comparing the image data acquired from the plurality of cameras with characteristic data that represents characteristics of the dangerous act; A detection method comprising:

9. an extraction unit that extracts a portion corresponding to a person from the imaging data based on person dictionary data for detecting a person; The risky behavior determination unit determines whether or not a person appearing in the extracted portion is engaging in a risky behavior based on risky behavior dictionary data that stores the feature data for detecting a risky behavior.

2. The work machine of claim 1.

Images