Systems for construction machinery

Abstract

The purpose is to prevent the theft of construction machinery. [Solution] A system for construction machinery having a learning model that takes information acquired from sensors on the construction machinery as input and outputs information to identify a person, a discrimination unit that determines whether or not a person is suspicious based on the suspiciousness of the person derived using the person identification information output from the learning model, and an operation instruction unit that instructs the construction machinery to take an action according to the result of the discrimination by the discrimination unit.

Description

Technical Field

[0001] This disclosure relates to a system for construction machinery.

Background Art

[0002] Conventionally, when an intruder into a vehicle is detected by vibration using a vibration sensor, an alarm signal is transmitted to a wireless receiver located at a position away from the vehicle, and the owner of the vehicle who is away from the vehicle is notified that a suspicious person has invaded the vehicle. A vehicle anti-theft system is known.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] At a construction site or the like where construction machinery is arranged, there are various people, and in some cases, an unspecified number of workers use the construction machinery. In such a case, it is difficult to prevent theft of the construction machinery with the conventional technology described above.

[0005] This disclosure has been made in view of the above circumstances, and an object thereof is to prevent theft of construction machinery.

Means for Solving the Problems

[0006] A system for construction machinery according to an embodiment of the present invention has a learning model that takes as input information acquired from sensors of the construction machinery and outputs information for identifying a person, and based on the suspiciousness of the person derived using the information for identifying the person output from the learning model, a determination unit that determines whether the person is a suspicious person, and an operation instruction unit that instructs the construction machinery to perform an operation according to the result of the determination by the determination unit. It is a system for construction machinery. [Effects of the Invention]

[0007] It can prevent the theft of construction machinery. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram showing an example configuration of the SYS system for construction machinery. [Figure 2] This is a system configuration diagram for the SYS system used in construction machinery. [Figure 3] This is a left side view of the interior of cabin 10. [Figure 4] This diagram illustrates the control system and the functional configuration of an excavator used in construction machinery systems. [Figure 5] This figure shows an example of a human information storage unit. [Figure 6] This figure shows an example of a suspicious person information storage unit. [Figure 7] This is a sequence diagram illustrating the operation of a system for construction machinery. [Modes for carrying out the invention]

[0009] First, with reference to Figures 1 to 3, a system SYS for construction machinery, including a shovel (excavator) 100 according to an embodiment of the present invention, will be described. Figure 1 is a schematic diagram showing an example of the configuration of a system SYS for construction machinery. Figure 2 is a system configuration diagram of the system SYS for construction machinery. Figure 3 is a left side view of the interior of the cabin 10. Note that the shovel 100 in this embodiment is just one example of construction machinery. Construction machinery is not limited to the shovel 100.

[0010] The SYS system for construction machinery is designed to prevent the theft of the Shovel 100.

[0011] In this embodiment, the construction machinery system SYS uses a learning model 341, described later, to derive the likelihood of a person being suspicious from information detected by the sensors of the excavator 100, and determines whether the person is an unknown person (suspicious person) in the construction machinery system SYS. Then, if a person identified as unknown is detected inside the cabin 10, the construction machinery system SYS prevents theft of the excavator 100 by, for example, restricting the operation of the excavator 100, notifying the administrator that an unknown person has entered the cabin 10, or sounding an alarm. In the following description, an unknown person may be referred to as a suspicious person.

[0012] The SYS system for construction machinery mainly consists of a shovel 100, a support device 200, and a control device 300. The shovel 100, support device 200, and control device 300 that make up the SYS system for construction machinery may each consist of one unit or multiple units. In this embodiment, it includes one shovel 100, one support device 200, and one control device 300.

[0013] The support device 200 is a portable terminal device, such as a laptop, tablet PC, or smartphone, carried by a worker at a construction site. It may also be a computer carried by the operator of the shovel 100.

[0014] The management device 300 is a fixed terminal device, such as a computer installed in a management center outside the construction site. The management device 300 may also be a portable computer (for example, a laptop PC, tablet PC, smartphone, or other mobile terminal device).

[0015] The lower traveling body 1 of the excavator 100 is equipped with an upper slewing body 3 that can slew via a slewing mechanism 2. A boom 4 is attached to the upper slewing body 3. An arm 5 is attached to the tip of the boom 4, and a bucket 6 is attached to the tip of the arm 5. The boom 4, arm 5, and bucket 6 as working elements constitute an excavation attachment, which is an example of an attachment. The boom 4, arm 5, and bucket 6 are hydraulically driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively. The upper slewing body 3 is provided with a cab 10 as an operator's cab, and a power source such as an engine 11 is mounted thereon.

[0016] As shown in FIG. 2, the excavator 100 is composed of an engine 11, a main pump 14, a pilot pump 15, a control valve 17, an operating device 26, a controller 30, an engine control device 74, etc.

[0017] The engine 11 is the drive source of the excavator 100, and is, for example, a diesel engine that operates to maintain a predetermined rotational speed. The rotating shaft of the engine 11 is connected to the rotating shafts of the main pump 14 and the pilot pump 15.

[0018] The main pump 14 is a swash plate type variable displacement hydraulic pump that supplies hydraulic oil to the control valve 17 via an oil line 16. The discharge flow rate per rotation of the main pump 14 changes according to the change in the swash plate tilt angle. The swash plate tilt angle is controlled by, for example, a regulator 14a. The regulator 14a changes the swash plate tilt angle according to, for example, the change in the control current from the controller 30. On the excavator 100, a discharge pressure sensor for detecting the discharge pressure of the main pump 14 and a tilt angle sensor for detecting the swash plate tilt angle are attached.

[0019] The pilot pump 15 is a fixed displacement hydraulic pump that supplies hydraulic oil to various hydraulic control devices such as the operating device 26 via a pilot line 25.

[0020] The control valve 17 is a set of flow control valves that control the flow of hydraulic oil related to the hydraulic actuator. The control valve 17 selectively supplies the hydraulic oil received from the main pump 14 through the hydraulic oil line 16 to one or more hydraulic actuators according to the change in the pilot pressure corresponding to the operation direction and operation amount of the operating device 26. The hydraulic actuators include, for example, the boom cylinder 7, the arm cylinder 8, the bucket cylinder 9, the left travel hydraulic motor 1A, the right travel hydraulic motor 1B, the swing hydraulic motor 2A, etc. A hydraulic pressure sensor for detecting the pressure of the hydraulic oil in the hydraulic actuator may be attached to the excavator.

[0021] The operating device 26 is a device used by the operator of the excavator 100 for operating the hydraulic actuator. The operating device 26 receives the supply of hydraulic oil from the pilot pump 15 through the pilot line 25 to generate a pilot pressure. Then, through the pilot line 25a, the pilot pressure is applied to the pilot port of the corresponding flow control valve. The pilot pressure changes according to the operation direction and operation amount of the operating device 26. The pilot pressure sensor 15a detects the pilot pressure and outputs the detected value to the controller 30. Figure 3 shows the left operation lever 26L, the left travel pedal 26PL, and the left travel lever 26DL that constitute the operating device 26.

[0022] The controller 30 is a control device for controlling the excavator 100. In this embodiment, the controller 30 is composed of a computer equipped with a CPU, a volatile memory medium, a non-volatile memory medium, etc. The CPU of the controller 30 realizes the functions corresponding to each of those programs by executing the programs corresponding to various functions.

[0023] The engine control device 74 is a device that controls the engine 11. The engine control device 74 controls, for example, the fuel injection amount and the like so that the engine speed set via the input device is achieved. The engine control device 74 is connected to an engine speed sensor, an engine load rate sensor, a fuel injection amount sensor, and the like. The engine load rate sensor may be an engine torque sensor.

[0024] The transmitter S1, receiver S2, positioning device S3, attitude detection device S4, orientation detection device S5, camera S6, driving data acquisition device S7, and display device 40, all attached to the upper rotating body 3, are connected to the controller 30. The controller 30 performs various calculations based on the information output by each of the receiver S2, positioning device S3, attitude detection device S4, orientation detection device S5, camera S6, and driving data acquisition device S7. The controller 30 then transmits the information generated based on the calculation results to the outside via the transmitter S1 or displays it on the display device 40.

[0025] The transmitting device S1 transmits information to the outside of the shovel 100. The transmitting device S1 transmits information that can be received by, for example, at least one of the support device 200 and the management device 300 (hereinafter referred to as "external device"). In this embodiment, the transmitting device S1 transmits information that can be received by the external device to the external device via a satellite link, a mobile phone link, etc.

[0026] The receiving device S2 receives information from outside the shovel 100. The receiving device S2 receives information transmitted by an external device, for example. In this embodiment, the receiving device S2 receives information transmitted by an external device via a satellite link, a mobile phone link, or the like.

[0027] The positioning device S3 acquires information regarding the position of the shovel 100. In this embodiment, the positioning device S3 is a GNSS (GPS) receiver and measures the latitude, longitude, and altitude of the location of the shovel 100.

[0028] The attitude detection device S4 detects the attitude of the shovel 100. The attitude of the shovel 100 is, for example, the attitude of the excavation attachment. In this embodiment, the attitude detection device S4 includes a boom angle sensor, an arm angle sensor, a bucket angle sensor, and a machine tilt sensor. The boom angle sensor is a sensor that acquires the boom angle and includes, for example, a rotation angle sensor that detects the rotation angle of the boom foot pin, a stroke sensor that detects the stroke amount of the boom cylinder 7, and a tilt (acceleration) sensor that detects the tilt angle of the boom 4. A combination of an acceleration sensor and a gyro sensor may also be used. The same applies to the arm angle sensor and the bucket angle sensor. The machine tilt sensor is a sensor that acquires the machine tilt angle and detects, for example, the tilt angle of the upper slewing body 3 with respect to the horizontal plane. In this embodiment, the machine tilt sensor is a two-axis acceleration sensor that detects the tilt angle of the upper slewing body 3 around the longitudinal axis and the left-right axis. The longitudinal axis and left-right axis of the upper slewing body 3 are, for example, orthogonal to each other and pass through the shovel center point, which is a point on the slewing axis of the shovel 100. The aircraft tilt sensor may also be a 3-axis accelerometer.

[0029] The orientation detection device S5 detects the orientation of the shovel 100. The orientation detection device S5 is composed of, for example, a geomagnetic sensor, a resolver or encoder related to the pivot axis of the pivot mechanism 2, a gyro sensor, etc. In this embodiment, the orientation detection device S5 is composed of a combination of a 3-axis geomagnetic sensor and a gyro sensor. It may also be a pair of GNSS receivers.

[0030] The controller 30 can acquire information regarding the trajectory of the bucket's toe based on the outputs of the positioning device S3, the attitude detection device S4, and the orientation detection device S5.

[0031] The controller 30, display device 40, engine control device 74, etc., operate by receiving power from the battery 70. The battery 70 is charged by a generator driven by the engine 11. Power from the battery 70 is also supplied to the starter of the engine 11, etc. The starter is driven by power from the battery 70 to start the engine 11.

[0032] Camera S6 outputs the acquired image to the controller 30. In this embodiment, camera S6 includes a rear camera S6B that captures the space behind the shovel 100. Camera S6 may also include a right camera S6R that captures the space to the right of the shovel 100, a left camera S6L that captures the space to the left of the shovel 100, and a front camera S6F that captures the space in front of the shovel 100. Camera S6 also includes an interior camera S6C that captures the operator inside the cabin 10. The controller 30 may perform various image processing on the image of the operator captured by the interior camera S6C to identify or authenticate the operator.

[0033] Camera S6 is, for example, a monocular camera having an image sensor such as a CCD or CMOS, and outputs the captured image to the display device 40. Camera S6 may also be a stereo camera, a depth image camera, etc. Furthermore, camera S6 may be replaced by other spatial recognition devices such as a 3D depth image sensor, an ultrasonic sensor, a millimeter-wave radar, a LiDAR or an infrared sensor, or it may be replaced by a combination of other spatial recognition devices and a camera.

[0034] The operation data acquisition device S7 acquires operation data, which is information relating to the operation of the shovel 100. In this embodiment, the operation data acquisition device S7 includes a pilot pressure sensor 15a, a discharge pressure sensor, a tilt angle sensor, an operating hydraulic pressure sensor, an engine speed sensor, an engine load rate sensor, a fuel injection amount sensor, a positioning device S3, an attitude detection device S4, an orientation detection device S5, etc. The operation data may also include information relating to the working environment. In this case, the operation data acquisition device S7 may include an outside temperature sensor, an inside temperature sensor, a pressure sensor, a humidity sensor, an illuminance sensor, a machine tilt sensor, a vibration sensor, etc.

[0035] The driver's seat DS is the seat in which the operator of the shovel 100 sits. The display device 40 is a device that displays various information and is located near the driver's seat DS inside the cabin 10. In this embodiment, the display device 40 can display images captured by the camera S6. The image captured by the camera S6 may be a composite image obtained by combining images captured by multiple cameras. The composite image may be subjected to various image processing, such as viewpoint transformation processing. The display device 40 may be a portable terminal device such as a notebook PC, tablet PC, or smartphone.

[0036] Although Shovel 100 is a hydraulic excavator, the construction machinery described in this disclosure is not limited to hydraulic excavators. Shovel 100 may be an EV excavator driven solely by an electric motor, a hybrid excavator, or a wheeled excavator. Furthermore, the construction machinery described in this disclosure may be other than Shovel 100, such as a crane, asphalt finisher, or forklift.

[0037] Next, the hardware configuration of the management device 300 in this embodiment will be described. The management device 300 in this embodiment is a computer having a CPU 301, a storage device 302, a communication device 303, an input device 304, and a display device 305, all of which are interconnected by a bus.

[0038] The CPU 301 controls the overall operation of the management device 300. The storage device 302 stores programs executed by the CPU 301 and various information related to the shovel 100. The communication device 303 communicates with the shovel 100 and support devices via the network.

[0039] The input device 304 is for inputting information to the management device 300 and can be implemented as, for example, a keyboard or a pointing device. The display device 305 displays various types of information output from the management device 300 and can be implemented as a display or the like.

[0040] The details of the support device 200 in this embodiment are omitted from the description because the hardware configuration may be the same as that of the management device 300.

[0041] Next, referring to Figure 4, the functions of the control device 300 and the excavator 100 of the SYS system for construction machinery in this embodiment will be described. Figure 4 is a diagram illustrating the functional configuration of the control device and the excavator of the system for construction machinery.

[0042] First, the functions of the shovel 100 will be described. The shovel 100 in this embodiment includes a communication control unit 31, a start control unit 32, an information acquisition unit 33, a human detection unit 34, and an operation control unit 35. The functions of each of these units are realized by the CPU of the controller 30 of the shovel 100 reading and executing a program stored on a non-volatile storage medium or the like.

[0043] The communication control unit 31 controls communication between the shovel 100 and the management device 300 and the support device 200. For example, the communication control unit 31 may transmit image data acquired by the camera S6 and operation data acquired by the operation data acquisition device S7 to the management device 300. The image data acquired by the camera S6 may be included in the operation data.

[0044] The start control unit 32 determines whether the state of the shovel 100 meets the conditions for initiating the process of determining whether the person inside the cabin 10 is an unknown person. If the state of the shovel 100 meets the conditions, the start control unit 32 initiates this process. In the following description, the process of determining whether the person inside the cabin 10 is an unknown person and notifying the system may be referred to as the anti-theft process. Also, in the following description, the conditions for initiating the anti-theft process will be referred to as the start conditions.

[0045] The starting condition in this embodiment may be, for example, the engine 11 of the shovel 100 being turned on. Alternatively, the starting condition may be the controller 30 being activated. In this case, the engine 11 does not necessarily have to be turned on.

[0046] Furthermore, the starting condition may be that the door to the cabin 10 is opened from the outside. Alternatively, the starting condition may be that a person is detected inside the cabin 10 by the person detection unit 34. Furthermore, the starting condition may be that a person is detected around the shovel 100.

[0047] The starting conditions are not limited to the examples described above. The starting conditions may be arbitrarily set by the administrator of the SYS system for construction machinery, etc. Furthermore, the starting conditions may be a combination of several of the conditions described above.

[0048] When the start control unit 32 initiates the theft prevention process, the information acquisition unit 33 acquires information used in the theft prevention process. Specifically, the information acquisition unit 33 may acquire image data acquired by the camera S6. The image data acquired by the information acquisition unit 33 may include image data taken inside the cabin 10 and image data taken around the shovel 100.

[0049] Furthermore, the image data acquired by the information acquisition unit 33 is not limited to image data captured by the camera S6 of the shovel 100. The information acquisition unit 33 may also acquire image data captured by a camera installed at the construction site where the shovel 100 is working.

[0050] Furthermore, the information acquisition unit 33 may acquire data such as driving data and operation history as information used in the anti-theft process. In addition, the information acquisition unit 33 in this embodiment may acquire information that can detect a person's skeleton and gestures.

[0051] The information acquired by the information acquisition unit 33 can be any data acquired in a time series. Furthermore, the information acquired by the information acquisition unit 33 can be described as sensor values ​​output from various sensors on the shovel 100.

[0052] The person detection unit 34 detects people inside the cabin 10 based on the information acquired by the information acquisition unit 33. The person detection unit 34 may also detect people around the shovel 100 based on the information acquired by the information acquisition unit 33. Specifically, the person detection unit 34 may detect people attempting to enter the cabin 10.

[0053] The operation control unit 35 controls the operation of the shovel 100 in accordance with instruction information received from the control device 300. For example, the operation control unit 35 restricts the operation of the shovel 100. Specifically, restricting the operation includes stopping the discharge of hydraulic fluid from the main pump 14 in order to stop all operation of the shovel 100. Restricting the operation also includes controlling the amount of hydraulic fluid discharged from the main pump 14 in order to maintain the travel speed of the shovel 100 below a predetermined level. Furthermore, restricting the operation also includes disabling the operation of attachments.

[0054] Furthermore, the operation control unit 35 outputs a warning sound using a speaker or the like installed on the shovel 100.

[0055] Next, the functions of the management device 300 will be described. The management device 300 in this embodiment includes a person information storage unit 310, a suspicious person information storage unit 320, a communication control unit 330, a person identification processing unit 340, an operation instruction unit 350, and a storage control unit 360.

[0056] The person information storage unit 310 and the suspicious person information storage unit 320 may be implemented by the storage device 302 of the management device 300. The communication control unit 330, person identification processing unit 340, operation instruction unit 350, and storage control unit 360 may be implemented by the CPU 301 of the management device 300 reading and executing a program stored in the storage device 302.

[0057] The person information storage unit 310 stores person information about people known in the SYS system for construction machinery. The suspicious person information storage unit 320 stores suspicious person information indicating the status of the shovel 100 when a person identified by the person identification processing unit 340 based on person features is determined to be an unknown person. In other words, the suspicious person information storage unit 320 stores suspicious person information indicating the status of the shovel 100 when a person inside the cabin 10 is determined to be a suspicious person. Details of the person information storage unit 310 and the suspicious person information storage unit 320 will be described later.

[0058] The communication control unit 330 controls communication between the management device 300 and the shovel 100 and support device 200. Specifically, the communication control unit 330 receives information transmitted from the shovel 100 and transmits the instruction information generated by the operation instruction unit 350 to the shovel 100.

[0059] The person identification processing unit 340 has a learning model 341. The person identification processing unit 340 uses the learning model 341 to determine from the information received from the shovel 100 whether the person inside the cabin 10 or the person around the shovel 100 is an unknown person or a known person.

[0060] Here, we will explain the difference between unknowns and knowns. In this embodiment, an unknown person may be a person not known to the administrator of the construction machinery system SYS, etc., and whose corresponding person information is not stored in the person information storage unit 310. In this embodiment, a known person may be a person known to the administrator of the construction machinery system SYS, etc., and whose corresponding person information is stored in the person information storage unit 310. In other words, a known person is not a suspicious person, but a person scheduled to work at the construction site where the shovel 100 is deployed.

[0061] Furthermore, the learning model 341 in this embodiment is a type of deep learning model. When the learning model 341 receives information from the shovel 100 as input, it outputs person features that identify a person. In other words, the learning model 341 is a learning model that takes information acquired by the sensors of the shovel 100 as input and outputs information that identifies a person.

[0062] In this embodiment, for example, image data received from the shovel 100 may be input to the learning model 341. When image data is input to the learning model 341, it outputs a person feature that identifies the person contained in the image shown by the image data.

[0063] Note that the data input to the learning model 341 does not have to be image data. The data input to the learning model 341 can be any data arranged in time series, such as driving data or data showing operation history.

[0064] The person features in this embodiment may be vector-based information whose elements are variable values ​​of the image data input to the learning model 341. In this embodiment, for example, the elements included in the person features may include elements that indicate the attributes of the person identified by the person features. Elements that indicate attributes may be, for example, the type of license held or the type of work permitted, and may be values ​​of items included in the person information.

[0065] Furthermore, the learning model 341 in this embodiment may be a pre-trained model that is primarily composed of a neural network. In that case, the learning model 341 may have been trained using a dataset as training data that associates image data from inside the cabin 10 when a person whose information is stored in the person information storage unit 310 has operated the shovel 100 in the past with the person's information. In other words, the learning model 341 may be a pre-trained model that uses a dataset as training data that associates person information with image data from when the person identified by the person information is operating the shovel 100.

[0066] In this embodiment, when acquiring training data, the training data acquisition period may be set in advance in the shovel 100 or the control device 300. The training data acquisition period may be a period during which the person operating the shovel 100 is determined, such as the scheduled operating time of the shovel 100.

[0067] If the learning data acquisition period is set for the shovel 100, the start control unit 32 may control the learning data acquisition period so that the anti-theft process is not started.

[0068] Furthermore, if the learning data acquisition period is set in the management device 300, the human identification processing unit 340 may choose not to perform any processing during the learning data acquisition period, or it may choose to stop generating instruction information by the operation instruction unit 350.

[0069] When the learning model 341 outputs person features, the person recognition processing unit 340 uses the person features to determine whether the person identified by the person features is known or unknown to the learning model 341. In other words, the person recognition processing unit 340 determines whether the learning model 341 has previously learned training data containing person information of the person identified by the output person features.

[0070] In this embodiment, the person identification processing unit 340 uses the value obtained by inputting person features into a pre-prepared mathematical formula as a suspicious person identification feature, and if the suspicious person identification feature is greater than or equal to a predetermined threshold, it identifies the person identified by the person features as an unknown person. The predetermined threshold may be stored in the person identification processing unit 340 in advance.

[0071] The suspicious person identification features in this embodiment are information indicating the likelihood of a person being identified as suspicious by the person features, and are derived using the person features output from the learning model 341.

[0072] More specifically, the suspicious person identification feature is a feature designed so that its numerical value differs significantly between the case where the learning model 341 has not learned the personal information of people identified by the personal features and the case where it has learned the personal information of people identified by the personal features.

[0073] In this embodiment, by deriving suspicious person identification features as indicators of suspiciousness, the learning model 341 can classify individuals whose personal information it has not previously learned as unknown.

[0074] In this embodiment, the person identification processing unit 340 calculates suspicious person identification features and compares them with a predetermined threshold to determine whether the person identified by the person features is an unknown person, but it is not limited to this.

[0075] The person identification processing unit 340 may, for example, determine that a person identified by the person features is an unknown person if the suspicious person identification features calculated based on a mathematical formula have been at a predetermined value for a certain period of time. In this case, a predetermined threshold does not need to be provided. Alternatively, the person identification processing unit 340 may use a model that takes the person features as input data instead of a mathematical formula and outputs a determination result of whether or not the person identified by the person features is an unknown person.

[0076] The person identification processing unit 340 only needs to determine whether or not a person identified by the person features is suspicious, based on the likelihood of that person being suspicious.

[0077] The operation instruction unit 350 generates instruction information for the shovel 100 according to the discrimination result by the human discrimination processing unit 340. The generated instruction information is transmitted to the shovel 100 by the communication control unit 330.

[0078] Specifically, if the person identified by the person features is an unknown person, the operation instruction unit 350 may generate instruction information to restrict the operation of the shovel 100 or to output a warning sound. If the person identified by the person features is a known person, the operation instruction unit 350 may generate instruction information to restrict the operation of the shovel 100 based on the known person and the corresponding person information.

[0079] The storage control unit 360 acquires information indicating the status of the shovel 100 when the person identified by the person features is determined to be an unknown person, and stores it in the suspicious person information storage unit 320 as suspicious person information. In this embodiment, for example, when the person identified by the person features is determined to be an unknown person, the storage control unit 360 stores all or part of the information received from the shovel 100 at that time in the suspicious person information storage unit 320 as suspicious person information.

[0080] Next, the human information storage unit 310 and the suspicious person information storage unit 320 will be described with reference to Figures 5 and 6. Figure 5 is a diagram showing an example of the human information storage unit.

[0081] The personnel information storage unit 310 of this embodiment stores personnel information entered by the administrator of the SYS system for construction machinery, etc. The personnel information stored in the personnel information storage unit 310 may include personnel ID, name, licenses held, and types of work that can be performed, as information items.

[0082] The value of the "Person ID" field is identification information that identifies a person. The person identified by the Person ID is a worker performing tasks at a construction site, an operator of a Shovel 100, etc. The value of the "Name" field indicates the name of the person identified by the Person ID.

[0083] The value of the item "Licenses Held" indicates the licenses held by the person identified by the person ID. The value of the item "Permitted Work" indicates the types of work that the person identified by the person ID is permitted to perform. In other words, the value of the item "Permitted Work" indicates the operation of the shovel 100 permitted for the person identified by the person ID. In this embodiment, the values ​​of the items "Licenses Held" and "Permitted Work" are information that indicates the attributes of the person identified by the person ID.

[0084] Note that the items of information included in the person information are not limited to the example shown in Figure 5. The items of information included in the person information may include items other than those shown in Figure 5, or may be only a part of the items shown in Figure 5.

[0085] Figure 6 shows an example of a suspicious person information storage unit. The suspicious person information storage unit 320 stores suspicious person information according to the results of processing by the person identification processing unit 340. The suspicious person information stored in the suspicious person information storage unit 320 includes the following information items: record ID, date and time, location, aircraft number, and image.

[0086] The value of the "Record ID" field is identification information used to identify suspicious person information. The value of the "Date and Time" field indicates the date and time when the person identified by the personal features was determined to be an unknown person in the SYS system for construction machinery. The value of the "Location" field indicates the location when the person identified by the personal features was determined to be an unknown person in the SYS system for construction machinery. In the example in Figure 6, the value of the "Location" field is the name of the construction site, but it is not limited to this. The value of the "Location" field may be an address, etc.

[0087] The value of the item "Machine Number" is identification information that identifies Shovel 100. The value of the item "Image" indicates image data that includes an image of a person identified as an unknown person. More specifically, the image indicated by the value of the item "Image" may include images of frames showing the suspicious person, extracted from video data, and images acquired by camera S6 at the time before and after the suspicious person was recognized. In this case, the images acquired by camera S6 may be images acquired by all cameras on Shovel 100. Furthermore, the image indicated by the value of the item "Image" may be a video from the time the suspicious person appears in the image until the suspicious person disappears from the image.

[0088] Note that the items of information included in the suspicious person information are not limited to the example shown in Figure 6. The suspicious person information may include image data of the area around the shovel 100, etc.

[0089] Next, the operation of the SYS system for construction machinery will be explained with reference to Figure 7. Figure 7 is a sequence diagram illustrating the operation of the system for construction machinery.

[0090] In the SYS system for construction machinery of this embodiment, when the start control unit 32 determines that the state of the shovel 100 satisfies the start conditions (step S701), the information acquisition unit 33 acquires information to be used for theft prevention processing (step S702). Subsequently, the communication control unit 31 of the shovel 100 transmits the information acquired by the information acquisition unit 33 to the management device 300 (step S703).

[0091] When the control device 300 receives information from the shovel 100, the person recognition processing unit 340 acquires person features (step S704). Specifically, the person recognition processing unit 340 inputs the received information to the learning model 341 and obtains person features output from the learning model 341.

[0092] Next, the person identification processing unit 340 calculates suspicious person identification features (step S705). Specifically, the person identification processing unit 340 takes the person features output from the learning model 341 and inputs the value obtained by a pre-prepared mathematical formula to use as the suspicious person identification features.

[0093] Next, the person identification processing unit 340 determines whether the person identified by the person features is an unknown person or not (step S706). Specifically, the person identification processing unit 340 determines that the person identified by the person features is an unknown person if the suspicious person identification features are above a predetermined threshold.

[0094] In Figure 7, the process from step S707 to step S710 is the process when the person identified by the person features is determined to be an unknown person, and the process from step S711 to step S715 is the process when the person identified by the person features is determined to be a known person.

[0095] If the person identified by the person characteristics is an unknown person, the control device 300 generates instruction information via the operation instruction unit 350 to instruct the restriction of the operation of the shovel 100 (step S707), and the communication control unit 330 transmits the instruction information to the shovel 100 (step S708).

[0096] Furthermore, the management device 300, using the storage control unit 360, extracts suspicious person information when a person identified by personal features is determined to be an unknown person, and stores it in the suspicious person information storage unit 320 (step S709).

[0097] When the shovel 100 receives instruction information from the communication control unit 31, the operation control unit 35 restricts the operation of the shovel 100 according to the instruction information (step S710).

[0098] Specifically, the operation control unit 35 may, for example, restrict the operation of the shovel 100 so that the travel speed of the shovel 100 is below a predetermined speed.

[0099] By restricting the operation of the shovel 100 in this way, for example, if a suspicious person were to drive the shovel 100 with the intention of theft, they would only be able to drive it at a low speed, thus deterring theft.

[0100] Furthermore, by restricting the movement of the shovel 100 in this way, for example, if an unknown person is responsible for transporting the shovel 100 from one construction site to another, the shovel 100 can be moved without hindering the transport.

[0101] In this embodiment, the instruction information may include information instructing the shovel 100 to restrict its operation and information instructing the shovel 100 to emit a warning sound. This can deter the continuation of illegal activities by suspicious individuals.

[0102] Furthermore, according to this embodiment, since the learning model 341 identifies an unknown person who has not learned human information as a suspicious person, theft can be prevented even if the shovel 100 is used by an unspecified number of people working at a construction site. In addition, in this embodiment, the operation of the shovel 100 is restricted when it is identified as a suspicious person, so theft of the shovel 100 can be prevented even if it is left unlocked.

[0103] The above describes the process when a person identified by their characteristics is determined to be an unknown individual.

[0104] Next, we will explain the process when the person identified by the person features is determined to be a known person. In this embodiment, the learning model 341 may output a person ID to identify the person if the person identified by the person features is a known person.

[0105] In this embodiment, the management device 300, via the operation instruction unit 350, refers to the person information stored in the person information storage unit 310 that corresponds to the person ID output from the learning model 341 (step S711).

[0106] Next, the operation instruction unit 350 determines whether the person ID and the person information corresponding to it contain information that should restrict the operation of the shovel 100 (step S712).

[0107] If it is determined that there is no need to restrict the operation of the shovel 100, the control device 300 may terminate the process.

[0108] Specifically, the operation instruction unit 350 may determine whether or not it is necessary to restrict the operation of the shovel 100 based on the values ​​of the "licenses held" item and the "tasks that can be performed" item included in the person information. For example, the operation instruction unit 350 may determine that it is necessary to restrict the operation of the shovel 100 if the value of the "licenses held" item does not include a license for the shovel 100.

[0109] Furthermore, the operation instruction unit 350 may determine that it is necessary to restrict the operation of the shovel 100 if the value of the item "Licenses Held" includes a license for the shovel 100, but the value of the item "Permitted Work" is empty, or if the work to be performed at the construction site is not included in the value of the item "Permitted Work". In other words, when the operation instruction unit 350 determines that a person identified by the personal features is not a suspicious person, it determines whether it is necessary to restrict the operation of the shovel 100 according to the attributes of that person.

[0110] Furthermore, the management device 300 may maintain management information indicating the work schedule for each construction site, and the operation instruction unit 350 may refer to the management information to identify the work scheduled to be performed at the construction site.

[0111] The process from step S713 to step S715 in Figure 7 shows the process when it is determined in step S712 that it is necessary to restrict the operation of the shovel 100.

[0112] The operation instruction unit 350 generates instruction information to restrict the operation of the shovel 100 (step S713).

[0113] Here, the operation instruction unit 350 may generate instruction information corresponding to the person information. For example, if the value of the item "license held" does not include a license for the shovel 100, the operation instruction unit 350 generates instruction information to stop all operations of the shovel 100. In other words, the operation instruction unit 350 stops all operations of the shovel 100 if the person in the cabin 10 does not have a license for the shovel 100. Therefore, in this embodiment, it is possible to prevent unlicensed persons from operating the shovel 100.

[0114] Furthermore, the operation instruction unit 350 may, for example, generate instruction information to disable the operation of attachments on the shovel 100 if the value of the item "License Held" includes a license for the shovel 100, but the value of the item "Permitted Work" is only "Driving". In other words, the operation instruction unit 350 restricts operations other than those that perform permitted work. Therefore, in this embodiment, it is possible to prevent a person inside the cabin 10 from performing work that is not permitted for them.

[0115] Thus, if the operation instruction unit 350 determines that the person identified by the personal features is not a suspicious person and that it is necessary to restrict the operation of the shovel 100, it instructs the shovel 100 to restrict its operation according to the attributes of this person.

[0116] The processes in steps S714 and S715 in Figure 7 are the same as those in steps S708 and S710, so their explanation will be omitted.

[0117] In this embodiment, even if the person detected inside the cabin 10 is not a suspicious person but a known person, the operation of the shovel 100 can be restricted according to the attributes of the detected person. Therefore, according to this embodiment, safety can be improved by ensuring that only appropriate people operate the shovel 100 at the construction site.

[0118] Furthermore, in this embodiment, suspicious person information is stored in the suspicious person information storage unit 320. The suspicious person information stored in the suspicious person information storage unit 320 may be displayed on a display device 305 or the like of the management device 300. In this case, the suspicious person information may be displayed in response to the operation of the user of the management device 300, or each time suspicious person information is stored in the suspicious person information storage unit 320, the newly stored suspicious person information may be displayed in a pop-up format or the like. In addition, when new suspicious person information is stored in the suspicious person information storage unit 320, the management device 300 may display a notification on the display device 305 indicating that the suspicious person information storage unit 320 has been updated.

[0119] In this embodiment, by displaying information about suspicious individuals, administrators of the SYS system for construction machinery can gain a detailed understanding of the situation when an unknown person is detected, thereby contributing to the prevention of theft.

[0120] Furthermore, if the person identification processing unit 340 determines that the person identified by the person characteristics is an unknown person, the management device 300 may send a notification to the support device 200 indicating that an unknown person has entered the cabin 10. In this way, for example, workers working at a construction site where the shovel 100 is located can be notified of the intrusion of an intruder into the shovel 100.

[0121] In this embodiment, the human identification processing unit 340 and the operation instruction unit 350 of the management device 300 may be provided on the controller 30 of the shovel 100. In that case, the human identification processing unit 340 and the operation instruction unit 350 do not need to be provided on the management device 300.

[0122] In this way, by providing the human identification processing unit 340 and the operation instruction unit 350 on the shovel 100, it is possible to help prevent the theft of the shovel 100 even in environments where, for example, a communication environment between the shovel 100 and the management device 300 is not in place.

[0123] Furthermore, in this embodiment, the human detection unit 34 of the shovel 100 may be provided on the control device 300.

[0124] Furthermore, the processing performed by the human identification processing unit 340 and the operation instruction unit 350 in this embodiment may be applied to systems other than the SYS system for construction machinery. Specifically, for example, the processing performed by the human identification processing unit 340 and the operation instruction unit 350 may be applied to general security systems. A security system may be, for example, a system consisting of security cameras and management devices installed in a parking lot, or a system consisting of security cameras and management devices installed in an apartment building. Alternatively, a security system may be a system consisting of security cameras and management devices installed in a store.

[0125] Preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the embodiments described above. Various modifications, substitutions, etc., can be applied to the embodiments described above without departing from the scope of the present invention. Furthermore, features described separately can be combined as long as no technical inconsistencies arise. [Explanation of Symbols]

[0126] 1. Lower running body 2. Swivel mechanism 3. Upper rotating body 4 Boom 5 Arms 6 buckets 30 controllers 32 Start Control Unit 33 Information Acquisition Department 34 Human Detection Unit 35. Operation Control Unit 100 Shovel 200 Support equipment 300 Management device 310 Person information storage section 320 Suspicious Person Information Storage Unit 340 people identification processing unit 341 Learning Models 350 Operation instruction section 360 Storage Control Unit

Claims

1. A learning model that takes information acquired from sensors on construction machinery as input and outputs information to identify a person, and a discrimination unit that determines whether or not a person is suspicious based on the suspiciousness of the person derived using the person identification information output from the learning model, A system for construction machinery, comprising: an operation instruction unit that instructs the construction machinery to perform an action according to the result of the discrimination by the discrimination unit.

2. The information used to identify the person is a person feature that identifies the person, The aforementioned discrimination unit is Using the aforementioned person characteristics, we derive the suspicious person identification characteristics that indicate the likelihood of the person being suspicious. The aforementioned suspicious person identification features are, The system for construction machinery according to claim 1, wherein the feature is designed such that the numerical values ​​differ when the learning model has not learned person information about the person identified by the person feature compared to when it has learned person information about the person identified by the person feature.

3. The aforementioned operation instruction unit, If the discrimination unit determines that the person is a suspicious person, A system for a construction machine according to claim 1 or 2, which instructs the construction machine to restrict its operation, issue an alarm, or notify an administrator.

4. The learning model has a person information storage unit that stores person information about the person that was trained on the learning model, The aforementioned operation instruction unit, If the discrimination unit determines that the person is not a suspicious person, A system for construction machinery according to claim 1 or 2, which refers to the information identifying the person and the corresponding person information, and determines whether it is necessary to restrict the operation of the construction machinery according to the attributes of the person.

5. The aforementioned operation instruction unit, The construction machine system according to claim 4, which, when it is determined that it is necessary to restrict the operation of the construction machine, instructs the construction machine to restrict its operation according to the attributes of the person.

6. If the aforementioned attribute indicates that the person does not possess a license for the construction machinery, The aforementioned operation instruction unit, The construction machine system according to claim 5, which instructs the construction machine to stop all operations.

7. If the aforementioned attribute indicates the type of work permitted for the person, The aforementioned operation instruction unit, The system for a construction machine according to claim 5, which instructs the construction machine to restrict actions other than those for performing the work permitted to the person.

Images