Identification system, identification device, and identification method

Abstract

To efficiently update a trained model for use in person identification.SOLUTION: An identification system comprises: a data acquisition unit which acquires data about an object person; an identification unit which inputs the data to a trained model that has been trained to identify a prescribed number of identification objects, to identify which of the identification objects the object person corresponds to; a number-of-unknown-persons determination unit which determines the number of unknown persons, being object persons not identified as any of the identification objects; and a display unit which, if the number of unknown persons is equal to or larger than a prescribed number, displays a notification indicating that processing for updating the trained model should be executed.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to an identification system, an identification device, and an identification method. [Background technology]

[0002] There is known a technology for identifying a person using a machine learning model. For example, Patent Document 1 discloses a person authentication device that calculates facial features from a face image extracted from a video including a person, and authenticates the person from the facial features using a face authentication model of a registered person. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-181157 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technology, it takes a lot of work to update a trained model in order to add a person to be identified.

[0005] One aspect of the present invention aims to efficiently update a trained model used for person identification. [Means for solving the problem]

[0006] An identification system according to one embodiment of the present invention includes a data acquisition unit that acquires data regarding a subject, an identification unit that identifies which of a predetermined number of identification targets the subject is by inputting the data into a trained model that has been trained to be able to identify the predetermined number of identification targets, a number-of-unknowns determination unit that determines the number of unknown persons that are subjects not identified by any of the identification targets, and a display unit that displays a notification that a process to update the trained model should be executed when the number of unknown persons is equal to or greater than a predetermined threshold. Effect of the Invention

[0007] According to one aspect of the present invention, it is possible to efficiently update a trained model used for person identification. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of an overall configuration of an identification system according to a first embodiment. [Diagram 2] FIG. 1 is a side view showing an example of a shovel according to a first embodiment. [Diagram 3] FIG. 2 is a top view showing an example of a shovel according to the first embodiment. [Figure 4] FIG. 1 is a diagram illustrating an example of a configuration of a shovel according to a first embodiment. [Diagram 5] FIG. 2 is a block diagram illustrating an example of a computer according to the first embodiment. [Figure 6] 1 is a block diagram showing an example of a functional configuration of an identification system according to a first embodiment. [Figure 7] FIG. 4 is a diagram illustrating an example of sensor data according to the first embodiment. [Figure 8] FIG. 2 is a diagram illustrating an example of an identification model according to the first embodiment. [Figure 9] 4 is a block diagram showing an example of a functional configuration of a model update unit according to the first embodiment. FIG. [Figure 10] FIG. 4 is a diagram showing an example of data processing according to the first embodiment; [Figure 11] FIG. 4 is a diagram showing an example of a clustering result according to the first embodiment. [Figure 12] FIG. 2 is a diagram showing an example of a personal identification database according to the first embodiment. [Figure 13] FIG. 4 is a sequence diagram showing an example of an identification method according to the first embodiment. [Figure 14] 5 is a flowchart showing an example of a clustering process according to the first embodiment. [Figure 15] 5 is a flowchart showing an example of an update determination process according to the first embodiment. [Figure 16] FIG. 4 is an image diagram showing an example of a management screen according to the first embodiment. [Figure 17] FIG. 4 is an image diagram showing an example of a notification screen according to the first embodiment. [Figure 18] FIG. 13 is a diagram illustrating an example of an identification model according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted.

[0010] [First embodiment] A first embodiment of the present invention is an identification system for identifying an operator of a work machine. The identification system has a function of identifying the operator based on a trained identification model. The trained identification model is trained so as to be able to identify a predetermined number of people. Hereinafter, a person to be identified by the trained identification model is also simply referred to as an "identification target", and a person who cannot be identified by the trained model is also referred to as an "unknown person".

[0011] In this embodiment, an example will be described in which a shovel is used as an example of a work machine, but this embodiment does not limit the work machine to a shovel. The work machine may be any work machine that operates according to the operation of an operator. For example, the work machine may be a construction machine, a standard machine, an application machine, a forestry machine, or a transport machine based on a hydraulic shovel. In addition, the work machine may be a road machine including an asphalt finisher, etc., a forklift, a crawler crane, or an overhead crane that can be moved according to the operation of an operator, or a fixed power crane.

[0012] For work machines, there is a demand for a system that can automatically identify whether the operator is a specific person. If it is possible to identify the operator of a work machine, it is expected that this can be applied to, for example, health care, operator management systems, automatic daily report creation to report the work content of each operator, and information processing systems to accumulate events that lead to serious accidents (near-misses) and take measures to prevent recurrence.

[0013] There is known a technology for identifying an individual using an external device such as a fingerprint authentication device. However, a fingerprint authentication device or the like is not usually installed on a work machine, and it is necessary to install such an external device on the target work machine. In a work machine, it is preferable to be able to identify an individual using a simple device such as a camera.

[0014] Therefore, in this embodiment, the operator is identified by inputting data obtainable by the work machine into a trained identification model. Among the data obtainable by the work machine, image data or machine operation data, etc., can be used for personal identification. Note that image data includes still images and videos. With regard to images, raw images (full-body images), facial images, skeletal data, equipment images, etc. are effective data for personal identification. Skeletal data and machine operation data are capable of capturing subtle habits of an individual, and are effective data for personal identification.

[0015] It is expected that the performance of personal identification will be improved by using a facial image. However, since a work machine operator may be wearing, for example, a hat, a mask, sunglasses, etc., it is often impossible to capture a clear facial image. If a clear facial image cannot be captured, personal identification using a facial image is difficult. The accuracy of personal identification can be improved by using a whole-body image, skeletal data, equipment images, machine operation data, etc. in addition to a facial image.

[0016] The present embodiment aims to provide a system or device capable of efficiently updating a discrimination model used for identifying a person. In the present embodiment, when the number of unknown persons that cannot be identified by the discrimination model is equal to or greater than a predetermined threshold, a notification is output to the effect that processing to update the discrimination model should be executed. In one aspect, according to the present embodiment, the model is updated at an appropriate timing, and the effort required for updating the model can be reduced.

[0017] <Overall composition> The overall configuration of the identification system according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the overall configuration of the identification system according to the first embodiment.

[0018] 1, the identification system 1000 includes a work machine 100, an identification device 200, and a terminal device 300. The work machine 100, the identification device 200, and the terminal device 300 are connected to each other so as to be able to communicate data with each other via a communication network N such as a LAN (Local Area Network) or the Internet.

[0019] The work machine 100 is a work machine that operates according to the operation of an operator. The work machine 100 uses various sensors to collect data related to the operator. The work machine 100 transmits the collected data to the identification device 200 at any timing. The work machine 100 may transmit the collected data to the identification device 200 in response to a request from the identification device 200.

[0020] The identification device 200 is an information processing device such as a personal computer, workstation or server that identifies the operator of the work machine 100. The identification device 200 identifies which of the identification targets the operator is, using a trained identification model that has been trained to be able to identify a predetermined number of identification targets. In this embodiment, the identification device 200 outputs as an identification result that the operator is an unknown person who is not one of the identification targets.

[0021] The terminal device 300 is an information processing terminal such as a personal computer, a smartphone, or a tablet terminal operated by a user of the identification system 1000. The user of the identification system 1000 is, for example, an administrator who operates the identification system 1000. The terminal device 300 displays a management screen for managing trained identification models. The terminal device 300 is an example of a display unit.

[0022] The overall configuration of the identification system 1000 shown in FIG. 1 is one example, and various system configuration examples are possible depending on the application and purpose. For example, the identification system 1000 may include multiple units of one or more of the work machine 100, the identification device 200, and the terminal device 300. For example, the identification device 200 may be realized by multiple computers, or may be realized as a cloud computing service. For example, the identification device 200 and the terminal device 300 may be realized by a standalone computer that has the functions that each should have. The division of devices such as the work machine 100, the identification device 200, and the terminal device 300 shown in FIG. 1 is one example.

[0023] <Outline of the excavator> An overview of the shovel according to the present embodiment will be described with reference to Fig. 2 and Fig. 3. Fig. 2 is a top view showing an example of the shovel according to the first embodiment. Fig. 3 is a side view showing an example of the shovel according to the first embodiment.

[0024] 2 and 3, the shovel 100 includes a lower traveling body 1, an upper rotating body 3 mounted on the lower traveling body 1 so as to be rotatable via a rotating mechanism 2, an attachment AT for performing various tasks, and a cabin 10. Hereinafter, the front of the shovel 100 (upper rotating body 3) corresponds to the direction in which the attachment for the upper rotating body 3 extends when the shovel 100 is viewed in a plan view (top view) from directly above along the rotation axis of the upper rotating body 3. In addition, the left and right sides of the shovel 100 (upper rotating body 3) correspond to the left and right sides, respectively, as viewed from an operator seated in a driver's seat in the cabin 10.

[0025] The lower traveling body 1 includes, for example, a pair of left and right crawlers 1C. Specifically, the crawlers 1C include a left crawler 1CL and a right crawler 1CR. The lower traveling body 1 causes the excavator 100 to travel by hydraulically driving the left crawler 1CL and the right crawler 1CR by a left traveling hydraulic motor 2ML and a right traveling hydraulic motor 2MR, respectively.

[0026] The upper rotating body 3 rotates relative to the lower traveling body 1 as a result of the rotating mechanism 2 being hydraulically driven by the rotating hydraulic motor 2A. In other words, the rotating hydraulic motor 2A is a rotating drive part that drives the upper rotating body 3 as a driven part, and can change the orientation of the upper rotating body 3.

[0027] The attachment AT includes a boom 4 , an arm 5 , and a bucket 6 .

[0028] A boom 4 is attached to the front center of the upper rotating body 3 so as to be able to tilt up and down, an arm 5 is attached to the tip of the boom 4 so as to be able to rotate up and down, and a bucket 6 is attached to the tip of the arm 5 so as to be able to rotate up and down.

[0029] The bucket 6 is used, for example, for excavation work. In addition, instead of the bucket 6, another working tool may be attached to the tip of the arm 5 depending on the work content, etc. The other working tool may be, for example, another type of bucket, such as a large bucket, a slope bucket, or a dredging bucket. The other working tool may also be a type of working tool other than a bucket, such as an agitator, a breaker, or a grapple.

[0030] The boom 4, the arm 5, and the bucket 6 are hydraulically driven by hydraulic oil discharged from a main pump 14 through a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, which serve as hydraulic actuators.

[0031] The shovel 100 may be configured such that some of the driven elements, such as the lower traveling body 1, the upper rotating body 3, the boom 4, the arm 5, and the bucket 6, are electrically driven. In other words, the shovel 100 may be a hybrid shovel, an electric shovel, or the like, in which some of the driven elements are driven by an electric actuator.

[0032] The operating device 26 is a device used by an operator to operate the actuator. The operating device 26 includes, for example, an operating lever and an operating pedal. The actuator includes at least one of a hydraulic actuator and an electric actuator.

[0033] The controller 30 is a control device for controlling the shovel 100. For example, the controller 30 is mainly composed of a computer including a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), a non-volatile auxiliary storage device, various input / output interfaces, etc. The controller 30 reads out a program from the non-volatile storage device, loads it into the volatile storage device, and has the CPU execute the program to realize various functions.

[0034] The boom angle sensor S1 is attached to the boom 4 and detects the elevation angle of the boom 4 relative to the upper rotating body 3 (hereinafter referred to as the "boom angle"), for example, the angle formed by a straight line connecting the fulcrums at both ends of the boom 4 relative to the rotation plane of the upper rotating body 3 in a side view. The boom angle sensor S1 may include, for example, a rotary encoder, an acceleration sensor, a six-axis sensor, an inertial measurement unit (IMU), etc. The boom angle sensor S1 may also include a potentiometer using a variable resistor, a cylinder stroke sensor that detects the stroke amount of a hydraulic cylinder (boom cylinder 7) corresponding to the boom angle, etc. The same applies to the arm angle sensor S2 and the bucket angle sensor S3 below. A detection signal corresponding to the boom angle by the boom angle sensor S1 is taken into the controller 30.

[0035] The arm angle sensor S2 is attached to the arm 5 and detects the rotation angle of the arm 5 relative to the boom 4 (hereinafter referred to as the “arm angle”), for example, the angle formed by a line connecting the fulcrums at both ends of the arm 5 with respect to a line connecting the fulcrums at both ends of the boom 4 in a side view. A detection signal corresponding to the arm angle by the arm angle sensor S2 is input to the controller 30.

[0036] The bucket angle sensor S3 is attached to the bucket 6 and detects the rotation angle of the bucket 6 with respect to the arm 5 (hereinafter referred to as the “bucket angle”), for example, the angle formed by a line connecting the fulcrum of the bucket 6 and its tip (tip) with respect to a line connecting the fulcrums of both ends of the arm 5 in a side view. A detection signal corresponding to the bucket angle by the bucket angle sensor S3 is input to the controller 30.

[0037] The machine body inclination sensor S4 detects the inclination state of the machine body (upper rotating body 3 or lower running body 1) with respect to a horizontal plane. The machine body inclination sensor S4 is attached to, for example, the upper rotating body 3, and detects the inclination angles around two axes in the forward / backward and left / right directions (hereinafter, "fore / aft inclination angle" and "left / right inclination angle") of the shovel 100 (i.e., upper rotating body 3). The machine body inclination sensor S4 may include, for example, a rotary encoder, an acceleration sensor, a six-axis sensor, an IMU, etc. A detection signal corresponding to the inclination angle (fore / aft inclination angle and left / right inclination angle) by the machine body inclination sensor S4 is input to the controller 30.

[0038] The rotation angle sensor S5 outputs detection information related to the rotation state of the upper rotating body 3. The rotation angle sensor S5 detects, for example, the rotation angular velocity and rotation angle of the upper rotating body 3. The rotation angle sensor S5 may include, for example, a gyro sensor, a resolver, a rotary encoder, etc. A detection signal corresponding to the rotation angle and the rotation angular velocity of the upper rotating body 3 detected by the rotation angle sensor S5 is input to the controller 30.

[0039] The positioning device PS measures the position and orientation of the upper rotating body 3. The positioning device PS is, for example, a Global Navigation Satellite System (GNSS) compass, and detects the position and orientation of the upper rotating body 3, and a detection signal corresponding to the position and orientation of the upper rotating body 3 is input to the controller 30. Furthermore, among the functions of the positioning device PS, the function of detecting the orientation of the upper rotating body 3 may be substituted by a direction sensor attached to the upper rotating body 3.

[0040] The cabin 10 is a cockpit where an operator sits, and is mounted on the front left side of the upper rotating body 3.

[0041] The imaging device S11 is attached inside the cabin 10. The imaging device S11 is installed at a position where the operator in the cabin 10 is within the angle of view. The imaging device S11 has an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor) and acquires image data of at least the upper body of the operator. The imaging device S11 may be, for example, a monocular wide-angle camera having a very wide angle of view. The imaging device S11 may be a stereo camera or the like. The image data captured by the imaging device S11 is taken into the controller 30.

[0042] The communication device T1 communicates with external devices through a predetermined network including a mobile communication network with a base station as a terminal, a satellite communication network, the Internet, etc. The communication device T1 is, for example, a mobile communication module compatible with mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), and 5G (5th Generation), or a satellite communication module for connecting to a satellite communication network.

[0043] The external device is, for example, an identification device 200 that identifies an operator who operates the shovel 100. This allows the shovel 100 to transmit (upload) various types of information to the identification device 200 and receive various signals (for example, information signals and control signals) from the identification device 200.

[0044] The identification device 200 may be, for example, a cloud server or an on-premise server installed in a remote location different from the work site of the shovel 100. The identification device 200 may be, for example, an edge server installed inside the work site of the shovel 100 (for example, a management office of the work site) or in a location relatively close to the work site (for example, a communication facility such as a nearby base station). The identification device 200 may be a management terminal device used at the work site.

[0045] In response to the operation of an operator in the cabin 10, the shovel 100 operates actuators (e.g., hydraulic actuators) to drive operating elements (hereinafter referred to as "driven elements") such as the lower running body 1, upper rotating body 3, boom 4, arm 5, and bucket 6.

[0046] Fig. 4 is a schematic diagram showing an example of the configuration of the shovel according to the first embodiment. In Fig. 4, a mechanical power system, a hydraulic oil line, a pilot line, and an electric control system are indicated by double lines, solid lines, dashed lines, and dotted lines, respectively.

[0047] The drive system of the excavator 100 according to this embodiment includes the engine 11, the regulator 13, the main pump 14, and the control valve unit 17. Moreover, the hydraulic drive system of the excavator 100 according to this embodiment includes hydraulic actuators such as the traveling hydraulic motors 1L, 1R, the swing hydraulic motor 2A, the boom cylinder 7, the arm cylinder 8, and the bucket cylinder 9, which hydraulically drive the lower traveling body 1, the upper rotating body 3, the boom 4, the arm 5, and the bucket 6, respectively, as described above.

[0048] The engine 11 is a main power source in the hydraulic drive system, and is mounted, for example, on the rear of the upper rotating body 3. Specifically, the engine 11 rotates at a constant speed at a preset target speed under direct or indirect control by a controller 30 (described later), and drives the main pump 14 and the pilot pump 15. The engine 11 is, for example, a diesel engine that uses diesel as fuel.

[0049] The regulator 13 controls the discharge amount of the main pump 14. For example, the regulator 13 adjusts the angle (tilt angle) of the swash plate of the main pump 14 in response to a control command from the controller 30.

[0050] The main pump 14 is mounted on the rear of the upper rotating body 3, for example, like the engine 11, and supplies hydraulic oil to the control valve unit 17 through a high-pressure hydraulic line. The main pump 14 is driven by the engine 11 as described above. The main pump 14 is, for example, a variable displacement hydraulic pump, and as described above, under the control of the controller 30, the tilt angle of the swash plate is adjusted by the regulator 13 to adjust the stroke length of the piston, thereby controlling the discharge flow rate (discharge pressure).

[0051] The control valve unit 17 is a hydraulic control device that controls the hydraulic system in the excavator 100. In this embodiment, the control valve unit 17 includes control valves 171 to 176. The control valve 175 includes a control valve 175L and a control valve 175R, and the control valve 176 includes a control valve 176L and a control valve 176R. The control valve unit 17 is configured to selectively supply hydraulic oil discharged by the main pump 14 to one or more hydraulic actuators through the control valves 171 to 176. The control valves 171 to 176 control, for example, the flow rate of hydraulic oil flowing from the main pump 14 to the hydraulic actuators and the flow rate of hydraulic oil flowing from the hydraulic actuators to a hydraulic oil tank. The hydraulic actuators include a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, traveling hydraulic motors 1L and 1R, and a swing hydraulic motor 2A. More specifically, the control valve 171 corresponds to the traveling hydraulic motor 1L, the control valve 172 corresponds to the traveling hydraulic motor 1R, and the control valve 173 corresponds to the swing hydraulic motor 2A. In addition, the control valve 174 corresponds to the bucket cylinder 9, the control valve 175 corresponds to the boom cylinder 7, and the control valve 176 corresponds to the arm cylinder 8.

[0052] The pilot pump 15 is an example of a pilot pressure generating device, and is configured to be able to supply hydraulic oil to hydraulic control devices via a pilot line. In this embodiment, the pilot pump 15 is a fixed displacement hydraulic pump. However, the pilot pressure generating device may be realized by the main pump 14. That is, the main pump 14 may have a function of supplying hydraulic oil to various hydraulic control devices via a pilot line, in addition to a function of supplying hydraulic oil to the control valve unit 17 via a hydraulic oil line. In this case, the pilot pump 15 may be omitted.

[0053] The discharge pressure sensor 28 is configured to detect the discharge pressure of the main pump 14. In this embodiment, the discharge pressure sensor 28 outputs a detected value to the controller 30. The discharge pressure sensor 28 includes, for example, discharge pressure sensors 28L and 28R, as described later.

[0054] The operating device 26 is a device used by an operator to operate the actuator. The operating device 26 includes, for example, an operating lever and an operating pedal. The actuator includes at least one of a hydraulic actuator and an electric actuator.

[0055] The operation sensor 29 is configured to detect the operation content of the operator using the operating device 26. In this embodiment, the operation sensor 29 detects the operation direction and operation amount of the operating device 26 corresponding to each actuator, and outputs the detected value to the controller 30. In this embodiment, the controller 30 controls the opening area of ​​the proportional valve 31 according to the output of the operation sensor 29. Then, the controller 30 supplies the hydraulic oil discharged by the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17. The pressure of the hydraulic oil (pilot pressure) supplied to each pilot port is, in principle, a pressure according to the operation direction and operation amount of the operating device 26 corresponding to each hydraulic actuator. In this way, the operating device 26 is configured to be able to supply the hydraulic oil discharged by the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17.

[0056] The proportional valve 31, which functions as a control valve for machine control, is disposed in a pipe connecting the pilot pump 15 and the pilot port of the control valve in the control valve unit 17, and is configured so as to be able to change the flow passage area of ​​the pipe. In this embodiment, the proportional valve 31 operates in response to a control command output by the controller 30. Therefore, the controller 30 can supply the hydraulic oil discharged by the pilot pump 15 to the pilot port of the control valve in the control valve unit 17 via the proportional valve 31, regardless of the operation of the operating device 26 by the operator.

[0057] With this configuration, the controller 30 can operate the hydraulic actuator corresponding to a specific operating device 26 even when no operation is being performed on that specific operating device 26.

[0058] The control system of the shovel 100 according to this embodiment includes a controller 30, a display device 40, an input device 42, a storage medium 47, and a communication device T1.

[0059] The controller 30 is provided, for example, in the cabin 10, and performs drive control of the shovel 100. The functions of the controller 30 may be realized by any hardware, software, or a combination thereof.

[0060] For example, the controller 30 sets a target rotation speed based on an operation by an operator, etc., and performs drive control to rotate the engine 11 at a constant speed.

[0061] Furthermore, for example, the controller 30 outputs a control command to the regulator 13 as necessary to change the discharge amount of the main pump 14.

[0062] Note that some of the functions of the controller 30 may be realized by another controller (control device). That is, the functions of the controller 30 may be realized in a distributed manner by a plurality of controllers.

[0063] The display device 40 is provided in a location that is easily visible to an operator seated in the cabin 10, and displays various information images under the control of the controller 30. The display device 40 may be connected to the controller 30 via an in-vehicle communication network such as a Controller Area Network (CAN), or may be connected to the controller 30 via a one-to-one dedicated line.

[0064] The input device 42 is provided within reach of an operator seated in the cabin 10, accepts various operational inputs by the operator, and outputs signals corresponding to the operational inputs to the controller 30. The input device 42 includes a touch panel mounted on the display of a display device that displays various information images, knob switches provided at the tips of the lever portions of the operation levers 26A to 26B, button switches, levers, toggles, rotary dials, etc., provided around the display device 40. A signal corresponding to the content of an operation performed on the input device 42 is taken into the controller 30.

[0065] The storage medium 47 is provided, for example, in the cabin 10, and stores various pieces of information under the control of the controller 30. The storage medium 47 is, for example, a non-volatile storage medium such as a semiconductor memory. The storage medium 47 may store information output by various devices during operation of the shovel 100, or may store information obtained via various devices before operation of the shovel 100 is started.

[0066] <Computer> The controller 30, the identification device 200, and the terminal device 300 of the shovel 100 are realized by, for example, a computer. Fig. 5 is a block diagram showing an example of the hardware configuration of the computer according to the first embodiment.

[0067] 5, the computer 500 includes a central processing unit (CPU) 501, a read only memory (ROM) 502, a random access memory (RAM) 503, a hard disk drive (HDD) 504, an input device 505, a display device 506, a communication interface (I / F) 507, and an external I / F 508. The CPU 501, the ROM 502, and the RAM 503 form a so-called computer. Each piece of hardware of the computer 500 is connected to each other via a bus line 509. The input device 505 and the display device 506 may be connected to the external I / F 508 for use.

[0068] The CPU 501 is a calculation device that reads out programs and data from a storage device such as the ROM 502 or the HDD 504 onto the RAM 503 and executes processing to realize the overall control and functions of the computer 500. The computer 500 may have a GPU (Graphics Processing Unit) in addition to or instead of the CPU 501.

[0069] The ROM 502 is an example of a non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. The ROM 502 functions as a main storage device that stores various programs and data required for the CPU 501 to execute various programs installed in the HDD 504. Specifically, the ROM 502 stores boot programs such as a Basic Input / Output System (BIOS) and an Extensible Firmware Interface (EFI) that are executed when the computer 500 is started, and data such as an Operating System (OS) setting and a network setting.

[0070] The RAM 503 is an example of a volatile semiconductor memory (storage device) in which programs and data are erased when the power is turned off. The RAM 503 is, for example, a dynamic random access memory (DRAM) or a static random access memory (SRAM). The RAM 503 provides a working area in which various programs installed in the HDD 504 are expanded when the CPU 501 executes them.

[0071] The HDD 504 is an example of a non-volatile storage device that stores programs and data. The programs and data stored in the HDD 504 include an OS, which is basic software that controls the entire computer 500, and applications that provide various functions on the OS. Note that the computer 500 may use a storage device that uses a flash memory as a storage medium (e.g., an SSD (Solid State Drive) or the like) instead of the HDD 504.

[0072] The input device 505 includes a touch panel, operation keys or buttons, a keyboard or mouse, a microphone for inputting sound data such as voice, etc., which are used by the user to input various signals.

[0073] The display device 506 is composed of a display such as a liquid crystal display or an organic EL (Electro-Luminescence) display for displaying a screen, a speaker for outputting sound data such as voice, and the like.

[0074] The communication I / F 507 is an interface that connects to a communication network and enables the computer 500 to perform data communication.

[0075] The external I / F 508 is an interface with external devices, such as a drive device 510.

[0076] The drive device 510 is a device for setting the recording medium 511. The recording medium 511 here includes media that record information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, or a magneto-optical disk. The recording medium 511 may also include semiconductor memories that record information electrically, such as a ROM or a flash memory. This allows the computer 500 to read and / or write data from and to the recording medium 511 via the external I / F 508.

[0077] The various programs to be installed in the HDD 504 are installed, for example, by setting the distributed recording medium 511 in a drive device 510 connected to the external I / F 508 and reading out the various programs recorded in the recording medium 511 by the drive device 510. Alternatively, the various programs to be installed in the HDD 504 may be installed by being downloaded from a network different from the communication network via the communication I / F 507.

[0078] <Functional configuration> The functional configuration of the identification system 1000 according to this embodiment will be described with reference to Fig. 6. Fig. 6 is a block diagram showing an example of the functional configuration of the identification system according to the first embodiment.

[0079] <Work machines> As shown in FIG. 6, the work machine 100 includes a data generating unit 101 and a data output unit 102.

[0080] The data generating unit 101 and the data output unit 102 are realized, for example, by a process executed by a CPU 501 according to a program loaded from an HDD 504 onto a RAM 503 shown in FIG.

[0081] The data generating unit 101 generates data related to the operator of the work machine 100 (hereinafter also referred to as the "subject"). The data related to the subject includes image data and machine operation data. Hereinafter, the data generated by the data generating unit 101 is also referred to as "sensor data".

[0082] Fig. 7 is a diagram showing an example of sensor data according to the first embodiment. As shown in Fig. 7, the sensor data includes image data, machine operation data, and the like. The image data is an image of an operator seated in the cockpit of the work machine 100 captured by the imaging device S11. The image data is, for example, an image including the operator's face, an image including the upper half of the operator's body, or an image including the operator's entire body.

[0083] The machine operation data is time-series data of observed values ​​observed by various sensors S1-S5 equipped on the work machine 100 when the operator of the work machine 100 operates the work machine 100. Examples of the machine operation data include travel lever data and turning data. The machine operation data is not limited to travel lever data and turning data, and any data observable by a sensor equipped on the work machine 100 can be used.

[0084] The image data and the sensor data may be acquired by corresponding the observation time. Since the sensor data is time-series data, when the image data captured by the imaging device S11 is a still image, the image data may be generated by arranging in time series still images captured near the center of the time interval during which the sensor data is observed.

[0085] The data output unit 102 outputs the sensor data generated by the data generation unit 101 to the identification device 200. The data output unit 102 transmits the sensor data to the identification device 200 using the communication device T1. The data output unit 102 may transmit the sensor data to the identification device 200 every time the sensor data is generated. The data output unit 102 may transmit the sensor data generated in the immediately previous time interval to the identification device 200 at a predetermined time interval. The data output unit 102 may transmit untransmitted sensor data to the identification device 200 in response to a request from the identification device 200.

[0086] The data output unit 102 stores the sensor data in a portable storage medium connected to the external I / F 508. The portable storage medium storing the sensor data may be connected to the external I / F 508 of the identification device 200. The identification device 200 may read the sensor data from the portable storage medium connected to the external I / F 508.

[0087] <Identification device> As shown in FIG. 6 , the classification device 200 includes a model storage unit 201, a model update unit 202, a data acquisition unit 203, a data processing unit 204, a classification unit 205, a score calculation unit 206, a clustering unit 207, a number of unknowns determination unit 208, a screen provision unit 209, and a data storage unit 210.

[0088] The model update unit 202, data acquisition unit 203, data processing unit 204, classification unit 205, score calculation unit 206, clustering unit 207, number of unknowns determination unit 208 and screen provision unit 209 are realized, for example, by processing executed by a CPU 501 of a program expanded from HDD 504 to RAM 503 shown in FIG. 5.

[0089] The model storage unit 201 and the data storage unit 210 are realized, for example, using the RAM 503 or the HDD 504 shown in FIG.

[0090] Trained discrimination models and information related to the discrimination models are stored in the model storage unit 201. The discrimination models are trained to be able to discriminate a predetermined number of discrimination targets.

[0091] The discrimination model may be trained based on training data collected in advance. The discrimination model may be machine-learned by supervised learning. As an example, the training data is data in which identification information for identifying the identification target is added as correct answer data to input data obtained by processing sensor data related to the identification target.

[0092] Fig. 8 is a diagram showing an example of a discrimination model according to the first embodiment. As shown in Fig. 8, the discrimination model 400 includes a feature extraction unit 401 and a probability calculation unit 402. Input data obtained by processing sensor data related to a subject is input to the discrimination model 400.

[0093] The feature extraction unit 401 generates feature data indicating the features of a subject based on the input data input to the discrimination model 400. The feature data is a feature vector with a number of dimensions depending on the size of the image data. Hereinafter, the number of dimensions of the feature data is assumed to be D.

[0094] The feature extraction unit 401 is a machine learning model based on deep learning. The feature extraction unit 401 is, for example, a convolutional neural network (CNN). Specifically, the convolutional neural network is ResNet or VGG16. The feature extraction unit 401 is not limited to the convolutional neural network, and other classifiers may be used. The other classifier may be, for example, a uniquely designed feature extractor.

[0095] The probability calculation unit 402 calculates, for each classification target, the probability that the subject is the classification target based on the feature data generated by the feature extraction unit 401, and outputs the calculated probability.

[0096] The probability calculation unit 402 includes a fully connected layer and an activation function layer of a neural network. The activation function is, for example, a softmax function. The output dimension (the number of output nodes) of the fully connected layer is, for example, the same as the number of classification targets. Hereinafter, the number of classification targets is assumed to be M. The probability calculation unit 402 outputs a probability value for each classification target by inputting the output of the fully connected layer to a softmax function. The probability calculation unit 402 is not limited to the fully connected layer and the activation function layer, and other classifiers may be used. The other classifiers may be, for example, a support vector machine (SVM) or the like.

[0097] The model update unit 202 updates the identification model 400 stored in the model storage unit 201. The model update unit 202 updates the identification model 400 in response to an instruction from a user of the terminal device 300. The model update unit 202 may automatically update the identification model 400 when the number of unknown persons becomes equal to or greater than a predetermined threshold.

[0098] Fig. 9 is a diagram showing an example of the functional configuration of a model updating unit according to the first embodiment. As shown in Fig. 9, the model updating unit 202 includes an additional learning unit 251, a re-learning unit 252, and a deletion unit 253. The additional learning unit 251 is an example of a first updating unit. The re-learning unit 252 is an example of a second updating unit. The deletion unit 253 is an example of a third updating unit.

[0099] The additional learning unit 251 executes additional learning processing. The additional learning processing is processing for updating the probability calculation unit 402 of the discrimination model 400 based on feature data generated by the feature extraction unit 401 of the discrimination model 400. The additional learning unit 251 updates the probability calculation unit 402 using only feature data related to unknown persons. The additional learning unit 251 executes additional learning processing so that the number of discrimination targets increases by the number of unknown persons. The number of unknown persons is acquired from the number-of-unknown persons determination unit 208. Here, the number of unknown persons used by the additional learning unit 251 is set to N1.

[0100] Specifically, the additional learning unit 251 increases the output of the fully connected layer of the probability calculation unit 402 by N1 dimensions and increases the existing model parameters (weights) w ij The remaining D×N1 model parameters (weights) are learned while keeping (0≦i≦D-1, 0≦j≦M-1) fixed. When the update by the additional learning unit 251 is completed, the number of classification objects that can be classified by the classification model 400 increases to M2 = M + N1.

[0101] The relearning unit 252 executes a relearning process. The relearning process is a process for updating the feature extraction unit 401 and the probability calculation unit 402 of the discrimination model 400 based on feature data generated by the feature extraction unit 401 of the discrimination model 400. The relearning unit 252 updates the feature extraction unit 401 and the probability calculation unit 402 using feature data related to the discrimination target and feature data related to unknown persons. The relearning unit 252 executes the relearning process so that the discrimination targets are increased by the number of unknown persons. Here, the number of unknown persons used by the relearning unit 252 is set to N2.

[0102] Specifically, the re-learning unit 252 first performs the same process as the additional learning unit 251. That is, the re-learning unit 252 increases the output of the fully connected layer of the probability calculation unit 402 by N2 dimensions and recalculates the existing model parameters (weights) w ijThe remaining D×N2 model parameters (weights) are trained while keeping (0≦i≦D-1, 0≦j≦M-1) fixed. Next, the re-learning unit 252 trains both the feature extraction unit 401 and the probability calculation unit 402 of the discrimination model 400 based on the feature data on the discrimination target and the feature data on the unknown person.

[0103] The additional learning process only involves a process of adding an output dimension to the probability calculation unit 402, and therefore involves a significantly smaller amount of calculations than the re-learning process in which the entire discrimination model 400 is trained. On the other hand, the re-learning process re-learns the entire discrimination model 400, and therefore the discrimination accuracy of the updated discrimination model 400 is higher than that of the additional learning process. In this embodiment, whether to perform the additional learning process or the re-learning process is determined depending on the number of unknown persons. This makes it possible to reduce the number of times that the re-learning process, which involves a large amount of calculations, is performed, and to efficiently update the discrimination model 400.

[0104] The deletion unit 253 executes a deletion process. The deletion process is a process of updating the probability calculation unit 402 of the discrimination model 400 in response to an instruction from a user. The instruction from the user includes information (such as discrimination information) specifying the discrimination target to be deleted. Specifically, the deletion unit 253 deletes an output node of a dimension corresponding to the specified discrimination information from among the output dimensions of the fully connected layer of the probability calculation unit 402.

[0105] The data acquisition unit 203 acquires sensor data related to the subject from the work machine 100. The data acquisition unit 203 acquires the sensor data by receiving sensor data transmitted from the work machine 100. The data acquisition unit 203 may acquire the sensor data by reading sensor data output to a portable storage medium by the work machine 100.

[0106] The data processing unit 204 processes the sensor data acquired by the data acquisition unit 203 to generate input data for the identification model 400. The data processing unit 204 generates face image data, skeletal data, and equipment data based on the image data. The data processing unit 204 generates input data that is a collection of image data, face image data, skeletal data, equipment data, and machine operation data, all of which are combined into one tensor data.

[0107] 10 is a diagram showing an example of data processing according to the first embodiment. The data processing is processing in which the data processing unit 204 processes sensor data to generate input data.

[0108] 10, in the data processing, input data D3 is generated by processing image data D1 and machine operation data D2. The image data D1 and machine operation data D2 are included in the sensor data. The input data D3 is data to be input to the identification model 400.

[0109] The image data D1 is processed into face image data D12, skeletal data D13, and accessory data D14. The raw image data D11 is image data that has not been processed from the image data D1. The face image data D12 is image data that shows a face detected from the image data D1. If no face is detected from the image data D1, the face image data D12 is a white image or the like.

[0110] The skeleton data D13 is data indicating a skeleton detected from the image data D1. For example, the skeleton may be detected by key point detection. The skeleton data D13 is an image in which the positions of the detected key points are blurred using a Gaussian filter. The skeleton data D13 includes a blurred image for each key point.

[0111] The equipment data D14 is image data indicating equipment detected from the image data D1. When multiple pieces of equipment are detected, the equipment data D14 includes an image of each piece of equipment detected. Examples of equipment include hats, name tags, sunglasses, etc. The equipment data D14 does not have to be generated.

[0112] The machine operation data D2 is processed into two-dimensional data D21 with the same number of elements as the image data D1. As an example, the two-dimensional data D21 is data obtained by normalizing each element of the machine operation data D2 to a value between 0 and 255 and adjusting it to the spatial size of the image data D1. As an example, the size may be adjusted by increasing or decreasing the number of elements by averaging adjacent elements or by interpolation.

[0113] The input data D3 is tensor data that integrates raw image data D11, face image data D12, skeletal data D13, equipment data D14, and two-dimensional data D21. The number of channels of the input data D3 varies depending on the number of key points and equipment detected from the image data D1. For example, if there are 0 equipment and 17 key points, the input data D3 will have 20 channels.

[0114] The identification unit 205 identifies which of the identification targets the subject is based on the identification model 400 read from the model storage unit 201. The identification unit 205 inputs the input data processed by the data processing unit 204 to the identification model 400 to identify the subject.

[0115] The identification unit 205 identifies a subject based on, for example, the probability value for each classification target output from the probability calculation unit 402 of the classification model 400. The identification unit 205 outputs, for example, the identification information (ID) of the classification target showing the maximum probability value among the probability values ​​for each classification target as the classification result. For example, when M=4, it is assumed that the probability values ​​of ID=1 to ID=4 are 0.7, 0.2, 0.1, and 0.0, respectively. In this case, the identification unit 205 outputs ID=1, which has the highest probability value, as the classification result.

[0116] The score calculation unit 206 calculates a score S indicating whether the subject is a subject to be identified or an unknown person, based on the output from the probability calculation unit 402 of the identification model 400. The score S may be a numerical value indicating the degree to which the subject is either a subject to be identified or an unknown person.

[0117] As an example, the score calculation unit 206 calculates the output value p i The maximum value of (i=1,2,...,M) max(p i ) is taken as the score S. As another example, the score calculation unit 206 may convert the EnergyScore into a negative value and take the result as the score S. The EnergyScore is calculated by, for example, multiplying the input value to the softmax function by q i (i = 1, 2, . . . , M), and the calculation can be done using equation (1).

[0118]

number

[0119] The score calculation unit 206 further determines whether the subject is one of the identification targets or an unknown person based on the score S. As an example, the score calculation unit 206 determines whether the subject is one of the identification targets or an unknown person by comparing the score S with a predetermined threshold λ. If the score S is equal to or greater than the threshold λ, the score calculation unit 206 determines that the subject is one of the identification targets. If the score S is less than the threshold λ, the score calculation unit 206 determines that the subject is an unknown person. The threshold λ may be determined appropriately so as not to deteriorate the detection rate of the identification target.

[0120] The clustering unit 207 clusters the feature data related to a subject determined to be an unknown person by the score calculation unit 206. Every time a subject is determined to be an unknown person by the score calculation unit 206, the clustering unit 207 adds the feature data related to the subject to be clustered, and executes clustering.

[0121] Fig. 11 is a diagram showing an example of a clustering result according to the first embodiment. In the clustering result shown in Fig. 11, the solid line plot indicates existing feature data, and the dotted line plot indicates newly added feature data. Note that in Fig. 11, the clustering result is visualized in two dimensions to improve visibility.

[0122] As shown in FIG. 11, it is assumed that existing feature data are clustered into k-1 groups (clusters) (k=3 in FIG. 11). In this case, it is assumed that feature data f for which the distance d between new feature data fj and the center of gravity of group G is the smallest belongs to group Gp. The distance d may be the distance to an element fi (∈G) of group G. If the distance d is less than a predetermined threshold dλ, it is determined that the feature data fj belongs to group Gp. On the other hand, if the distance d is equal to or greater than the predetermined threshold dλ, a new group Gk is created, and it is determined that the feature data fj belongs to group Gk.

[0123] The clustering unit 207 may compress the dimensions of the feature data using principal component analysis (PCA) etc. The clustering unit 207 may standardize (normalize) each element of the feature data to a range of 0 to 1 etc.

[0124] The number of unknowns determination unit 208 determines the number of unknown persons based on the clustering result by the clustering unit 207. Specifically, the number of unknowns determination unit 208 determines the number of groups included in the clustering result as the number of unknown persons. The number of unknowns determination unit 208 may determine the number of groups that include a predetermined number or more of feature data among the groups included in the clustering result as the number of unknown persons.

[0125] The number of unknown persons determination unit 208 further determines the update process to be executed based on the number of unknown persons. As an example, the number of unknown persons determination unit 208 determines the update process to be executed by comparing the number of unknown persons with a predetermined threshold. The predetermined threshold includes a first threshold N1 and a second threshold N2 that is greater than the first threshold N1.

[0126] Specifically, the number of unknowns determination unit 208 determines that updating is unnecessary if the number of unknown persons is less than the threshold N1. The number of unknowns determination unit 208 determines that additional learning should be performed if the number of unknown persons is equal to or greater than the threshold N1 and less than the threshold N2. The number of unknowns determination unit 208 determines that re-learning should be performed if the number of unknown persons is equal to or greater than the threshold N2.

[0127] The screen providing unit 209 transmits screen data for displaying the management screen to the terminal device 300. The screen data is, for example, screen data described in HTML (HyperText Markup Language) or the like, and may include an application described in JavaScript (registered trademark) or the like.

[0128] The screen providing unit 209 embeds information about the identification model 400 read from the model storage unit 201 into the screen data. The information about the identification model 400 includes information indicating the current identification model mode, information indicating an update process to be executed, information about the number of identification targets, and information about the number of unknown persons.

[0129] The mode of the discriminative model is information indicating whether the current discriminative model is an incremental learning model or a re-learning model. The incremental learning model means that the most recently executed update process was incremental learning. The re-learning model means that the most recently executed update process was re-learning.

[0130] An individual identification database for managing information used for individual identification is constructed in the data storage unit 210. The individual identification database stores the sensor data acquired by the data acquisition unit 203, the identification result by the identification unit 205, the feature data output from the feature extraction unit 401, and the clustering result by the clustering unit 207.

[0131] Fig. 12 is a diagram showing an example of the personal identification database according to the first embodiment. As shown in Fig. 12, the personal identification database has, as data items, a data ID, an identification ID, a date and time, image data, machine operation data, feature data, a clustering result, and an additional learning flag.

[0132] The data ID is identification information for identifying the sensor data. The identification ID is identification information identified by the identification unit 205. The date and time is the date and time when the sensor data was generated. The image data is information indicating the image data included in the sensor data. The machine operation data is information indicating the machine operation data included in the sensor data.

[0133] The feature data is information indicating the feature data generated by the feature extraction unit 401. The feature data is updated when the feature extraction unit 401 performs additional learning or re-learning. The clustering result is information indicating the clustering result by the clustering unit 207. The additional learning flag is a truth value (e.g., true or false, or 1 or 0) indicating that the data is the subject of additional learning. A provisional identification ID according to the clustering result is assigned to the identification ID of the data that is the subject of additional learning.

[0134] Terminal Device As shown in FIG. 6, the terminal device 300 includes a communication control unit 301 and a screen display unit 302.

[0135] The communication control unit 301 and the screen display unit 302 are realized, for example, by processing that is executed by the CPU 501 according to a program loaded onto the RAM 503 from the HDD 504 shown in FIG.

[0136] The communication control unit 301 controls communication with the identification device 200. The communication control unit 301 receives screen data for displaying a management screen from the identification device 200. The communication control unit 301 transmits a control signal to the identification device 200 to instruct the identification device 200 to update the identification model.

[0137] The screen display unit 302 performs control to display a management screen on the display device 506 based on the screen data received from the identification device 200. The screen display unit 302 accepts operations on the management screen.

[0138] <Processing Procedure> The identification method executed by the identification system 1000 according to the first embodiment will be described with reference to Fig. 13. Fig. 13 is a sequence diagram showing an example of the identification method according to the first embodiment.

[0139] In step S101, the data generation unit 101 of the work machine 100 generates sensor data related to the subject. Specifically, the data generation unit 101 generates image data D1 obtained by capturing an image of the subject using the imaging device S11. The data generation unit 101 also generates machine operation data D2 observed by the sensors S1 to S5. The data generation unit 101 sends the sensor data including the image data D1 and the machine operation data D2 to the data output unit 102.

[0140] In step S102, the data output unit 102 of the work machine 100 receives the sensor data from the data generation unit 101. Next, the data output unit 102 transmits the sensor data to the identification device 200.

[0141] In step S103, the data acquisition unit 203 of the identification device 200 receives sensor data from the work machine 100. Next, the data acquisition unit 203 stores the received sensor data in the data storage unit 210. Specifically, the data acquisition unit 203 stores image data D1 included in the sensor data in the image data of the personal identification database. In addition, the data acquisition unit 203 stores machine operation data D2 included in the sensor data in the machine operation data of the personal identification database. At this time, values ​​are automatically stored as the data ID and date and time of the personal identification database.

[0142] In step S104, the data processing unit 204 of the identification device 200 reads out sensor data related to the subject from the data storage unit 210. Next, the data processing unit 204 processes the read sensor data to generate input data related to the subject. Then, the data processing unit 204 sends the generated input data to the identification unit 205.

[0143] Specifically, the data processing unit 204 reads out image data D1 and machine operation data D2 in which no value is stored in the identification ID from the personal identification database. Next, the data processing unit 204 detects a face, a skeleton, and equipment from the image data to generate raw image data D11, face image data D12, skeleton data D13, and equipment data D14. The data processing unit 204 also processes the machine operation data D2 into two-dimensional data D21. Then, the data processing unit 204 combines the raw image data D11, face image data D12, skeleton data D13, equipment data D14, and two-dimensional data D21 into one tensor data. As a result, input data related to the subject is generated.

[0144] In step S105, the classification unit 205 of the classification device 200 receives input data related to the subject from the data processing unit 204. Next, the classification unit 205 reads out the trained classification model 400 from the model storage unit 201. Then, the classification unit 205 inputs the received input data to the classification model 400.

[0145] The feature extraction unit 401 of the discrimination model 400 generates feature data indicating the features of a subject based on input data. The feature extraction unit 401 inputs the feature data to the probability calculation unit 402. The probability calculation unit 402 calculates and outputs a probability value for each discrimination target based on the feature data.

[0146] The classification unit 205 obtains a probability value for each classification target output from the classification model 400. Next, the classification unit 205 generates a classification result based on the probability value for each classification target. The classification result includes classification information of the classified classification target. Then, the classification unit 205 sends the output from the feature extraction unit 401 (i.e., feature data) and the output from the probability calculation unit 402 (i.e., the probability value for each classification target) to the score calculation unit 206.

[0147] In step S106, the score calculation unit 206 of the classification device 200 receives the feature data and the probability value for each classification target from the classification unit 205. Next, the score calculation unit 206 calculates a score S based on the probability value for each classification target.

[0148] In step S107, the score calculation unit 206 of the identification device 200 determines whether the subject is identifiable or not based on the score S calculated in step S106. In other words, the score calculation unit 206 determines whether the subject is any of the identification targets or an unknown person based on the score S. If it is determined that the subject is any of the identification targets (YES), the identification unit 205 proceeds to step S108. On the other hand, if it is determined that the subject is an unknown person (NO), the identification unit 205 sends feature data related to the subject to the clustering unit 207, and proceeds to step S109.

[0149] In step S108, the score calculation unit 206 of the identification device 200 stores the feature data related to the subject and the identification result in the data storage unit 210. Specifically, the identification unit 205 stores the feature data related to the subject in the feature data of the personal identification database, and stores the identification result in the identification ID.

[0150] In step S109, the clustering unit 207 of the classification device 200 receives feature data related to the subject from the score calculation unit 206. Next, the clustering unit 207 adds the received feature data to the clustering target and executes clustering. Then, the clustering unit 207 sends the clustering result to the number of unknowns determination unit 208.

[0151] <Details of clustering process> The clustering process executed by the clustering unit 207 will be described in more detail with reference to Fig. 14. Fig. 14 is a flowchart showing an example of the clustering process according to the first embodiment.

[0152] In step S121, the clustering unit 207 performs a predetermined pre-processing on the feature data fj related to the subject. The pre-processing includes, for example, dimensionality reduction or normalization. The pre-processing is not necessarily performed.

[0153] In step S122, the clustering unit 207 calculates the distance between the feature data fj and each of the other feature data.

[0154] In step S123, the clustering unit 207 identifies the feature data f that is closest to the feature data fj, based on the distance calculated in step S122.

[0155] In step S124, the clustering unit 207 identifies the distance d between the feature data f identified in step S123 and the center of gravity of the group G, and identifies the group Gp to which the feature data f belongs.

[0156] In step S125, the clustering unit 207 determines whether the distance d identified in step S124 is equal to or greater than a predetermined threshold dλ. If the distance d is equal to or greater than the threshold dλ (YES), the clustering unit 207 proceeds to step S127. On the other hand, if the distance d is less than the threshold dλ (NO), the clustering unit 207 proceeds to step S126.

[0157] In step S126, the clustering unit 207 adds the feature data fj to the group Gp identified in step S123.

[0158] In step S127, the clustering unit 207 creates a new group Gk.

[0159] In step S128, the clustering unit 207 adds the feature data fj to the group Gk created in step S127.

[0160] In step S129, the clustering unit 207 stores the clustering result in the individual identification database. Specifically, the clustering unit 207 stores information (e.g., p or k) indicating the group to which the feature data fj is added in the clustering result of the data corresponding to the feature data fj.

[0161] Returning to FIG. 13, the description will be given. In step S110, the number of unknowns determination unit 208 of the identification device 200 receives the clustering result from the clustering unit 207. Next, the number of unknowns determination unit 208 determines the number of unknown persons based on the clustering result. Then, the number of unknowns determination unit 208 determines the update process to be executed based on the number of unknown persons. Then, the number of unknowns determination unit 208 stores the number of unknown persons and the determination result in the model storage unit 201.

[0162] <Details of update determination process> The update determination process executed by the number of unknowns determination unit 208 will be described in more detail with reference to Fig. 15. Fig. 15 is a flowchart showing an example of the update determination process according to the first embodiment.

[0163] In step S141, the number of unknowns determination unit 208 acquires the current number of groups K and the number of data Cp of feature data belonging to each group Gp based on the clustering result.

[0164] In step S142, the number-of-unknown persons determination unit 208 initializes a counter C to 0, and initializes an index p to 1. The counter C is a variable for counting the number of unknown persons. The index p is a variable indicating the group number.

[0165] In step S143, the number of unknowns determination unit 208 determines whether the group Gp corresponds to an unknown person. Specifically, the number of unknowns determination unit 208 determines whether the number of data Cp is equal to or greater than a predetermined threshold Cth. If the number of data Cp is equal to or greater than the threshold Cth (YES), the number of unknowns determination unit 208 proceeds to step S144. On the other hand, if the number of data Cp is less than the threshold Cth (NO), the number of unknowns determination unit 208 skips step S144 and proceeds to step S145.

[0166] In step S144, the number of unknowns determination unit 208 adds 1 to the counter C.

[0167] In step S145, the number of unknowns determination unit 208 determines whether or not the index p is equal to the number of groups K. In other words, the number of unknowns determination unit 208 determines whether or not the number of data Cp has been compared with the threshold value Cth for all groups Gp. If the index p is equal to the number of groups K (YES), the number of unknowns determination unit 208 proceeds to step S147. On the other hand, if the index p is not equal to the number of groups K (NO), the number of unknowns determination unit 208 proceeds to step S146.

[0168] In step S146, the number of unknowns determination unit 208 adds 1 to the index p. After that, the number of unknowns determination unit 208 returns the process to step S143, and executes the processes from step S143 to step S145 again.

[0169] In step S147, the number of unknowns determination unit 208 updates the personal identification database for the data belonging to the group Gp determined in step S143 to correspond to an unknown person. Specifically, the number of unknowns determination unit 208 generates a provisional identification ID for each group Gp, and stores the provisional identification ID in the identification ID of the data belonging to that group Gp. In addition, the number of unknowns determination unit 208 updates the additional learning flag of the data storing the provisional identification ID to a value (e.g., true or 1) indicating that the data is to be subject to additional learning.

[0170] In step S148, the unknown number determination unit 208 determines whether the counter C is equal to or greater than the threshold N1. If the counter C is equal to or greater than the threshold N1 (YES), the unknown number determination unit 208 proceeds to step S149. On the other hand, if the counter C is less than the threshold N1 (NO), the unknown number determination unit 208 proceeds to step S152.

[0171] In step S149, the number of unknowns determination unit 208 determines whether the counter C is equal to or greater than the threshold N2. The threshold N2 is set to be greater than the threshold N1. If the counter C is equal to or greater than the threshold N2 (YES), the number of unknowns determination unit 208 proceeds to step S151. On the other hand, if the counter C is less than the threshold N2 (NO), the number of unknowns determination unit 208 proceeds to step S150.

[0172] In step S150, the unit for determining the number of unknowns 208 determines that the update processing to be executed on the discrimination model 400 is additional learning. The unit for determining the number of unknowns 208 stores in the model storage unit 201 information indicating that additional learning should be executed.

[0173] In step S151, the unit for determining the number of unknowns 208 determines that the update process to be executed on the identification model 400 is re-learning. The unit for determining the number of unknowns 208 stores in the model storage unit 201 information indicating that re-learning should be executed.

[0174] In step S152, the number of unknowns determination unit 208 determines that the identification model 400 does not need to be updated. The number of unknowns determination unit 208 stores in the model storage unit 201 information indicating that the model is not needed to be updated.

[0175] When the number of unknowns determination unit 208 determines in step S150 or step S151 that the update process should be executed, it may notify the model update unit 202 of that effect. The model update unit 202 may automatically execute the update process in response to the notification from the number of unknowns determination unit 208. Note that FIG. 13 shows a processing procedure when the update process is executed in response to an update instruction from the terminal device 300.

[0176] Returning to FIG. 13 , in step S111, the screen providing unit 209 of the identification device 200 reads out information about the identification model 400 from the model storage unit 201. Next, the screen providing unit 209 generates screen data for displaying a management screen. Information about the identification model 400 is embedded in the screen data. Then, the screen providing unit 209 transmits the screen data to the terminal device 300.

[0177] In step S112, the communication control unit 301 of the terminal device 300 receives the screen data from the identification device 200. Next, the communication control unit 301 sends the received screen data to the screen display unit 302. The screen display unit 302 displays a management screen on the display device 506 based on the screen data received from the communication control unit 301.

[0178] ≪Management screen≫ Fig. 16 is an image diagram showing an example of a management screen according to the first embodiment. As shown in Fig. 16, a management screen 800 has a type display field 801, a type selection button 802, a mode display field 803, a mode selection button 804, a number of recognition targets display field 805, a number of unknowns display field 806, a period designation button 807, a number of data list 808, a recognition target list 809, a data list 810, an update button 811, a cancel button 812, a data input button 813, a recognition execution button 814, a clustering button 815, an additional learning button 816, a re-learning button 817, a delete button 818, and a setting change button 819.

[0179] The type display field 801 displays the type of the current classification model 400. The type display field 801 may display a wording indicating the structure of the classification model 400, such as "ResNet50" or "VGG16", or may display a name that is uniquely set by the user.

[0180] The type selection button 802 is a button for selecting the type of the identification model 400. When the user presses the type selection button 802, a list of types of the identification model 400 may be displayed in a select box format or the like so that the user can select the type.

[0181] The mode display field 803 displays the current mode of the discriminative model 400. The mode display field 803 may display a wording indicating the mode of the discriminative model 400, such as "additional learning mode" or "relearning mode".

[0182] The mode selection button 804 is a button for selecting a mode of the discrimination model 400. When the user presses the mode selection button 804, a list of modes of the discrimination model 400 may be displayed in a select box format or the like so that the modes can be selected.

[0183] A number of classification targets display field 805 displays the number of classification targets that can be classified by the classification model 400. A number of unknown persons display field 806 displays the number of unknown persons that cannot be classified by the classification model 400.

[0184] The period designation button 807 is a button for designating a period to be identified. When the user presses the period designation button 807, a screen for inputting the start and end dates of the period may be displayed. When the user designates a period, the display contents of the management screen 800 may be updated based on the data accumulated during the designated period.

[0185] The data count list 808 displays the number of data items stored in the personal identification database for each identification target that can be identified by the identification model 400. When a period is specified using the period specification button 807, the data count list 808 displays the number of data items stored in the specified period.

[0186] The classification target list 809 displays a list of classification targets that can be classified by the classification model 400. The classification target list 809 displays information indicating whether each classification target has been added by additional learning or re-learning. The classification target list 809 displays added classification targets in different colors.

[0187] A data list 810 displays a list of data stored in the personal identification database. The data list 810 displays a selection field, data ID, identification ID, date and time, image data, machine operation data, feature data, and group.

[0188] The data list 810 displays each data in a different color according to an arbitrary criterion. For example, data related to an unknown person is displayed in a different color from data related to the identification target. Furthermore, data manually added by operating the data input button 813 is displayed in an even different color.

[0189] The data list 810 displays information about all data included in the time series data. The data list 810 may display information about the beginning of the time series data (i.e., the earliest data in the time series data).

[0190] The selection state of the selection field can be changed by a user operation. By selecting data in the selection field, the user can select data to be subject to update processing (i.e., additional learning, re-learning, or deletion). The selection field for data related to an unknown person may be automatically selected.

[0191] The identification ID indicates the result of the identification by the identification unit 205. A provisional identification ID is set as the identification ID of data related to an unknown person. The provisional identification ID is updated when re-learning is performed. The identification ID may be changeable by a user operation. The identification information inputted in the identification ID may be used as a correct answer label when additional learning or re-learning is performed.

[0192] The update button 811 is a button for updating the personal identification database with the data displayed in the data list 810. When the user presses the update button 811, a control signal instructing an update may be transmitted from the terminal device 300 to the identification device 200. The control signal instructing an update includes the data edited in the data list 810.

[0193] The cancel button 812 is a button for stopping the update of the personal identification database. The cancel button 812 may be controlled so that it can be pressed only from the start of the update of the personal identification database until it is completed. When the user presses the cancel button 812, the personal identification database is rolled back to the state before the update started. The rollback may be performed in multiple stages. In other words, if multiple updates have been performed, it may be possible to roll back any number of times.

[0194] The data input button 813 is a button for manually storing data about the subject in the personal identification database. When the user presses the data input button 813, a screen for selecting sensor data is displayed, and the sensor data selected by the user may be stored in the personal identification database.

[0195] The identification execution button 814 is a button for identifying data that has been newly stored in the personal identification database by operating the data input button 813. When the user presses the identification execution button 814, the subject is identified by the data processing unit 204 and the identification unit 205. If the score S is equal to or greater than the threshold, the identification ID in the personal identification database is updated and displayed in the identification ID in the data list 810.

[0196] The clustering button 815 is a button for executing clustering processing. The clustering target is all data whose score S is less than a threshold value.

[0197] The additional learning button 816 is a button for instructing the identification device 200 to execute additional learning processing. When the user presses the additional learning button 816, a control signal instructing the identification device 200 to perform additional learning may be transmitted from the terminal device 300 to the identification device 200. The additional learning button 816 may be controlled so as to be pressable only when the number of unknown persons is equal to or greater than a threshold value N1.

[0198] The relearning button 817 is a button for instructing the identification device 200 to execute a relearning process. When the user presses the relearning button 817, a control signal instructing the identification device 200 to relearn may be transmitted from the terminal device 300 to the identification device 200. The relearning button 817 may be controlled so as to be pressable only when the number of unknown persons is equal to or greater than a threshold value N2.

[0199] The delete button 818 is a button for instructing the identification device 200 to execute a deletion process. When the user presses the delete button 818, a control signal instructing the identification device 200 to delete may be transmitted from the terminal device 300 to the identification device 200. The control signal instructing the deletion may include the identification ID of the data selected in the selection field.

[0200] The setting change button 819 is a button for changing various parameters used by the identification device 200. The parameters used by the identification device 200 include various thresholds. When the user presses the setting change button 819, a screen for editing the parameters of the identification device 200 is displayed.

[0201] The management screen 800 updates the display contents at a predetermined time interval during startup. When the unknown number determination unit 208 determines that update processing should be performed, the management screen 800 pops up a notification screen. The notification screen displays information indicating the update processing to be performed and information regarding the unknown number of people.

[0202] ≪Notification screen≫ Fig. 17 is an image diagram showing an example of a notification screen according to the first embodiment. As shown in Fig. 17, a notification screen 850 has an unknown number of people display area 851, an update process display area 852, an execute button 853, and a cancel button 854.

[0203] Information related to the unknown number of people is displayed in the unknown number of people display area 851. The unknown number of people display area 851 displays a notice including the unknown number of people (for example, "The unknown number of people has become 3") as information related to the unknown number of people.

[0204] The update process display area 852 displays information indicating the update process to be executed. The information displayed in the update process display area 852 is displayed based on information indicating the update process to be executed that is stored in the model storage unit 201. The information indicating the update process to be executed is stored in the model storage unit 201 when the number of unknowns determination unit 208 executes an update determination process. The update process display area 852 displays a notification that an update process should be executed, as information indicating the update process to be executed. The notification that an update process should be executed is a notification that additional learning should be executed (for example, "Additional learning is recommended"), or a notification that re-learning should be executed (for example, "Re-learning is recommended").

[0205] The execute button 853 is a button for executing the update process to be executed. When the user presses the execute button 853, a control signal instructing an update is transmitted from the terminal device 300 to the identification device 200. The update process is the update process displayed in the update process display area 852.

[0206] The cancel button 854 is a button for closing the notification screen 850. When the user presses the cancel button 854, the notification screen 850 closes and the screen returns to the management screen 800 without executing the update process that should be executed.

[0207] 13. In step S113, the communication control unit 301 of the terminal device 300 transmits to the identification device 200 a control signal instructing the updating of the identification model 400 in response to an operation on the management screen 800 or the notification screen 850. The control signal includes information indicating the update process to be executed.

[0208] For example, when an additional learning button 816 on the management screen 800 is pressed, the communication control unit 301 transmits a control signal to the identification device 200 to instruct additional learning. When a re-learning button 817 on the management screen 800 is pressed, the communication control unit 301 transmits a control signal to the identification device 200 to instruct re-learning. When a delete button 818 on the management screen 800 is pressed, the communication control unit 301 transmits a control signal to the identification device 200 to instruct deletion.

[0209] In step S114, the model update unit 202 of the identification device 200 receives a control signal from the terminal device 300. Next, the model update unit 202 executes an update process indicated by the control signal. Specifically, if the control signal is a signal instructing additional learning, the model update unit 202 executes the additional learning unit 251. If the control signal is a signal instructing re-learning, the model update unit 202 executes the re-learning unit 252. If the control signal is a signal instructing deletion, the model update unit 202 executes the deletion unit 253.

[0210] The additional learning unit 251 acquires the number N1 of unknown persons from the number-of-unknown persons determination unit 208. Next, the additional learning unit 251 reads out feature data related to the unknown persons from the data storage unit 210. Then, the additional learning unit 251 increases the output of the probability calculation unit 402 of the discriminative model 400 by N1 dimensions. In addition, the additional learning unit 251 updates the probability calculation unit 402 using the read out feature data. Then, the additional learning unit 251 stores in the model storage unit 201 information indicating that the current discriminative model 400 is an additional learning model.

[0211] The relearning unit 252 obtains the number of unknown persons N2 from the number-of-unknown persons determination unit 208. Next, the relearning unit 252 reads out feature data related to the classification target and feature data related to unknown persons from the data storage unit 210. Then, the relearning unit 252 increases the output of the probability calculation unit 402 of the classification model 400 by N2 dimensions. In addition, the relearning unit 252 updates the feature extraction unit 401 and the probability calculation unit 402 using the read out feature data. Then, the relearning unit 252 stores information indicating that the current classification model 400 is a re-learned model in the model storage unit 201.

[0212] The deletion unit 253 acquires the identification information of the identification target to be deleted from the control signal. Next, the deletion unit 253 deletes the dimension corresponding to the acquired identification information from the output of the probability calculation unit 402 of the identification model 400.

[0213] <Advantages of the First Embodiment> The identification system 1000 in this embodiment inputs data on a subject into a trained identification model 400 that has been trained to be able to identify a predetermined number of identification targets, thereby identifying which of the identification targets the subject is, and displays a notification that an update process for the identification model 400 should be performed when the number of unknown persons not identified as any of the identification targets is equal to or greater than a predetermined number. A user of the identification system 1000 can know the timing to update the identification model by referring to the notification that an update process should be performed. In one aspect, according to this embodiment, the trained model used for identifying persons can be efficiently updated.

[0214] The identification system 1000 displays information about the number of unknown persons. By referring to the information about the number of unknown persons, the user of the identification system 1000 can appropriately determine whether or not to update the identification model.

[0215] The identification model 400 includes a feature extraction unit 401 that extracts feature data indicating the features of a subject from data related to the subject, and a probability calculation unit 402 that calculates the probability that the subject is each of the identification subjects based on the feature data. The identification system 1000 updates the probability calculation unit 402 based on data related to an unknown person. The identification system 1000 updates the feature extraction unit 401 and the probability calculation unit 402 based on data related to the identification subject and data related to the unknown person. The identification system 1000 updates the probability calculation unit 402 so that a specified identification subject becomes unidentifiable. According to this embodiment, it is possible to update the identification model by selecting an appropriate update process.

[0216] When the number of unknown persons is equal to or greater than a first threshold, the identification system 1000 displays a notification to the effect that processing should be performed to update the probability calculation unit 402. When the number of unknown persons is equal to or greater than a second threshold that is greater than the first threshold, the identification system 1000 displays a notification to the effect that processing should be performed to update the feature extraction unit 401 and the probability calculation unit 402. According to this embodiment, the user of the identification system 1000 can easily determine the update processing to be performed.

[0217] The management screen 800 displays whether the current identification model 400 is an additionally trained model or a retrained model. By referring to the mode of the identification model 400, the user can manually execute retraining when the identification accuracy of the identification model 400 is low even if the number of unknowns is below the threshold.

[0218] The management screen 800 can update the identification model 400 simply by assigning an identification ID to an unknown person. Since the model parameters related to the identification target are fixed, the effect of updating the identification model 400 on the identification of the identification target can be minimized.

[0219] [Second embodiment] In the first embodiment, a configuration has been described in which the score calculation unit 206 calculates the score S based on the output from the probability calculation unit 402 of the discriminant model 400. In the second embodiment, a configuration will be described in which the score calculation unit 206 calculates the score S based on the output from the feature extraction unit 401 of the discriminant model 400.

[0220] In this embodiment, the score calculation unit 206 calculates the score S based on, for example, the k-nearest neighbor method or the Mahalanobis distance, etc. For example, when the k-nearest neighbor method is used, the distance between the set F of feature data related to the classification target and the feature data fj related to the subject is calculated, and the distance dk between the feature data that is the kth closest is set as the score S.

[0221] [Third embodiment] In the first embodiment, a configuration is described in which a single classification model is used to perform personal identification based on input data in which image data and machine operation data are combined into a single tensor data. In the second embodiment, a classification model is trained for each of a plurality of input data pieces obtained by processing image data and machine operation data, and personal identification is performed based on the classification results of each classification model.

[0222] Fig. 18 is a diagram showing an example of a classification model according to the third embodiment. As shown in Fig. 18, the classification model 400 includes a raw image classification model 400-1, a face image classification model 400-2, a bone structure classification model 400-3, an accessory classification model 400-4, a machine operation classification model 400-5, and an integration unit 403.

[0223] The raw image recognition model 400-1 receives raw image data D11 generated by the data processing unit 204. The raw image recognition model 400-1 calculates and outputs a probability value for each classification target based on the raw image data D11. The output of the raw image recognition model 400-1 is input to the integration unit 403.

[0224] The face image identification model 400-2 receives face image data D12 generated by the data processing unit 204. The face image identification model 400-2 calculates and outputs a probability value for each classification target based on the face image data D12. The output of the face image identification model 400-2 is input to the integration unit 403.

[0225] The skeleton identification model 400-3 receives skeleton data D13 generated by the data processing unit 204. The skeleton identification model 400-3 calculates and outputs a probability value for each classification target based on the skeleton data D13. The output of the skeleton identification model 400-3 is input to the integration unit 403.

[0226] The equipment identification model 400-4 receives the equipment data D14 generated by the data processing unit 204. The equipment identification model 400-4 calculates and outputs a probability value for each identification target based on the equipment data D14. The output of the equipment identification model 400-4 is input to the integration unit 403.

[0227] The machine operation identification model 400-5 receives the machine operation data D2 acquired by the data acquisition unit 203. The two-dimensional data D21 generated by the data processing unit 204 may be input to the machine operation identification model 400-5. The machine operation identification model 400-5 calculates and outputs a probability value for each identification target based on the machine operation data D2 or the two-dimensional data D21. The output of the machine operation identification model 400-5 is input to the integration unit 403.

[0228] The integration unit 403 integrates the outputs of the raw image identification model 400-1, the face image identification model 400-2, the bone structure identification model 400-3, the accessory identification model 400-4, and the machine operation identification model 400-5. As an example, the integration unit 403 calculates a probability value for each identification target by averaging the outputs of the raw image identification model 400-1, the face image identification model 400-2, the bone structure identification model 400-3, the accessory identification model 400-4, and the machine operation identification model 400-5.

[0229] [Other embodiments] In each of the above embodiments, an example has been described in which the terminal device 300 displays the management screen 800, but the device that displays the management screen 800 may be another device. For example, the management screen 800 may be displayed on a display device 40 mounted on the work machine 100. Also, for example, it may be displayed on a display device 506 connected to the identification device 200.

[0230] In each of the above embodiments, an example has been described in which the identification device 200 stores a trained identification model and identifies a subject, but the work machine 100 may store a trained identification model. The work machine 100 may process sensor data input to the controller 30 and input the processed data to the trained identification model 400 to identify the operator in the cabin 10.

[0231] The identification system described in the above embodiment can be used to identify individual operators of work machines, and can be applied to, for example, operator health management, automatic daily report creation (log creation), and automatic engine drive systems (stopping operation of the machine if the person is not eligible).

[0232] In the above embodiment, the identification target is the operator of a work machine, but the identification target may be anything. For example, the present invention can be applied to an object detection system at a construction site, a monitoring system that monitors anomalies in a device or system that has multiple types of anomalies, and the like.

[0233] [supplement] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit) implemented by an electronic circuit, and an ASIC (Application Specific Integrated Circuit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), conventional circuit module, and other devices designed to execute each function described above.

[0234] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims. [Explanation of symbols]

[0235] 100:Work machinery 101: Data generation unit 102: Data output section 200: Identification device 201: Model memory section 202: Model Update Department 203: Data acquisition section 204: Data Processing Department 205: Identification unit 206: Score calculation section 207: Clustering Department 208: Unknown number of people determination section 209: Screen provision department 210: Data storage unit 300: Terminal device 301: Communication control unit 302: Screen display section 1000 Identification System

Claims

1. a data acquisition unit configured to acquire data relating to a subject; An identification unit configured to input the data into a trained model trained to be able to identify a predetermined number of identification targets, thereby identifying whether the subject is one of the identification targets; a number-of-unknowns determination unit configured to determine a number of unknown persons who are the subjects and are not identified by any of the identification subjects; A display unit configured to display a notification that a process for updating the trained model should be executed when the number of unknown persons is equal to or greater than a predetermined threshold; An identification system comprising:

2. The display unit is configured to display information regarding the number of unknown persons. The identification system of claim 1 .

3. The trained model is a feature extraction unit configured to extract feature data indicative of features of the subject from the data; a probability calculation unit configured to calculate a probability that the subject is each of the identification targets based on the feature data; 2. The identification system of claim 1, comprising:

4. a first update unit configured to update the probability calculation unit based on the data regarding the unknown person; The identification system of claim 3.

5. a second update unit configured to update the feature extraction unit and the probability calculation unit based on the data on the target and the data on the unknown person; The identification system of claim 3.

6. Further comprising a third update unit configured to update the probability calculation unit so that the designated classification target becomes unclassifiable. The identification system of claim 3.

7. The display unit is configured to display a notification that a process for updating the probability calculation unit should be executed when the number of unknown persons is equal to or greater than a first threshold value. An identification system according to any one of claims 4 to 6.

8. A clustering unit configured to cluster feature data indicating features of the unknown person, The number-of-unknown persons determination unit is configured to determine the number of unknown persons based on a clustering result. An identification system according to any one of claims 1 to 6.

9. a data acquisition unit configured to acquire data relating to a subject; An identification unit configured to input the data into a trained model trained to be able to identify a predetermined number of identification targets, thereby identifying whether the subject is one of the identification targets; a number-of-unknowns determination unit configured to determine a number of unknown persons who are the subjects and are not identified by any of the identification subjects; An output unit configured to output a notification that a process for updating the trained model should be executed when the number of unknown persons is equal to or greater than a predetermined threshold; An identification device comprising:

10. The identification device The procedure for obtaining data about the subjects; A step of inputting the data into a trained model trained to be able to distinguish a predetermined number of objects to be identified, thereby identifying which of the objects the subject is; determining the number of unknown persons who are the subjects and who are not identified by any of the identification subjects; When the number of unknown persons is equal to or greater than a predetermined threshold, displaying a notification that a process for updating the trained model should be executed; Identification method to perform.

Images