Individual identification system, model learning system, individual identification method, model learning method, and program

Abstract

To provide an individual identification system, a model learning system, an individual identification method, a model learning method, and a program which can easily perform individual identification.SOLUTION: An individual identification system includes: first acquisition means 41 for acquiring an image of an individual, which is captured with an imaging device; detection means 43 for detecting at least one region of the individual on the basis of the acquired image; and identification means 45 for identifying the individual on the basis of information on at least each one detected region of the individual and a learned model based on machine learning using the information on at least each one region of the individual in the image of the individual as learning data.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an individual identification system, a model learning system, an individual identification method, a model learning method, and a program. [Background technology]

[0002] In recent years, research has been conducted into techniques for analyzing (estimating) the behavior of workers (subjects), such as analyzing the movement lines of workers at manufacturing sites and visualizing their proficiency. Here, in order to analyze the behavior of each of a plurality of subjects, it is necessary to perform individual identification to identify each of the plurality of subjects. Thus, an individual identification system for identifying each of a plurality of subjects is known (for example, Patent Document 1).

[0003] In the technology described in Patent Document 1, the target is identified by having the target wear clothing on which identification information indicating the target is formed, and then detecting the identification information based on an image obtained by photographing the target with an infrared camera. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2006-28658 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the technology described in Patent Document 1, in order to individually identify each of multiple subjects, it is necessary to manufacture special clothing with identification information formed thereon for each of the multiple subjects, which may increase the costs of individual identification.

[0006] The present invention has been made in consideration of the above problems, and aims to provide an individual identification system, a model learning system, an individual identification method, a model learning method, and a program that enable easy individual identification. [Means for solving the problem]

[0007] In order to solve the above problems, firstly, the present invention provides an individual identification system comprising: a first acquisition means for acquiring an image of an individual captured by an imaging device; a detection means for detecting at least one part of the individual based on the acquired image; and an identification means for identifying the individual based on information about each of the detected at least one part of the individual and a trained model based on machine learning using information about each of the at least one part of the individual in the image of the individual as training data (Invention 1).

[0008] According to this invention (Invention 1), an individual is identified based on information about at least one part of the individual detected based on an image of the individual captured by an imaging device, and a trained model, so that the individual can be identified by using the image of the individual captured by an imaging device. This makes it possible to identify the individual without using, for example, clothing or the like on which the individual's identification information is formed, making it easy to identify the individual.

[0009] In the above invention (Invention 1), a second acquisition means is provided for acquiring information regarding the condition of the individual measured by a measuring device, and the identification means may identify the individual based on information regarding each of at least one part of the detected individual, information regarding the condition of the individual, and a trained model based on machine learning using information regarding each of at least one part of the individual in the image of the individual and information regarding the condition of the individual as training data (Invention 2).

[0010] According to this invention (Invention 2), an individual is identified based on information about at least one part of the individual detected based on an image of the individual captured by an imaging device, information about the individual's condition, and a trained model, so that the individual can be identified by using the image of the individual captured by the imaging device and information about the individual's condition measured by a measuring device. This makes it possible to identify an individual without using, for example, clothing or the like on which the individual's identification information is formed, so that individual identification can be performed easily and with high accuracy.

[0011] In the above invention (Invention 2), the identification means may identify the individual based on the transition of information regarding each of at least one part of the detected individual within a predetermined period of time, the transition of information regarding the condition of the individual within the predetermined period of time, and the trained model (Invention 3).

[0012] According to this invention (Invention 3), an individual can be easily identified based on the transition (change) within a specified period of time of each of the information relating to at least one part of the individual and the information relating to the condition of the individual.

[0013] In the above invention (Invention 1), the identification means may identify the individual based on the coordinates in the acquired image of each of the detected at least one part of the individual and a trained model based on machine learning using the coordinates of each of the at least one part of the individual in the image of the individual as training data (Invention 4).

[0014] According to this invention (Invention 4), the individual can be easily identified by using the coordinates of at least one part of the individual in the image.

[0015] In the above invention (Invention 4), a conversion means is provided for converting coordinates in the acquired image of each of at least one part of the detected individual based on a depth of the individual in the acquired image, and the identification means may identify the individual based on the coordinates of each of the converted at least one part and the trained model (Invention 5).

[0016] According to this invention (Invention 5), it becomes possible to identify an individual using the coordinates of at least one part of the individual that have been converted according to the depth of the individual in the captured image (i.e., the distance between the imaging device and the individual (shooting distance)). This makes it possible to improve the accuracy of individual identification compared to, for example, a case in which the coordinates are not converted according to the depth of the individual.

[0017] In the above invention (Invention 5), the conversion means may convert coordinates within the acquired image of each of at least one detected part of the individual based on the depth of the individual estimated using MiDaS (Invention 6).

[0018] According to this invention (Invention 6), by using MiDaS, the depth of an individual in a captured image can be easily estimated.

[0019] In the above invention (Invention 4), a conversion means is provided for converting coordinates in the acquired image of each of at least one detected part of the individual based on a distance between at least two parts of the individual measured in advance, and the identification means may identify the individual based on the converted coordinates of each of the at least one part and the trained model (Invention 7).

[0020] According to this invention (Invention 7), it becomes possible to identify an individual using the coordinates of at least one part of the individual that have been converted according to a distance between at least two parts of the individual that has been measured in advance (which may be, for example, an actual measurement of the distance between at least two parts of the individual, or a measurement of the distance between at least two parts of the individual in an image captured in advance), thereby improving the accuracy of individual identification compared to, for example, a case in which the coordinates are not converted according to the distance between at least two parts.

[0021] In the above invention (Invention 7), the conversion means may convert the coordinates in the acquired image of each of at least one detected part of the individual based on a distance between a neck part and a waist part of the individual measured in advance (Invention 8).

[0022] According to this invention (Invention 8), by using the distance between the neck and waist of the individual measured in advance (for example, this may be an actual measurement of the distance between the neck and waist of the individual, or a measurement of the distance between the neck and waist of the individual in an image captured in advance), the coordinates of each of at least one part of the individual in the captured image can be easily converted.

[0023] In the above invention (Invention 1), the detection means may detect at least one joint of the individual as the at least one part of the individual (Invention 9).

[0024] According to this invention (Invention 9), the individual can be identified using information about at least one each of the individual's joints in the image.

[0025] In the above invention (Invention 2), the information on the state of the individual may include at least one of the position, number of rotations, angular velocity, angular acceleration, attitude, acceleration, speed, and temperature of the individual (Invention 10).

[0026] According to this invention (Invention 10), an individual can be identified using at least one of the position, number of rotations, angular velocity, angular acceleration, attitude, acceleration, speed, and temperature of the individual.

[0027] In the above invention (Invention 2), the timing at which the state of the individual is measured may be the same as the timing at which an image of the individual is captured (Invention 11).

[0028] According to this invention (Invention 11), it is possible to identify individuals using images of the individuals obtained at the same time and information on the condition of the individuals.

[0029] Secondly, the present invention provides a model learning system including a first information acquisition means for acquiring information relating to at least one part of an individual in an image of the individual, and a learning means for learning a model used to identify the individual by machine learning using the acquired information relating to at least one part as learning data (Invention 12).

[0030] According to this invention (Invention 12), it is possible to learn a model used to identify individuals by machine learning using information about at least one part of an individual in an image as training data, and by using this model, individual identification can be easily performed.

[0031] Thirdly, the present invention provides a method for identifying an individual, in which a computer executes each of the steps of acquiring an image of the individual captured by an imaging device, detecting at least one part of the individual based on the acquired image, and identifying the individual based on information about each of the detected at least one part of the individual and a trained model based on machine learning using information about each of the at least one part of the individual in the image of the individual as training data (Invention 13).

[0032] Fourthly, the present invention provides a model learning method in which a computer executes the steps of acquiring information about at least one respective part of an individual in an image of the individual, and learning a model to be used to identify the individual by machine learning using the acquired information about at least one respective part as learning data (Invention 14).

[0033] Fifth, the present invention provides a program for causing a computer to realize the following functions: acquiring an image of an individual captured by an imaging device; detecting at least one part of the individual based on the acquired image; and identifying the individual based on information on each of the detected at least one part of the individual and a trained model based on machine learning using information on each of the at least one part of the individual in the image of the individual as training data (Invention 15).

[0034] Sixth, the present invention provides a program for enabling a computer to realize a function of acquiring information relating to at least one respective part of an individual in an image of the individual, and a function of learning a model to be used to identify the individual by machine learning using the acquired information relating to at least one respective part as learning data (Invention 16). Effect of the Invention

[0035] According to the individual identification system, model learning system, individual identification method, model learning method, and program of the present invention, individual identification can be easily performed. [Brief description of the drawings]

[0036] [Figure 1] 1 is a diagram illustrating a basic configuration of an individual identification system and a model learning system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing a configuration of an identification device. [Diagram 3]FIG. 2 is a functional block diagram for explaining functions that play a major role in the individual identification system and the model learning system. [Figure 4] FIG. 11 is a diagram showing an example of the configuration of first acquired data. [Diagram 5] FIG. 11 is a diagram showing an example of the configuration of second acquired data. [Figure 6] FIG. 13 is a diagram illustrating an example of the relationship between the distance from a measuring device and the strength of a received signal. [Figure 7] FIG. 13 is a diagram showing an example of a detection result of at least one part of an individual. [Figure 8] FIG. 1A shows an example of transforming the coordinates of at least one part of an individual based on the depth of the individual in an image, and FIG. 1B shows an example of transforming the coordinates of at least one part of an individual. [Figure 9] FIG. 13 is a diagram showing an example of a case where the coordinates of at least one part of an individual are transformed based on the distance between at least two parts of the individual in an image. [Figure 10] FIG. 4 is a diagram illustrating an example of a configuration of learning data. [Figure 11] 1 is a flowchart illustrating an example of main processing of a model learning system according to an embodiment of the present invention. [Figure 12] 4 is a flowchart showing an example of main processing of the individual identification system according to one embodiment of the present invention. [Figure 13] FIG. 13 is a diagram illustrating another example of the configuration of learning data. [Figure 14] FIG. 11 is a diagram showing an example of division of functions of an individual identification system and a model learning system between an identification device and a learning device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0037] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. However, this embodiment is merely an example, and the present invention is not limited thereto.

[0038] (1) Basic configuration of the individual identification system and model learning system Fig. 1 is a diagram showing a schematic basic configuration of an individual identification system and a model learning system according to an embodiment of the present invention. As shown in Fig. 1, in the individual identification system and the model learning system according to this embodiment, when an individual (in this embodiment, a subject T) is present in a predetermined space SP, such as indoors, an image of the subject T captured by an imaging device 10 provided at a predetermined position in the space SP (in the example of Fig. 1, the upper center of the space SP) and information on the state of the subject T measured by a measuring device 20 are acquired by an identification device 30.

[0039] The identification device 30 is configured to identify (individual identification) the subject T based on the captured image of the subject T and information related to the measured condition of the subject T. Here, each of the imaging device 10 and the measuring device 20, and the identification device 30 are connected to a communication network NW (network) such as the Internet or a LAN (Local Area Network).

[0040] The imaging device 10 may be, for example, an imaging device (e.g., a digital camera or a digital video camera) that captures moving images and / or still images, and is provided to capture the space SP at a predetermined position within the space SP. The imaging device 10 may be provided on a predetermined moving object, and capture a predetermined imaging range while moving in accordance with the movement of the moving object. The imaging device 10 is configured to perform imaging processing at a predetermined frame rate (e.g., 30 fps (flames per second) or the like), and transmit the captured image to the identification device 30 via a communication network NW. The imaging device 10 may perform imaging processing when it receives a predetermined imaging instruction signal from the identification device 30.

[0041] Here, the imaging device 10 may be an imaging device (e.g., an omnidirectional camera) that captures an omnidirectional image (e.g., an image of 360° (360° in the horizontal direction in the example of FIG. 1)). In this case, the subject T can be imaged over a wide range.

[0042] Furthermore, the imaging device 10 may be an imaging device that captures an infrared image (for example, an infrared camera, etc.). This makes it possible to detect the subject T in the captured image, even if the subject T is in an environment with poor visibility (for example, at night, in a dark place, in bad weather, etc.).

[0043] Furthermore, the imaging device 10 may be an imaging device that captures a stereo image (for example, a stereo camera, etc.). Here, a stereo image may be, for example, a set of two images having a predetermined parallax.

[0044] In this embodiment, an example is described in which imaging is performed using one imaging device 10, but the space SP may also be imaged using multiple imaging devices (e.g., two imaging devices spaced apart in the horizontal and / or vertical directions).

[0045] The measuring device 20 is a device that measures the condition of the subject T continuously or intermittently (for example, at a predetermined interval (for example, every 300 milliseconds or 1 second, etc.)). Here, the information on the condition of the subject T may be, for example, information on the position of the subject T, a value representing the physical condition of the subject T, a value obtained by substituting a value representing the position or physical condition of the subject T into a predetermined calculation formula, or information representing the position or degree of the physical condition of the subject T. In addition, the information on the position of the subject T may include, for example, position information (for example, at least one of latitude, longitude, and altitude) measured using a position measurement technology such as GPS (Global Positioning System), or may include information on the received signal strength (RSSI) when a signal transmitted by one device is received by the other device. Here, the information on the received signal strength may be, for example, a value of the received signal strength, a value obtained by substituting the value of the received signal strength into a predetermined calculation formula, or information representing the degree of the received signal strength.

[0046] Furthermore, the measuring device 20 may be, for example, a device that measures the position of the subject T (e.g., a GPS sensor, etc.), or a device that measures the physical condition of the subject T (e.g., acceleration in three axial directions (which may be the acceleration of a specified part of the subject T), angular velocity in three axial directions (which may be the angular velocity of a specified part of the subject T), rotation speed, heart rate (pulse), blood pressure, body temperature, sweat amount, number of steps, walking speed, posture, exercise intensity (e.g., heart rate ÷ maximum heart rate), or calories burned, etc.) (e.g., a heart rate monitor, blood pressure monitor, thermometer, sweat meter, three-axis acceleration sensor, three-axis gyro sensor, motion sensor, etc.).

[0047] Furthermore, as shown in FIG. 1, the measuring device 20 may be composed of a first measuring device 21 that can be carried by the subject T or worn on the body of the subject T, and multiple (in the example of FIG. 1, three) second measuring devices 22 each arranged at a different position within the space SP, which perform wireless communication with the first measuring device 21 when the first measuring device 21 is present within the space SP.

[0048] In this case, the first measuring device 21 may be configured to be able to perform wireless communication with the multiple second measuring devices 22 using a predetermined wireless communication method (e.g., wireless LAN (e.g., Wi-Fi (registered trademark))) when present in the space SP. Furthermore, the first measuring device 21 may be configured to transmit a wireless signal (e.g., a probe request, etc.) including its own identification information (e.g., a MAC (Media Access Control) address, etc.) and / or identification information (e.g., the subject's ID or name, etc.) of a user (subject T) who possesses the first measuring device 21 at a predetermined interval (e.g., an interval of several hundred milliseconds, etc.) in order to perform wireless communication with the multiple second measuring devices 22. Here, the identification information of the user (subject T) may be stored in the first measuring device 21 in advance, for example. Furthermore, the first measuring device 21 may be, for example, a device that can be attached to the subject T (e.g., a wearable device) or a portable device that can be carried by the subject T. Furthermore, the first measuring device 21 may be a communication device operated by an individual user, such as a mobile terminal, a smartphone, a PDA (Personal Digital Assistant), a personal computer, a television receiver with two-way communication capabilities (including so-called multi-function smart televisions), etc.

[0049] The second measuring devices 22 may be provided at positions where they can perform wireless communication with the first measuring device 21 using a predetermined wireless communication method (for example, wireless LAN (for example, Wi-Fi (registered trademark))) in the space SP. The second measuring devices 22 may be, for example, devices that relay wireless communication between two or more first measuring devices 21 present in the space SP, devices that relay wireless communication between the first measuring device 21 and another device (not shown) present in the space SP, or devices that relay communication between the first measuring device 21 and another device (for example, the identification device 30, etc.) connected via a communication network NW. The second measuring devices 22 may be packet capture devices.

[0050] Either one of the first measuring device 21 and the multiple second measuring devices 22 may be provided with an RSSI circuit that detects the received signal strength (RSSI) when receiving a signal transmitted by the other one. Furthermore, this signal may include identification information (e.g., MAC address, etc.) of the measuring device (first measuring device 21 or second measuring device 22) that transmitted the signal, and when the received signal strength of this signal is detected by the RSSI circuit, the detected received signal strength and the identification information of the device that transmitted the signal may be stored in a storage device (not shown) provided in the measuring device (first measuring device 21 or second measuring device 22) that received the signal in a state where they are associated with each other.

[0051] Although the case where wireless communication is performed between the first measuring device 21 and the multiple second measuring devices 22 using Wi-Fi (registered trademark) is described here as an example, the communication method is not limited to this case. For example, wireless communication methods such as Bluetooth (registered trademark), ZigBee (registered trademark), UWB, and optical wireless communication (e.g., infrared) may be used, or a wired communication method such as USB may be used.

[0052] The measuring device 20 (the first measuring device 21 and / or the multiple second measuring devices 22) is configured to transmit information regarding the detected condition to the identification device 30 via the communication network NW each time the measuring device 20 detects the condition of the subject T.

[0053] In the present embodiment, a case where one first measuring device 21 is provided on the subject T is described as an example, but multiple first measuring devices 21 may be provided on the subject T. In addition, in the present embodiment, a case where three second measuring devices 22 are provided in the space SP is described as an example, but the number of second measuring devices 22 provided in the space SP may be two or less, or four or more.

[0054] Furthermore, the imaging device 10 and the measuring device 20 may be configured to communicate directly with the identification device 30 using a wired or wireless connection method, or may be configured to communicate with the identification device 30 via a specified relay device (not shown) by sending and receiving information between the imaging device 10 and the identification device 30 using a wired or wireless connection method.

[0055] The identification device 30 is configured to communicate with each of the imaging device 10 and the measuring device 20 via a communication network NW, and to acquire, over time, images captured by the imaging device 10 and information related to the condition of the subject T measured by the measuring device 20 via the communication network NW. The identification device 30 may be a terminal device operated by an individual user, such as a mobile terminal, a smartphone, a PDA, a personal computer, or a television receiver with a two-way communication function (including so-called multi-function smart televisions).

[0056] (2) Configuration of the identification device The configuration of the identification device 30 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the internal configuration of the identification device 30. As shown in Fig. 2, the identification device 30 includes a CPU (Central Processing Unit) 31, a ROM (Read Only Memory) 32, a RAM (Random Access Memory) 33, a storage device 34, a display processing unit 35, a display unit 36, an input unit 37, and a communication interface unit 38, and is provided with a bus 30a for transmitting control signals or data signals between the respective units.

[0057] When power is applied to the identification device 30, the CPU 31 loads various programs stored in the ROM 32 or the storage device 34 into the RAM 33 and executes them. In this embodiment, the CPU 31 reads out and executes the programs stored in the ROM 32 or the storage device 34, thereby realizing functions of a first acquisition means 41, a second acquisition means 42, a detection means 43, a conversion means 44, an identification means 45, a first information acquisition means 46, a second information acquisition means 47, and a learning means 48 (shown in FIG. 3) described below.

[0058] The storage device 34 may be a non-volatile storage device such as a flash memory, an SSD (Solid State Drive), a magnetic storage device (e.g., an HDD (Hard Disk Drive), a floppy disk (registered trademark), a magnetic tape, etc.), an optical disk, or a volatile storage device such as a RAM, and stores the programs executed by the CPU 31 and the data referenced by the CPU 31. The storage device 34 also stores first acquired data (shown in FIG. 4), second acquired data (shown in FIG. 5), and learning data (shown in FIG. 10), which will be described later.

[0059] The display processing unit 35 displays the display data provided from the CPU 31 on the display unit 36. The display unit 36 ​​is, for example, an LCD (Liquid Crystal Display) monitor including thin film transistors arranged in a matrix on a pixel basis, and displays the data to be displayed on the display screen by driving the thin film transistors based on the display data.

[0060] When the identification device 30 is a button input type device, the input unit 37 includes a button group including a plurality of instruction input buttons such as a direction button and a decision button for accepting user operation input, and a button group including a plurality of instruction input buttons such as a numeric keypad, and includes an interface circuit for recognizing the press (operation) input of each button and outputting it to the CPU 31.

[0061] When the identification device 30 is a touch panel input type device, the input unit 37 mainly accepts touch panel input by touching the display screen with a fingertip or a pen. The touch panel input type may be a known type such as a capacitance type.

[0062] Furthermore, when the identification device 30 is a device capable of receiving voice input, the input unit 37 may be configured to include a microphone for voice input, or may be provided with an interface circuit for outputting voice data input via an external microphone to the CPU 31. Furthermore, when the identification device 30 is a device capable of receiving moving images and / or still images, the input unit 37 may be configured to include a digital camera or digital video camera for image input, or may be provided with an interface circuit for receiving image data captured by an external digital camera or digital video camera and outputting it to the CPU 31.

[0063] The communication interface unit 38 includes an interface circuit for communicating with other devices (eg, the imaging device 10 and the measuring device 20) via the communication network NW.

[0064] (3) Overview of each function in the individual identification system and model learning system The functions realized by the individual identification system and model learning system of this embodiment will be described with reference to Fig. 3. Fig. 3 is a functional block diagram for explaining the functions that play a major role in the individual identification system and model learning system of this embodiment. In the functional block diagram of Fig. 3, the first acquisition means 41, the detection means 43 and the identification means 45 correspond to the main components of the individual identification system of the present invention, and the first information acquisition means 46 and the learning means 48 correspond to the main components of the model learning system of the present invention. The other means (the second acquisition means 42, the conversion means 44 and the second information acquisition means 47) are not necessarily essential components, but are components that make the present invention more preferable.

[0065] The first acquisition means 41 has a function of acquiring an image of the subject T (individual) captured by the imaging device 10.

[0066] The function of the first acquisition means 41 is realized, for example, as follows. First, when the subject T is present in the space SP, the imaging device 10 performs imaging processing at a predetermined frame rate (e.g., 30 fps, etc.), and transmits image data of the captured image to the identification device 30 via the communication network NW every time imaging processing is performed. Here, the image data of the image captured by the imaging device 10 may be transmitted to the identification device 30 in a state where it is associated with the imaging date and time and identification information of the imaging device 10 (e.g., the serial number and MAC (Media Access Control) address of the imaging device 10, etc.).

[0067] On the other hand, every time the CPU 31 of the identification device 30 receives (acquires) image data transmitted from the imaging device 10 via the communication interface unit 38, the CPU 31 stores the received image data in the first acquisition data shown in FIG. 4 in association with the imaging date and time of the image. The first acquisition data is data in which the image data of the image is described in association with the imaging date and time of the image. In this way, the first acquisition means 41 can acquire the image of the subject T captured by the imaging device 10.

[0068] The CPU 31 may acquire stereo images of the subject T (individual) captured by the imaging device 10. In this case, a three-dimensional model of the subject T can be generated based on the stereo images of the subject T, and therefore, in the function of the detection means 43 described later, it becomes possible to obtain the position (coordinates) of at least one part of the subject T in a three-dimensional space based on this three-dimensional model. This makes it possible to capture the position of at least one part of the subject T more accurately than, for example, when a two-dimensional model of the subject T is used, and therefore it is possible to improve the accuracy of identifying the subject T based on the image of the subject T.

[0069] Furthermore, for example, in the case where two or more imaging devices are provided spaced apart in the horizontal and / or vertical directions, the CPU 31 may acquire two images captured by each imaging device at substantially the same timing (for example, two images with the same imaging date and time, or two images with an imaging date and time difference within a predetermined range (for example, several milliseconds to several tens of milliseconds)) as a set of images having a predetermined parallax (i.e., stereo images).

[0070] The second acquisition means 42 has a function of acquiring information relating to the condition of the subject T (individual) measured by the measurement device 20.

[0071] Here, the information on the state of the subject T (individual) may include at least one of the position, number of rotations, angular velocity, angular acceleration, posture, acceleration, speed, and temperature of the subject T. The information on the position of the subject T may include position information (for example, at least one of latitude, longitude, and altitude) measured using a position measurement technology such as GPS, or may include information on the received signal strength (RSSI) when a signal transmitted by one of the first measuring device 21 and the multiple second measuring devices 22 is received by the other measuring device. This allows the individual identification of the subject T to be performed using at least one of the position, number of rotations, angular velocity, angular acceleration, posture, acceleration, speed, and temperature of the subject T.

[0072] Furthermore, the timing at which the condition of the subject T (individual) is measured may be the same as the timing at which an image of the subject T is captured. This allows the subject T to be individually identified using the image of the subject T and information on the condition of the subject T obtained at the same time.

[0073] The function of the second acquisition means 42 is realized, for example, as follows. First, when the subject T exists in the space SP, the first measuring device 21 continuously or intermittently measures the state of the subject T (for example, at least one of the position of the subject T, the rotation speed of the subject T, the angular velocity of the subject T in three axial or two axial directions, the angular acceleration of the subject T in three axial or two axial directions, the posture of the subject T, the acceleration of the subject T in three axial or two axial directions, the velocity of the subject T in three axial or two axial directions, and the temperature of the subject T) at a predetermined interval (for example, every 300 milliseconds or 1 second, etc.) and transmits information about the measured state to the identification device 30 via the communication network NW. Here, the information about the state of the subject T measured by the first measuring device 21 may be transmitted to the identification device 30 in a state associated with the measurement date and time and the identification information of the first measuring device 21 (for example, the serial number or MAC address of the first measuring device 21, etc.). Furthermore, the measuring device 20 may perform a measurement process every time the imaging device 10 performs an imaging process (that is, the measuring process by the measuring device 20 and the imaging process by the imaging device 10 may be performed synchronously with each other).

[0074] Here, a case will be described where the information on the condition of the subject T includes information on the position of the subject T, and the information on the position of the subject T includes the received signal strength (RSSI) of a signal transmitted and received in communication between the first measuring device 21 and the multiple second measuring devices 22. For example, when the subject T is present in the space SP, the first measuring device 21 performs wireless communication with each of the multiple second measuring devices 22, and stores the received signal strength (RSSI) value of the wireless signal detected (measured) by the RSSI circuit every time it receives a wireless signal (e.g., a beacon signal, etc.) transmitted at a predetermined interval (e.g., intervals of several hundred milliseconds, etc.) from each of the multiple second measuring devices 22 in a storage device (not shown) provided in the first measuring device 21. Here, the RSSI value may be stored in a state associated with the measurement date and time (e.g., the date and time when the first measuring device 21 received a wireless signal corresponding to the RSSI value from any of the second measuring devices 22) and the identification information of any of the second measuring devices 22 that transmitted the wireless signal corresponding to the RSSI value (e.g., the serial number or MAC address of the second measuring device 22). Then, the first measuring device 21 may transmit information on the state of the subject T measured by the first measuring device 21 within the predetermined period (e.g., every 3 seconds) to the identification device 30 via the communication network NW. Here, the information on the state of the subject T measured by the first measuring device 21 within the predetermined period may be transmitted to the identification device 30 in a state associated with the identification information of the first measuring device 21 (e.g., the serial number or MAC address of the first measuring device 21).

[0075] Here, an example has been described in which the first measuring device 21 transmits the RSSI value when it receives a wireless signal transmitted from each of the multiple second measuring devices 22 to the identification device 30 as information regarding the condition of the subject T, but each of the multiple second measuring devices 22 may also transmit the RSSI value when it receives a wireless signal transmitted from the first measuring device 21 to the identification device 30 as information regarding the condition of the subject T.

[0076] On the other hand, every time the CPU 31 of the identification device 30 receives (acquires) information on the condition of the subject T transmitted from the measurement device 20 via the communication interface unit 38, the CPU 31 stores the received information on the condition in the second acquisition data shown in FIG. 5 in a state in which the information on the condition is associated with the measurement date and time of the information. The second acquisition data is data in which information on the condition of the subject T (in the example of the figure, "measurement information") is described in a state in which the information is associated with each measurement date and time. In this way, the second acquisition means 42 can acquire the condition of the subject T measured by the measurement device 20 (here, at least one of the position of the subject T, the number of rotations of the subject T, the angular velocity of the subject T in three axial or two axial directions, the angular acceleration of the subject T in three axial or two axial directions, the posture of the subject T, the acceleration of the subject T in three axial or two axial directions, the velocity of the subject T in three axial or two axial directions, and the temperature of the subject T).

[0077] In addition, the CPU 31 of the identification device 30 may acquire at least one of the following based on the function of the second acquisition means 42: the position of the subject T, the average and / or variance of the rotation speed of the subject T, the average and / or variance of the angular velocity of the subject T in three axial or two axial directions, the average and / or variance of the angular acceleration of the subject T in three axial or two axial directions, the posture of the subject T, the average and / or variance of the acceleration of the subject T in three axial or two axial directions, the average and / or variance of the velocity of the subject T in three axial or two axial directions, and the average and / or variance of the temperature of the subject T.

[0078] In this embodiment, the received signal strength indicator (RSSI) of signals transmitted and received in communications between the first measuring device 21 and the multiple second measuring devices 22 is included in the information about the position of the subject T, and this received signal strength indicator (RSSI) can be used to determine the distance between the first measuring device 21 and each of the multiple second measuring devices 22, and ultimately the position of the first measuring device 21 in the space SP (i.e., the position of the subject T). To be more specific, the distance between the first measuring device 21 and each of the multiple second measuring devices 22 can be calculated, for example, by using the following formulas (1) and (2). P r =P t +Gr +G t -L...(1)

number

[0079] This makes it possible to obtain the distance between the first measuring device 21 and each of the multiple second measuring devices 22 by using the received signal strength indicator (RSSI) of the signal between the first measuring device 21 and each of the multiple second measuring devices 22. Also, it makes it possible to obtain the position of the first measuring device 21 in the space SP (i.e., the position of the subject T) by using the distance between the first measuring device 21 and each of the multiple second measuring devices 22.

[0080] The detection means 43 has a function of detecting at least one part of the subject T (moving object) based on the acquired image.

[0081] Furthermore, the detection means 43 may detect at least one joint of the subject T (moving body) as at least one part of the subject T. This makes it possible to identify the subject T (individual identification) using information on each of the at least one joint of the subject T.

[0082] The function of the detection means 43 is realized, for example, as follows. The CPU 31 of the identification device 30 may, for example, based on the function of the first acquisition means 41, set a node corresponding to each position of at least one part of the subject T in the image and calculate the coordinates of each node in the image every time image data transmitted from the imaging device 10 is stored in the acquired data. Here, the setting of each node and the calculation of the coordinates may be performed, for example, using a feature point (key point) detection technology (for example, OpenPose, etc.) by deep learning. The coordinates of each node may be two-dimensional coordinates, or may be three-dimensional coordinates when, for example, a three-dimensional model of the subject T is generated by image data (stereo image).

[0083] For example, when the imaging device 10 captures an omnidirectional image, the CPU 31 may expand the omnidirectional image captured by the imaging device 10 into a panoramic image, and set nodes and calculate the coordinates of the nodes for the expanded panoramic image.

[0084] In addition, when stereo images of subject T are acquired, CPU 31 may perform pre-processing (e.g., noise removal, etc.) on each image of a set of images having a predetermined parallax, and may further perform a well-known stereo matching process to generate a three-dimensional model of subject T.

[0085] As shown in FIG. 7, for example, the CPU 31 detects the position (in the example of the figure, coordinates in the X direction and coordinates in the Y direction) of at least one part of the subject T (in the example of the figure, "neck", "right shoulder", "left shoulder", "waist", "right buttock", "left buttock", etc.) in the image captured by the imaging device 10, and sets nodes corresponding to each of the detected parts (in the example of the figure, 14 nodes). Here, the coordinates of each node may be stored in the first acquired data in a state where each node is associated with the image in which it is set. Furthermore, when the CPU 31 cannot detect at least one part of the subject T because the part is captured in such a way that it is hidden by another object (for example, another moving object or installation), the CPU 31 may store the coordinates of the node corresponding to the part as NULL data in the first acquired data.

[0086] Note that, here, an example has been described in which at least one part of subject T (such as the neck, right shoulder, left shoulder, right buttock, and left buttock) is detected using an image captured by imaging device 10, but, for example, at least one joint of subject T (such as the right shoulder joint, right elbow joint, right hip joint, right knee joint, left shoulder joint, left elbow joint, left hip joint, and left knee joint) may be detected using an image captured by imaging device 10.

[0087] The conversion means 44 has a function of converting the coordinates of at least one part of the detected subject T in the acquired image based on the depth of the subject T (individual) in the acquired image. In this case, individual identification can be performed using the coordinates of at least one part of the subject T converted according to the depth of the subject T in the captured image (i.e., the distance (shooting distance) between the imaging device 10 and the subject T), so the accuracy of individual identification can be improved compared to a case where the coordinates are not converted according to the depth of the subject T, for example.

[0088] The conversion means 44 may also convert the coordinates in the captured image of at least one detected part of the subject T based on the depth of the subject T estimated using MiDaS. In this case, the depth of an individual in the captured image can be easily estimated by using MiDaS.

[0089] The function of the conversion means 44 is realized, for example, as follows. When the CPU 31 of the identification device 30 detects at least one part of the subject T (moving object) based on the function of the detection means 43, for example, it estimates the depth of the subject T in the image using MiDaS. Here, MiDaS (https: / / github.com / intel-isl / MiDaS) is a type of depth estimation model for monocular images for estimating the distance between a subject (e.g., subject T) and an imaging device (e.g., imaging device 10) from an image captured by the imaging device, that is, a depth map representing the depth. Note that, here, a case where the depth of the subject T in the image is estimated using MiDaS will be described as an example, but the depth of the subject T in the image may be estimated using other depth estimation models such as DPT (https: / / github.com / intel-isl / DPT), or may be estimated using other depth estimation techniques.

[0090] Also, when the CPU 31 estimates the depth of the subject T in the image, it converts the coordinates of each of the at least one detected part using the estimated depth. Here, with reference to FIG. 8, an example of this conversion method will be described. For example, as shown in the left diagram of FIG. 8(a), when the depth of the subject T in the image is estimated to be d (0 < d < 1), the CPU 31 changes the scale so as to multiply (here, enlarge) the region including the subject T in the left diagram of FIG. 8(a) (the region indicated by the broken line in the figure) by v (for example, 1 / d). Then, the CPU 31 implements the function of the identification means 45 described later using the coordinates of each of the at least one part in the image after the scale change shown in the right diagram of FIG. 8(a). In addition, when the coordinates of any one of the at least one part in the left diagram of FIG. 8(a) are set as (r cosθ, r sinθ) as shown in the left diagram of FIG. 8(b), for example, the coordinates of any one of the parts after the scale change by a factor of v (for example, 1 / d) are represented as (vr cosθ, vr sinθ) as shown in the right diagram of FIG. 8(b).

[0091] Note that the conversion means 44 may be provided with a function of converting the coordinates of each of the at least one part of the detected subject T in the acquired image based on the distance between at least two parts of the subject T (individual) measured in advance. In this case, since it becomes possible to perform individual identification using the coordinates of each of the at least one part of the subject T converted according to the distance between at least two parts of the subject T measured in advance (for example, it may be the measured value of the distance between at least two parts of the subject T, or the measured value of the distance between at least two parts of the subject T in the image captured in advance), for example, compared with the case where the coordinates are not converted according to the distance between at least two parts, the accuracy of individual identification can be improved.

[0092] Furthermore, the conversion means 44 may convert the coordinates of at least one part of the detected subject T in the captured image based on the distance between the neck and waist of the subject T (individual) measured in advance. This makes it possible to easily convert the coordinates of at least one part of the subject T in the captured image by using the distance between the neck and waist of the subject T measured in advance (for example, the actual measured value of the distance between the neck and waist of the subject T, or the measured value of the distance between the neck and waist of the subject T in the image captured in advance).

[0093] The function of the conversion means 44 in this case is realized, for example, as follows. Here, an example will be described in which the distance between the neck and waist of the subject T measured in advance is the measurement value (length L1 in FIG. 9) of the distance between the neck and waist of the subject T in an image captured in advance. For example, when the CPU 31 of the identification device 30 detects at least one part of the subject T (moving body) based on the function of the detection means 43, it obtains the distance in the image between the neck and waist of the at least one detected part (length L2 in FIG. 9). Here, the length between the neck and waist is the most stably detectable among the distances between two or more parts of the subject T, and has a small standard deviation, so it is suitable for use as a reference length when changing the scale. Then, the CPU 31 changes the scale so as to multiply (here, enlarge) the area including the subject T in the lower left diagram of FIG. 9 (the area indicated by the dashed line in the diagram) by v (for example, L1 / L2). Then, the CPU 31 performs the function of the identification means 45, which will be described later, using the coordinates of at least one part in the scale-changed image shown in the lower right diagram of FIG.

[0094] Here, the case where the conversion means 44 converts the coordinates in the acquired image of at least one part of the detected subject T based on the distance between the neck part and the waist part of the subject T measured in advance has been described as an example, but the present invention is not limited to this case. For example, the conversion means 44 may convert the coordinates in the acquired image of at least one part of the detected subject T based on the distance between either one of the neck part and the waist part of the subject T and other parts other than the neck part and the waist part, or based on the distance between two or more parts other than the neck part and the waist part of the subject T.

[0095] The identification means 45 has a function of identifying subject T based on information regarding each of at least one body part of the detected subject T (individual) and a trained model based on machine learning using information regarding each of at least one body part of subject T in an image of subject T as training data.

[0096] In addition, the identification means 45 may identify subject T based on information regarding each of at least one body part of the detected subject T (individual), information regarding the condition of subject T, and a trained model based on machine learning using information regarding each of at least one body part of subject T in the image of subject T and information regarding the condition of subject T as training data.

[0097] Here, the identification means 45 may identify the subject T based on the transition of information on at least one part of the detected subject T (individual) within a predetermined period, the transition of information on the condition of the subject T within the predetermined period, and the trained model. This makes it possible to easily identify the individual subject T based on the transition (change) of information on at least one part of the subject T and information on the condition of the subject T within a predetermined period.

[0098] Furthermore, the identification means 45 may identify the subject T based on the coordinates of at least one part of the detected subject T (individual) in the acquired image and a trained model based on machine learning using the coordinates of at least one part of the subject T in the image of the subject T as training data. This makes it possible to easily identify the subject T as an individual by using the coordinates of at least one part of the subject T in the image.

[0099] Furthermore, the identification means 45 may identify the subject T based on the coordinates of each of at least one part transformed based on the function of the transformation means 44 and the above-mentioned trained model.

[0100] The function of the identification means 45 is realized, for example, as follows. Here, a case will be described as an example in which the identification means 45 identifies the subject T based on the coordinates of at least one body part converted based on the function of the conversion means 44, information on the state of the subject T, and a trained model. When the CPU 31 of the identification device 30 converts the coordinates of at least one body part based on the function of the conversion means 44, the CPU 31 may input the converted coordinates of at least one body part and information on the state of the subject T acquired based on the function of the second acquisition means 42 into a trained model (described later) based on machine learning using the coordinates of at least one body part of the subject T in the image of the subject T and information on the state of the subject T as training data, thereby identifying the subject T (individual identification).

[0101] An example of the learning data is shown in Fig. 10. The learning data shown in Fig. 10 is data in which the coordinates of at least one part of the subject T in the image, information on the condition of the subject T (measured at the time when the image was captured, for example), and identification information (individual label) of the subject T (individual) (in the example of the figure, "Worker A", "Worker B", "Worker C", "Worker D", etc.) are described in a corresponding state. As a result of machine learning using this learning data, a learned model is configured that indicates the relationship between the coordinates of at least one part of the subject T in the image, information on the condition of the subject T, and the identification information of the subject T.

[0102] In addition, when identification information (e.g., the subject's ID or name) of the user (subject T) who possesses the first measuring device 21 or the user (subject T) to which the first measuring device 21 is provided is stored in the first measuring device 21, the identification information of the user (subject T) stored in the first measuring device 21 may be stored in the learning data as identification information (individual label) of the subject T (individual), as described below.

[0103] Therefore, the CPU 31 can identify the subject T (here, identify the subject T as either "Worker A," "Worker B," "Worker C," "Worker D," etc.) by inputting the coordinates of each of at least one body part converted based on the function of the conversion means 44 and information regarding the condition of the subject T acquired based on the function of the second acquisition means 42 into the learned model.

[0104] The CPU 31 may present the identification result of the subject T. For example, when the CPU 31 identifies the subject T based on the function of the identification means 45, the CPU 31 may display information on the identification result of the subject T, for example, on the display unit 36. Here, the information on the identification result of the subject T may be composed of text data or image data, etc. In addition, when the information on the identification result of the subject T is composed of audio data, the CPU 31 may output the information on the identification result of the subject T from an audio output device such as a speaker. Furthermore, the CPU 31 may transmit the information on the identification result of the subject T to another computer (e.g., a server, etc.) connected to the identification device 30 via the communication network NW.

[0105] In addition, the CPU 31 may perform individual identification of the subject T by inputting into the learned model the coordinates of at least one body part of each of the subject T in all images acquired within a specified period (i.e., the change in information relating to at least one body part within the specified period) and all information relating to the condition of the subject T acquired within the specified period (i.e., the change in information relating to the condition of the subject T within the specified period).

[0106] The first information acquisition means 46 has a function of acquiring information about at least one part of the subject T in an image obtained by capturing the subject T (individual).

[0107] The function of the first information acquisition means 46 is realized, for example, as follows. For example, like the functions of the first acquisition means 41 and the detection means 43, the CPU 31 of the identification device 30 sets nodes in the image corresponding to the positions of at least one body part of the subject T and calculates the coordinates of each node in the image every time it receives (acquires) image data transmitted from the imaging device 10 via the communication interface unit 38. Then, the CPU 31 stores the calculated coordinates of each of the at least one body part in the learning data shown in, for example, FIG. In addition, each time the coordinates of at least one body part are newly stored in the learning data, when correct answer data indicating any one of a plurality of identification information (in the example of FIG. 10, "Worker A", "Worker B", "Worker C", and "Worker D") is input using, for example, the input unit 37, the CPU 31 stores the input correct answer data (i.e., any one of "Worker A", "Worker B", "Worker C", and "Worker D") in the "Individual (Individual Label)" item corresponding to the coordinates of the newly stored at least one body part.

[0108] When storing the coordinates of at least one part in the learning data, the CPU 31 may convert the coordinates of at least one part so that the size of the subject T in each image is equal (for example, the depth of the subject T in each image is equal, or the distance between predetermined parts of the subject T in each image is equal), and store the converted coordinates of at least one part in the learning data. As a result, even if the distance between the imaging device 10 and the subject T differs in each image, the coordinates of at least one part are stored in the learning data in a state adjusted so that the distance is equal, so that the use of this learning data makes it possible to configure a model with improved individual identification accuracy.

[0109] The second information acquiring means 47 has a function of acquiring information relating to the measured condition of the subject T (individual).

[0110] The function of the second information acquisition means 47 is realized, for example, as follows. When the CPU 31 of the identification device 30 acquires information on at least one body part of the subject T based on the function of the first information acquisition means 46, the CPU 31 acquires information on the condition of the subject T measured by the measurement device 20 in the same manner as the function of the above-mentioned second acquisition means 42. Then, the CPU 31 stores the acquired information on the condition of the subject T in the learning data in a state in which the acquired information is associated with the coordinates of at least one body part stored in the learning data based on the function of the first information acquisition means 46. Here, the image capture date and time of the image corresponding to the coordinates of at least one body part in the learning data and the corresponding measurement date and time of the information on the condition of the subject T may be the same or may have a time difference within a predetermined range (for example, several milliseconds to several tens of milliseconds).

[0111] In addition, when the identification information (e.g., the ID or name of the subject) of the user (subject T) who possesses the first measuring device 21 or the user (subject T) to whom the first measuring device 21 is provided is stored in the first measuring device 21, the CPU 31 may acquire information on the condition of the subject T (individual) based on the function of the second information acquisition means 47, and may acquire the identification information of the user (subject T) by receiving the identification information of the user (subject T) transmitted from the first measuring device 21. In addition, the CPU 31 may store the acquired identification information of the user (subject T) in the learning data as the identification information (individual label) of the subject T (individual) in a state in which the acquired identification information of the user (subject T) is associated with each coordinate of at least one part stored in the learning data based on the function of the first information acquisition means 46 and the information on the condition of the subject T acquired based on the function of the second information acquisition means 47.

[0112] The learning means 48 has a function of learning a model used to identify the subject T (individual) by machine learning using the acquired information on each of at least one body part as learning data.

[0113] In addition, the learning means 48 may learn a model used to identify the subject T (individual) by machine learning using the acquired information regarding each of at least one body part and the acquired information regarding the condition of the subject T (individual) as learning data.

[0114] The function of the learning means 48 is realized, for example, as follows. When a predetermined model learning instruction is input using the input unit 37, the CPU 31 of the identification device 30 learns a model using the learning data shown in FIG. 10. The CPU 31 may learn using, for example, a time-series compatible neural network model. Here, for example, a recurrent neural network (RNN) or a long short-term memory (LSTM) which is an advanced version of an RNN can be applied as the time-series compatible neural network. The CPU 31 may also learn using any one of a number of models, such as a graph neural network (GNN) model, a convolutional neural network (CNN) model, a support vector machine (SVM) model, a fully connected neural network (FNN) model, a gradient boosting (HGB) model, and a wave net (WN) model. In this embodiment, the accuracy of individual identification can be improved by using any of the graph convolutional neural network (GCN) model, the graph attention network (GAT) model, and the graph convolutional LSTM (GC-LSTM) model, which are derivatives of GNN.

[0115] (4) Flow of main processing of the model learning system of this embodiment Next, an example of the flow of main processes performed by the model learning system of this embodiment will be described with reference to the flowchart of FIG.

[0116] The CPU 31 of the identification device 30 acquires information on at least one body part of the subject T in an image of the subject T (individual) based on the function of the first information acquisition means 46 (step S100). Here, as described above, when the CPU 31 detects the coordinates of at least one body part of the subject T in the image, the CPU 31 stores the coordinates of the detected at least one body part in the learning data. In addition, when correct answer data indicating any one of a plurality of identification information (for example, "worker A", "worker B", "worker C", "worker D") is input using, for example, the input unit 27, each time each coordinate of at least one body part is newly stored in the learning data, the CPU 31 may store the input correct answer data (i.e., any one of "worker A", "worker B", "worker C", "worker D") in the "individual (individual label)" item corresponding to the coordinates of the at least one body part newly stored.

[0117] Next, the CPU 31 of the identification device 30 acquires information on the measured condition of the subject T (individual) (step S102). Here, similar to the function of the second acquisition means 42 described above, when the CPU 31 acquires information on the condition of the subject T measured by the measurement device 20, the acquired information on the condition of the subject T is stored in the learning data in a state in which the information is associated with each coordinate of at least one body part stored in the learning data in step S100. In addition, when the CPU 31 acquires the identification information of the user (subject T) by receiving the identification information of the user (subject T) transmitted from the first measurement device 21, the CPU 31 may store the acquired identification information of the user (subject T) in the learning data as the identification information (individual label) of the subject T (individual) in a state in which the identification information is associated with each coordinate of at least one body part stored in the learning data in step S100 and the acquired information on the condition of the subject T.

[0118] Next, the CPU 31 of the identification device 30 learns a model used to identify the subject T (individual) by machine learning using the acquired information on at least one part and the acquired information on the condition of the subject T (individual) as learning data (step S104). Here, the CPU 31 may learn using, for example, any of a graph convolutional neural network (GCN) model, a graph attention network (GAT) model, and a graph convolutional LSTM (GC-LSTM) model.

[0119] In this way, it becomes possible to learn a model to be used to identify (individual identification) subject T through machine learning using information about at least one part of subject T in the image and information about the condition of subject T as training data.

[0120] (5) Flow of main processing of the individual identification system of this embodiment Next, an example of the flow of main processes performed by the individual identification system of this embodiment will be described with reference to the flowchart of FIG.

[0121] First, the CPU 31 of the identification device 30 acquires an image of the subject T (individual) captured by the imaging device 10 based on the function of the first acquisition means 41 (step S200). Next, the CPU 31 of the identification device 30 acquires information on the state of the subject T (individual) measured by the measurement device 20 based on the function of the second acquisition means 42 (step S202). Furthermore, the CPU 31 of the identification device 30 detects at least one part of the subject T (moving object) using the acquired image based on the function of the detection means 43 (step S204). Here, the CPU 31 may detect at least one joint of the subject T as at least one part of the subject T.

[0122] Next, the CPU 31 of the identification device 30 identifies the subject T based on the function of the identification means 45, using information on at least one part of the detected subject T (individual), information on the state of the subject T, and a trained model based on machine learning using information on at least one part of the subject T in the image of the subject T and information on the state of the subject T as training data (step S206). Here, the CPU 31 of the identification device 30 may identify the subject T based on the coordinates of at least one part of the detected subject T (individual) in the acquired image, information on the state of the subject T, and a trained model based on machine learning using coordinates of at least one part of the subject T in the image of the subject T and information on the state of the subject T as training data. The CPU 31 may also identify the subject T based on the coordinates of at least one part converted based on the function of the conversion means 44 described above and the trained model.

[0123] After step S206, the CPU 31 of the identification device 30 may display information about the identification result on, for example, the display unit 36. The CPU 31 of the identification device 30 may also transmit the information about the identification result to another computer (for example, a server, etc.) via the communication network NW.

[0124] As described above, according to the individual identification system, individual identification method, and program of the present embodiment, the subject T is identified based on information on at least one part of the subject T detected based on an image of the subject T captured by the imaging device 10, information on the condition of the subject T, and a trained model, so that it is possible to identify the subject T by using the image of the subject T captured by the imaging device 10 and information on the condition of the subject T measured by the measuring device 20. This makes it possible to perform individual identification without using, for example, clothing or the like on which identification information of the subject T is formed, so that individual identification can be performed easily and with high accuracy.

[0125] In addition, according to the model learning system, model learning method, and program of this embodiment, it is possible to learn a model used to identify subject T by machine learning using information regarding at least one part of subject T in the image and information regarding the condition of subject T as learning data, and by using this model, individual identification can be easily performed.

[0126] The program of the present invention may be stored in a computer-readable storage medium. The storage medium on which the program is recorded may be the ROM 32, RAM 33, or storage device 34 of the identification device 30 shown in Fig. 2. The storage medium may be a CD-ROM or the like that can be read by being inserted into a program reading device such as a CD-ROM drive. Furthermore, the storage medium may be a magnetic tape, a cassette tape, a flexible disk, an MO / MD / DVD, or a semiconductor memory.

[0127] The above-described embodiments are described for the purpose of facilitating understanding of the present invention, and are not described for the purpose of limiting the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design modifications and equivalents that fall within the technical scope of the present invention.

[0128] For example, in the above embodiment, the case where the identification means 45 identifies the subject T using information on at least one part of the subject T (individual) and information on the condition of the subject T is described as an example, but the present invention is not limited to this case. For example, the identification means 45 may identify the subject T based on information on at least one part of the subject T (individual) and a trained model based on machine learning using information on at least one part of the subject T in the image of the subject T as training data. That is, the identification means 45 may identify the subject T without using information on the condition of the subject T.

[0129] An example of the learning data in this case is shown in Fig. 13. The learning data shown in Fig. 13 is data in which the coordinates of at least one body part of the subject T in the image and the identification information (individual label) of the subject T (individual) (in the example of the figure, "Worker A", "Worker B", "Worker C", "Worker D", etc.) are described in a corresponding state. As a result of machine learning using this learning data, a learned model is constructed that indicates the relationship between the coordinates of at least one body part of the subject T in the image and the identification information of the subject T.

[0130] As described above, when identification information (e.g., the subject's ID or name) of the user (subject T) who possesses the first measuring device 21 or the user (subject T) to which the first measuring device 21 is provided is stored in the first measuring device 21, the identification information of the user (subject T) stored in the first measuring device 21 may be stored in the learning data as identification information (individual label) of the subject T (individual).

[0131] Here, the learning means 48 may learn a model used to identify the subject T (individual) by machine learning using the acquired information on at least one part as learning data. For example, when a predetermined model learning instruction is input using the input unit 37, the CPU 31 of the identification device 30 may learn a model using the learning data shown in FIG.

[0132] Furthermore, the identification means 45 may identify the subject T based on information on at least one body part of the detected subject T (individual) and a trained model based on machine learning using information on at least one body part of the subject T in the image of the subject T as training data. For example, the CPU 31 of the identification device 30 may identify the subject T (individual identification) by inputting the coordinates of at least one body part converted based on the function of the conversion means 44 into a trained model based on machine learning using the training data shown in FIG.

[0133] According to the individual identification system, individual identification method, and program in this case, the subject T is identified based on the learned model and information about at least one part of the subject T detected based on the image of the subject T captured by the imaging device 10, so it becomes possible to identify the subject T by using the image of the subject T captured by the imaging device 10. This makes it possible to perform individual identification without using, for example, clothing or the like on which identification information of the subject T is formed, making it easy to perform individual identification.

[0134] In the above embodiment, the individual is a person (subject T) as an example, but is not limited to this. The individual may be any object that can be a subject of individual identification, for example, a living thing other than a person, or an object such as a product.

[0135] Furthermore, in the above-described embodiment, an example has been described in which the neck, right shoulder, left shoulder, right buttock, and left buttock of subject T (moving object) are detected as parts of subject T, but other parts (e.g., head, hands, feet, joints, etc.) may also be detected.

[0136] Furthermore, in the above embodiment, the case where individual identification of one individual (subject T) is performed has been described as an example, but individual identification of each of a plurality of individuals may be performed simultaneously.

[0137] In the above embodiment, the case where one recognition device 30 is provided has been described as an example, but the present invention is not limited to this case. For example, a plurality of recognition devices 30 may be provided, and in this case, the operation content and processing result, etc. of any of the recognition devices 30 may be presented in real time on the other recognition devices 30, or the processing result, etc. of any of the recognition devices 30 may be shared among the plurality of recognition devices 30.

[0138] Furthermore, in the above embodiment, the coordinates of each of at least one part of the subject T in the image correspond to the "information on each of at least one part of the individual" of the present invention, but the present invention is not limited to this case. The "information on each of at least one part of the individual" may be, for example, information on the angle of each of at least one part of the subject T in the image, information on the distance between multiple parts of the subject T in the image, or the movement mode (e.g., movement speed or movement distance) of each of at least one part of the subject T in the image within a predetermined period of time.

[0139] Furthermore, in the above-mentioned embodiment, the identification device 30 is configured to realize each function of the first acquisition means 41, the second acquisition means 42, the detection means 43, the conversion means 44, the identification means 45, the first information acquisition means 46, the second information acquisition means 47, and the learning means 48, but this configuration is not limited to this. For example, a learning device 50 (shown in FIG. 14) configured from a computer or the like (for example, a general-purpose personal computer or a server computer) communicably connected to the identification device 30 via a communication network such as the Internet or a LAN may be provided, for learning a model used to identify an individual. In this case, the identification device 30 and the learning device 50 can have substantially the same hardware configuration, so that the learning device 50 can realize the function of at least one of the means 41 to 48 described in the above-mentioned embodiment. For example, the functions of the functional block diagram shown in FIG. 3 may be arbitrarily shared between the identification device 30 and the learning device 50 as shown in FIGS. 14(a) and 14(b).

[0140] Furthermore, the function of at least one of the above-mentioned means 41 to 48 may be realized by the imaging device 10 and / or the measuring device 20. [Industrial Applicability]

[0141] The individual identification system, model learning system, individual identification method, model learning method, and program of the present invention as described above can easily perform individual identification and can be suitably used, for example, in systems that analyze the behavior of individuals (e.g., people such as workers or non-human organisms), information provision systems that provide appropriate information according to the behavior of individuals, and monitoring systems for individuals (e.g., hospitalized patients, facility residents, pets, etc.), and therefore have extremely great industrial applicability. [Explanation of symbols]

[0142] 10...Imaging device 20…Measuring equipment 30...Identification device 41…First acquisition means 42…Second acquisition means 43...Detection means 44...Conversion method 45...Means of identification 46...First information acquisition means 47…Second information acquisition means 48…Learning methods 50...Learning device T…Target person

Claims

1. A first acquisition means for acquiring an image of the individual captured by an imaging device; A detection means for detecting at least one part of the individual based on the acquired image; an identification means for identifying the individual based on information regarding at least one part of the detected individual and a trained model based on machine learning using information regarding at least one part of the individual in an image of the individual as training data; An individual identification system comprising:

2. a second acquisition means for acquiring information regarding the state of the individual measured by the measuring device, The individual identification system described in claim 1, wherein the identification means identifies the individual based on information regarding each of at least one part of the detected individual, information regarding the condition of the individual, and a trained model based on machine learning using information regarding each of at least one part of the individual in the image of the individual and information regarding the condition of the individual as training data.

3. 3. The individual identification system of claim 2, wherein the identification means identifies the individual based on a transition in information regarding each of at least one part of the detected individual within a predetermined period of time, a transition in information regarding the condition of the individual within the predetermined period of time, and the trained model.

4. The individual identification system of claim 1, wherein the identification means identifies the individual based on coordinates in the acquired image of each of at least one detected part of the individual and a trained model based on machine learning using the coordinates of each of at least one part of the individual in the image of the individual as training data.

5. a transformation means for transforming coordinates within the acquired image of each of at least one detected portion of the individual based on a depth of the individual in the acquired image; The individual identification system according to claim 4 , wherein the identification means identifies the individual based on the transformed coordinates of each of the at least one part and the trained model.

6. The individual identification system according to claim 5 , wherein the conversion means converts coordinates within the acquired image of each of at least one part of the detected individual based on a depth of the individual estimated using MiDaS.

7. a conversion means for converting coordinates of at least one detected part of the individual in the acquired image based on a distance between at least two parts of the individual that has been previously measured; The individual identification system according to claim 4 , wherein the identification means identifies the individual based on the transformed coordinates of each of the at least one part and the trained model.

8. The individual identification system according to claim 7, wherein the conversion means converts the coordinates of each of at least one detected part of the individual in the acquired image based on a distance between a neck part and a waist part of the individual that is measured in advance.

9. The individual identification system according to claim 1 , wherein the detection means detects at least one joint of the individual as at least one part of the individual.

10. The individual identification system according to claim 2 , wherein the information on the state of the individual includes at least one of a position, a number of rotations, an angular velocity, an angular acceleration, an attitude, an acceleration, a speed, and a temperature of the individual.

11. The individual identification system according to claim 2 , wherein the state of the individual is measured at the same time as the image of the individual is captured.

12. a first information acquisition means for acquiring information about at least one part of an individual in an image of the individual; A learning means for learning a model used to identify the individual by machine learning using the acquired information on each of the at least one part as learning data; A model learning system comprising:

13. The computer acquiring an image of the individual captured by an imaging device; detecting at least one part of the individual based on the acquired image; identifying the individual based on information about at least one part of the detected individual and a trained model based on machine learning using information about at least one part of the individual in an image of the individual as training data; The individual identification method includes the steps of:

14. The computer obtaining information about each of at least one portion of an individual within an image of the individual; training a model for use in identifying the individual by machine learning using information relating to each of the at least one site obtained as training data; A model training method that executes each step of the process.

15. On the computer, A function of acquiring an image of the individual captured by an imaging device; detecting at least one part of the individual based on the acquired image; A function of identifying the individual based on information about at least one part of the detected individual and a trained model based on machine learning using information about at least one part of the individual in an image of the individual as training data; A program to achieve this.

16. On the computer, A function of acquiring information regarding each of at least one part of an individual in an image of the individual; a function of learning a model used to identify the individual by machine learning using information about each of the at least one acquired part as training data; A program to achieve this.

Images