Inspection device, inspection method, inspection program, learning device, learning method, and learning program

Abstract

The inspection device (20) has a force acquisition unit (21) that acquires a first input (D1b) indicating a force with which to grip the object (30b), an acceleration acquisition unit (22) that acquires a second input (D2b) indicating acceleration, and a determination unit (23) that determines the state of the object (30b) from the first input (D1b) and the second input (D2b) using a learned model for determining the state of the object (30b), which is generated from user information (U) from the first input (D1b) and the second input (D2b). The learning device (10) has a data acquisition unit (13) that acquires learning data including first data (D1a) indicating a force for gripping an object (30a), second data (D2a) indicating an acceleration, and third data (D3a) indicating a state of the object (30a) for a combination of the first data (D1a) and the second data (D2a), and a model generation unit (14) that generates a learned model for determining the state of the object.

Description

[Technical field]

[0001] The present disclosure relates to an inspection device, an inspection method, and an inspection program for inspecting the condition of an object, as well as a learning device, a learning method, and a learning program for generating a trained model used to inspect the condition of an object. [Background technology]

[0002] Conventionally, in the inspection of machinery and equipment, an inspection method is used in which an operator judges the condition of an object (e.g., a piece of equipment) (e.g., whether the object is fixed properly) by vibrating the object with his / her hand. In such an inspection method, the operator judges intuitively whether the object is in a normal condition through his / her own physical movements. Therefore, the judgment results are likely to vary due to the operator's subjective judgment.

[0003] There has also been a proposal for a device that objectively evaluates the tremor (i.e., shaking) of the fingers of a person's hand using an acceleration sensor (see, for example, Patent Document 1). However, this device is a device that evaluates the presence or absence of a disease by detecting the tremor of the fingers, and is not suitable for determining the condition of a target object. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2008-245917 A Summary of the Invention [Problem to be solved by the invention]

[0005] As described above, the conventional methods and devices have a problem in that it is difficult to accurately determine the state of an object.

[0006] An object of the present disclosure is to provide an inspection device, an inspection method, and an inspection program that enable accurate determination of the condition of an object, as well as a learning device, a learning method, and a learning program that generate a trained model that accurately determines the condition of an object. [Means for solving the problem]

[0007] The inspection device disclosed herein is a device for inspecting the condition of an object, and is characterized in having: a force acquisition unit that acquires a first input indicating the force with which the object is gripped detected by a force sensor attached to the fingers of a user's hand; an acceleration acquisition unit that acquires a second input indicating the acceleration detected by an acceleration sensor attached to the hand; and a determination unit that determines the state of the object from the first input and the second input acquired during a period in which the user performs a vibration action of the object, using a trained model for determining the state of the object from the first input and the second input acquired during the period, the trained model being generated using user information about the user.

[0008] The learning device of the present disclosure is a device that generates a trained model for determining the state of an object, and is characterized in having a data acquisition unit that acquires learning data including first data indicating the force of gripping the object detected by a force sensor attached to the fingers of a user's hand, second data indicating the acceleration detected by an acceleration sensor attached to the hand, third data indicating the state of the object for a combination of the first data and the second data during a period in which the user performs a vibration action on the object, and user information about the user, and a model generation unit that generates the trained model for determining the state of the object using the learning data. Effect of the Invention

[0009] By using the device, method, or program of the present disclosure, the state of an object can be determined with high accuracy. [Brief description of the drawings]

[0010] [Figure 1] 1 is a functional block diagram illustrating a configuration of a learning device according to a first embodiment. [Diagram 2] 2 is a diagram illustrating an example of a hardware configuration of the learning device in FIG. 1. [Diagram 3] 2 is a flowchart showing the operation of the learning device of FIG. 1. [Figure 4] FIG. 13 is a functional block diagram illustrating a schematic configuration of a learning device according to a modified example of the first embodiment. [Diagram 5] 13A to 13C are diagrams illustrating the relationship between the fixed state and feature amounts extracted from the force detection value feature amount and the acceleration detection value feature amount. [Figure 6] 5 is a flowchart showing the operation of the learning device of FIG. [Figure 7] FIG. 11 is a functional block diagram illustrating a configuration of an inspection device according to a second embodiment. [Figure 8] 8 is a diagram illustrating an example of a hardware configuration of the inspection device in FIG. 7. [Figure 9] 8 is a flowchart showing the operation of the inspection device of FIG. 7. [Figure 10] FIG. 11 is a functional block diagram illustrating a schematic configuration of an inspection device according to a modified example of the second embodiment. [Figure 11] 11 is a flowchart showing the operation of the inspection device of FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An inspection device, an inspection method, an inspection program, a learning device, a learning method, and a learning program according to embodiments will be described below with reference to the drawings. The following embodiments are merely examples, and the embodiments can be appropriately combined and each embodiment can be appropriately modified.

[0012] <1> Embodiment 1. 1 is a functional block diagram illustrating a schematic configuration of a learning device 10 according to the first embodiment. The learning device 10 is a device that generates a trained model for determining the state of a device 30a that is an object of inspection. The learning device 10 is a device that can implement the learning method according to the first embodiment. The learning device 10 is, for example, a computer that can execute the learning program according to the first embodiment.

[0013] As shown in FIG. 1, the learning device 10 includes a data acquisition unit 13 and a model generation unit 14. The data acquisition unit 13 acquires learning data stored in a learning data storage unit 12 serving as a storage device. The learning data includes a force detection value D1a, which is first data (sensor value) indicating a force for gripping the device 30a detected by a force sensor 51a attached to a finger of a hand 40a of a user as an operator, an acceleration detection value D2a, which is second data (sensor value) indicating an acceleration detected by an acceleration sensor 52a attached to the same hand 40a, and state information D3a (correct answer), which is third data indicating a state of the device 30a for a combination of the force detection value D1a and the acceleration detection value D2a during a period of a vibration operation of the device 30a performed by the same hand 40a. The learning data may include user information U relating to the user. The user information U will be described later. The force sensor 51a and the acceleration sensor 52a may be configured to be attached to a fingertip or a hand, or may be configured to be fixed to the fingertip or the hand by a member that is wrapped around the finger. The hand 40a may be provided with multiple acceleration sensors, multiple force sensors, or multiple acceleration sensors and multiple force sensors. The number, size, and mounting positions of the multiple acceleration sensors or multiple force sensors are preferably such that the wearability on the hand 40a or the workability using the hand 40a are not impaired.

[0014] The model generation unit 14 generates a trained model used to determine the state of the device 30a, which is the object of inspection, by using the learning data acquired by the data acquisition unit 13. The generated trained model is stored in the trained model storage unit 15, which serves as a storage device. The learning data storage unit 12 and the trained model storage unit 15 may be the same storage device. The generated trained model is used to determine the state of the device 30b, which is the object of inspection by the inspection device 20, which is an inference device described in the second embodiment (FIG. 7) described later. The determination of the state of the device 30b is, for example, a determination of whether the fixed state of the device 30b to the support member 31b is good or bad. The device 30a in FIG. 1 and the device 30b in FIG. 7 described later may be the same device, or may be different devices of a similar type (for example, a type in which the fastening states of the devices 30a and 30b to the support members 31a and 31b are similar).

[0015] The learning data input to the learning device 10 is created by a learning data generating device 11 and is stored in advance in a learning data storage unit 12. The learning data generating device 11 is, for example, a computer. The learning data generating device 11 and the learning device 10 may be the same computer.

[0016] The learning data includes a force detection value D1a, which is first data indicating a force with which the user grips the device 30a with a finger (thumb in FIG. 1) of the hand 40a of the user, an acceleration detection value D2a, which is second data indicating an acceleration detected by an acceleration sensor 52a attached to a finger (index finger in FIG. 1) of the same hand 40a, and status information D3a, which is third data indicating a status (e.g., a fixed status) of the device 30a for a combination of the force detection value D1a and the acceleration detection value D2a during a period in which the device 30a is vibrated by the same hand 40a (i.e., an operation of shaking the device with the hand 40a). The status information D3a is, for example, information indicating whether the fixed status of the device 30a with respect to the support member 31a is good or bad. It is desirable to collect the learning data for a plurality of users. In this case, the learning data includes user information U regarding the user.

[0017] The force sensor 51a is disposed, for example, on the pad of a finger of the hand 40a (i.e., the inside of the finger) so as to be interposed between the finger and the device 30a. The force sensor 51a may be attached to a finger other than the thumb of the hand 40a. The force sensor 51a may be a pressure sensor. A plurality of force sensors may be provided as the force sensor 51a. For example, two force sensors may be provided as the force sensor 51a, one for each of the two fingers that grip the device 30a.

[0018] The acceleration sensor 52a is attached to the same hand as the hand to which the force sensor 51a is attached. The acceleration sensor 52a may be attached to a finger other than the index finger of the hand 40a. The acceleration sensor 52a may also be attached to a position of the hand 40a other than the fingers (for example, the palm or the back of the hand). A one-axis, two-axis, or three-axis acceleration sensor can be used as the acceleration sensor 52a. A plurality of acceleration sensors may be provided on the hand 40a that performs the vibration motion as the acceleration sensor 52a.

[0019] Fig. 2 is a diagram illustrating an example of a hardware configuration of the learning device 10 in Fig. 1. As illustrated in Fig. 2, the learning device 10 has a processor 101 such as a CPU (Central Processing Unit), a memory 102 which is a volatile storage device, a non-volatile storage device 103 such as a hard disk drive (HDD) or a solid state drive (SSD), and an interface 104. The memory 102 is, for example, a semiconductor memory such as a RAM (Random Access Memory).

[0020] Each function of the learning device 10 is realized, for example, by a processing circuit. The processing circuit may be dedicated hardware, or may be a processor 101 that executes a program stored in a memory 102. The processor 101 may be any of a processing device, an arithmetic device, a microprocessor, a microcomputer, and a DSP (Digital Signal Processor).

[0021] Where the processing circuitry is dedicated hardware, the processing circuitry may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or any combination thereof.

[0022] When the processing circuit is the processor 101, the executed learning program is realized by software, firmware, or a combination of software and firmware. The learning program is installed in the learning device 10 via a network or from a recording medium. The software and firmware are written as a program and stored in the memory 102. The processor 101 can realize the functions of each part shown in FIG. 1 by reading and executing the learning program stored in the memory 102.

[0023] The learning device 10 may be partially realized by dedicated hardware and partially realized by software or firmware. In this way, the processing circuit can realize each of the above-mentioned functions by hardware, software, firmware, or any combination of these.

[0024] Fig. 3 is a flowchart showing the operation of the learning device 10 of Fig. 1. As shown in Fig. 3, in the learning phase, in step S11, the data acquisition unit 13 acquires learning data including a force detection value D1a which is the first data, an acceleration detection value D2a which is the second data, state information D3a which is the third data indicating the state of the device 30a for the combination of the force detection value D1a and the acceleration detection value D2a (i.e., the correct answer), and user information U.

[0025] In step S12, the model generation unit 14 generates a trained model in accordance with the training data created based on the acquired training data.

[0026] In step S13, the trained model storage unit 15 stores the trained model generated by the model generation unit 14.

[0027] By using a trained model generated using the learning device, learning method, and learning program according to embodiment 1, the state of the object can be judged with high accuracy since the judgment is not influenced by the operator's subjectivity.

[0028] (2) Variation of the first embodiment. 4 is a functional block diagram illustrating a schematic configuration of a learning device 10a according to a modification of the first embodiment. The learning device 10a is a device that generates a trained model for determining the state of an equipment that is an object of inspection. The learning device 10a is a device that can implement the learning method according to the modification of the first embodiment, and is, for example, a computer.

[0029] 1, the learning device 10a includes a data acquisition unit 13 and a model generation unit 14. The learning device 10a also includes an excitation period extraction unit 16, a feature extraction unit 17, a user adaptation processing unit 18, and a recording unit 19.

[0030] The vibration period extraction unit 16 extracts a vibration period, which is a period of vibration operation, based on at least one of the force detection value D1a, which is the first data (sensor value), and the acceleration detection value D2a, which is the second data (sensor value). For example, the vibration period extraction unit 16 can determine a period during which the force detection value D1a exceeds a predetermined first set value as a vibration period. The vibration period extraction unit 16 can also determine a period during which the acceleration detection value D2a exceeds a predetermined second set value as a vibration period. The vibration period extraction unit 16 may also determine a period during which the force detection value D1a exceeds a predetermined first set value and the acceleration detection value D2a exceeds a predetermined second set value as a vibration period. The vibration period extraction unit 16 may also determine a period designated by a user operation as a vibration period. Other known methods may be used to extract the vibration period.

[0031] The feature extraction unit 17 extracts a force detection value feature, which is a first feature, from the force detection value D1a, which is the first data, and extracts a force detection value feature, which is a second feature, from the acceleration detection value D2a, which is the second data. The force detection value feature is, for example, a variance value, a peak-to-peak value, an average value, etc. The acceleration detection value feature is, for example, a variance value, a peak-to-peak value, an average value, etc., and a value for acceleration on three axes may be used.

[0032] The user adaptation processing unit 18 weights the feature amounts based on the force detection value feature amount which is the first feature amount, the force detection value feature amount which is the second feature amount, and the user information U. The weighted feature amount x is calculated, for example, by the following formula (1).

[0033]

number

[0034] In formula (1), β a is a parameter that represents the overall trend of acceleration, Acc is the acceleration detection value feature value, and β p is a parameter that represents the overall force tendency, Prs is the force detection value feature value, β sbj is a parameter that represents an individual difference set for each user who is an operator performing the inspection.

[0035] The model generation unit 14 generates multiple trained models corresponding to user information U including at least one of the user's specific information and the user's attributes, and stores them in the trained model storage unit 15 as a storage unit. The user's attributes include the user's years of experience in inspection work (i.e., information indicating the number of times the work has been performed). The user's attributes also include the user's physical information. The physical information includes, for example, one or more pieces of information such as the user's height, weight, muscle mass, gender, and age. Here, the users are grouped by their attributes, and then a model is constructed for each group. In this way, the variance in the training data used to construct the model and the inference target data used for inference can be reduced, making it possible to estimate the state of the equipment with high accuracy.

[0036] 5(A) to 5(C) are diagrams showing the relationship between the feature amount obtained from the force detection value feature amount and the acceleration detection value feature amount and the fixed state. In FIG. 5(A) to 5(C), the input (horizontal axis) is the feature amount based on the acceleration sensor and the force sensor (e.g., pressure sensor) which are the vibration motion measuring sensors by the user, and the output (vertical axis) is the index value indicating the fixed state of the equipment, and the relationship between them is expressed by a hierarchical Bayesian model. FIG. 5(A) shows the overall tendency of multiple users, FIG. 5(B) shows the tendency of one user (worker #1), and FIG. 5(C) shows the tendency of one user (worker #2). In FIG. 5(A) to 5(C), the horizontal axis is the feature amount obtained from the force detection value feature amount and the acceleration detection value feature amount, and is, for example, the sum of the variance of the detected acceleration and the maximum value of the detected force (pressure). In FIG. 5(A) to 5(C), the vertical axis shows the degree of loosening of the equipment fastening part.

[0037] Fig. 6 is a flowchart showing the operation of the learning device 10a of Fig. 4. As shown in Fig. 6, in step S101, the feature extractor 17 extracts a force detection value feature from a force detection value D1a which is a first data during a vibration period, and extracts an acceleration detection value feature from an acceleration detection value D2a which is a second data during a vibration period.

[0038] In step S102, the user adaptation processing unit 18 weights each of the force detection value feature amount and the acceleration detection value feature amount based on the user information U.

[0039] In step S103, the user adaptation processing unit 18 associates the combination of the weighted force detection value feature and the weighted acceleration detection value feature with state information (third data) D3a (i.e., the correct answer) regarding the fixed state of the device at that time.

[0040] In step S104, the model generation unit 14 calculates the overall acceleration tendency parameter β a and the overall force tendency parameter β p and individual parameter β sbj and are trained using the Markov chain Monte Carlo method to generate a trained model.

[0041] In step S105, the recording unit 19 records the overall acceleration tendency parameter β a and the overall force tendency parameter β p and individual parameter β sbj and are stored in the learned model storage unit 15. The recording unit 19 can record the index value (for example, a numerical value indicating the degree of wobble) that is the result of estimating the state of the object, in association with the time of the vibration operation period, the name of the device as the object, etc.

[0042] By using a trained model generated using the learning device 10a, learning method, and learning program related to a modified example of embodiment 1, the state of the object can be accurately determined because the operator's subjectivity does not affect the judgment.

[0043] Furthermore, by using force information in addition to acceleration information, the equipment can be regarded as a transmission system when the excitation force is the input, and the fixed state can be regarded as the output, making it possible to estimate the equipment state with greater accuracy.

[0044] In addition, by taking into account the variability among users parametrically, the fixation state of the device can be estimated using a unified index that is robust to variability.

[0045] In addition, when acquiring learning data for a new user who has recently joined the workforce, it is not necessary to acquire observation data after setting all equipment to a loose state; instead, a learning model for estimating the fixed state of the new user can be constructed using data from other users, thereby reducing the burden of data collection.

[0046] 3. Embodiment 2 7 is a functional block diagram showing a schematic configuration of an inspection device 20 according to the second embodiment. The inspection device 20 is an inference device for inspecting the state of a device 30b that is an object of inspection, using a trained model that is generated by the learning device 10 or 10a and stored in advance in a trained model storage unit 24 as a storage device. The inspection device 20 is a device capable of implementing an inspection method according to the second embodiment. The inspection device 20 is, for example, a computer capable of executing an inspection program according to the second embodiment.

[0047] As shown in FIG. 7, the inspection device 20 has a force acquisition unit 21, an acceleration acquisition unit 22, and a determination unit 23 (i.e., an inference unit). The force acquisition unit 21 acquires a force detection value D1b as a first input indicating a force for gripping the device 30b, which is an object, detected by a force sensor 51b attached to a finger of the user's hand 40b. The acceleration acquisition unit 22 acquires an acceleration detection value D2b as a second input indicating an acceleration detected by an acceleration sensor 52b attached to the user's hand 40b. The determination unit 23 is an inference unit that determines (i.e., infers) the state of the device 30b from the force detection value D1b and the acceleration detection value D2b acquired during the vibration period, using a learned model for determining the state of the device 30b from the force detection value D1b and the acceleration detection value D2b acquired during the vibration period, which is a period during which the user performs a vibration operation of the device 30b. The state of the device 30b is, for example, a fixed state of the device 30b with respect to the support member 31b.

[0048] The force sensor 51b is, for example, the pad of a finger of the hand 40b, and is disposed so as to be interposed between the finger and the device 30b. The force sensor 51b may be attached to a finger other than the thumb of the hand 40b. The force sensor 51b may be a pressure sensor. A plurality of force sensors may be provided as the force sensor 51b. For example, two force sensors may be provided as the force sensor 51b on two fingers that grip the device 30a. The force sensor 51b is attached to the fingers of the hand 40b of the user as an operator, and outputs a force detection value D1b that is a first input (sensor value) indicating the force with which the device 30b is gripped.

[0049] The acceleration sensor 52b is attached to the same hand as the hand to which the force sensor 51b is attached. The acceleration sensor 52b may be attached to a finger other than the index finger of the hand 40b. The acceleration sensor 52b may also be attached to a position other than the fingers of the hand 40b (for example, the palm or the back of the hand). A one-axis, two-axis, or three-axis acceleration sensor may be used as the acceleration sensor 52b. As the acceleration sensor 52b, a plurality of acceleration sensors may be provided on the hand 40b performing the vibration operation. The acceleration sensor 52b is attached to the same hand 40b to which the force sensor 51b is attached, and outputs an acceleration detection value D2b which is a second input (sensor value) indicating acceleration. The force sensor 51b and the acceleration sensor 52b may be configured to attach the sensor to the fingertip or the hand, or may be configured to fix the sensor to the fingertip or the hand by a member wrapped around the finger. The hand 40b may also be provided with a plurality of acceleration sensors, a plurality of force sensors, or a plurality of acceleration sensors and a plurality of force sensors. Moreover, it is preferable that the number, size, and mounting positions of the multiple acceleration sensors or multiple force sensors are such that the ease of wearing the sensor on the hand 40b or the ease of operation using the hand 40b are not impaired.

[0050] Fig. 8 is a diagram illustrating an example of a hardware configuration of the inspection device 20 of Fig. 7. As illustrated in Fig. 8, the inspection device 20 has a processor 201 such as a CPU, a memory 202 which is a volatile storage device, a non-volatile storage device 203 such as an HDD or SSD, and an interface 204. The memory 202 is, for example, a semiconductor memory such as a RAM.

[0051] Each function of the inspection device 20 is realized by, for example, a processing circuit. The processing circuit may be dedicated hardware, or may be a processor 201 that executes a program stored in a memory 202.

[0052] Where the processing circuitry is dedicated hardware, the processing circuitry may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or any combination of these.

[0053] When the processing circuit is the processor 201, the inspection program to be executed is realized by software, firmware, or a combination of software and firmware. The inspection program is installed in the inspection device 20 via a network or from a recording medium. The software and firmware are written as a program and stored in the memory 202. The processor 201 can realize the functions of each part shown in FIG. 7 by reading and executing the learning program stored in the memory 202.

[0054] The inspection device 20 may be partially realized by dedicated hardware and partially realized by software or firmware. In this way, the processing circuit can realize each of the above-mentioned functions by hardware, software, firmware, or any combination of these.

[0055] Fig. 9 is a flowchart showing the operation of the inspection device 20 of Fig. 7. As shown in Fig. 9, in the inference phase, in step S21, the force acquisition unit 21 acquires a force detection value D1b which is a first input, and the acceleration acquisition unit 22 acquires an acceleration detection value D2b which is a second input.

[0056] In steps S22 and S23, the determination unit 23 inputs the detected force value D1b and the detected acceleration value D2 into the learning data to be used, and outputs a determination result (that is, an inference result).

[0057] By using the inspection device 20, the inspection method, and the inspection program according to the second embodiment, the state of the object can be judged with high accuracy since the judgment is not influenced by the subjective judgment of the operator.

[0058] {4} Variation of the second embodiment. 10 is a functional block diagram showing a schematic configuration of an inspection device 20a according to a modification of the second embodiment. The inspection device 20a is an inference device for inspecting the state of a device 30b that is an object of inspection, using a trained model that is generated by the learning device 10 or 10a and stored in advance in a trained model storage unit 24 as a storage device. The inspection device 20a is a device capable of implementing an inspection method according to a modification of the second embodiment, and is, for example, a computer.

[0059] 7, the inspection device 20a has a force acquisition unit 21, an acceleration acquisition unit 22, and a judgment unit 23 as an inference unit. The inspection device 20a also has a force feature extraction unit 25, a vibration period extraction unit 26, an acceleration feature extraction unit 27, a user adaptation processing unit 28, and a judgment result recording unit 29.

[0060] The vibration period extraction unit 26 extracts a vibration period, which is a period of a vibration operation, based on at least one of the force detection value D1b, which is the first input, and the acceleration detection value D2b, which is the second input. The method of extracting the vibration period may be the method described in the modification of the first embodiment.

[0061] The force feature amount extraction unit 25 extracts a force detection value feature amount, which is a first feature amount, from the force detection value D1b, which is a first input. The force detection value feature amount is, for example, a variance value, a peak-to-peak value, an average value, etc. The acceleration feature amount extraction unit 27 extracts an acceleration detection value feature amount, which is a second feature amount, from the acceleration detection value D2b, which is a second input. The acceleration detection value feature amount is, for example, a variance value, a peak-to-peak value, an average value, etc.

[0062] The user adaptation processing unit 28 weights the features based on the force detection value feature, which is the first feature, the force detection value feature, which is the second feature, and the user information U. The weighted feature x is calculated, for example, by the above formula (1). The user adaptation processing unit 28 selects a trained model to be used from the trained model storage unit 24 based on the user information U. Specifically, the user adaptation processing unit 18 reads the acceleration overall tendency parameter, the force overall tendency parameter, and the individual parameter from the trained model storage unit 15, and performs weighting processing of the input features. The user attributes include the user's years of experience in the inspection work (i.e., information indicating the number of times the work has been experienced), etc. The user attributes also include the user's physical information. The physical information includes, for example, one or more of the user's height, weight, muscle mass, sex, age, hand shape (palm size, finger length), grip strength, muscle endurance, explosive power, etc. measured in advance. Here, by selecting a corresponding trained model from trained models grouped by user attributes, the variation in the training data used to construct the trained model and the inference target data used for inference can be reduced, so that the state of the device can be estimated with high accuracy. The hand may be equipped with multiple acceleration sensors, multiple force sensors, or multiple acceleration sensors and multiple force sensors. In this case, the state of the hand (e.g., joint positions, joint angles) may be estimated from the observation information thereof, and the information may be used as attribute information of a new user and used for new input and selection of a training model.

[0063] The determination unit 23 receives the weighted force detection value feature amount and the weighted acceleration detection value feature amount as input, calculates a score g of the state of the device 30b, and determines the fixed state of the device 30b based on the score g. In the score calculation, an index value is estimated through a nonlinear function f using the processing result of the user adaptation processing unit 28 using the learned parameters. In the case of grouping, a corresponding learned model is selected, and an index value is estimated. The score calculation is performed, for example, using the following formula (2) using a hierarchical Bayes model and regression analysis.

[0064]

number

[0065] In formula (2), g is an index value indicating the fixed state (e.g., the amount of rotation of the screw), x is a weighted feature, and β a is a parameter that represents the overall trend of acceleration, Acc is the acceleration detection value feature value, and β p is a parameter that represents the overall force tendency, Prs is the force detection value feature value, β sbj is a parameter that represents the individual difference set for each user who performs the inspection, and σ sbj is a parameter that represents the variability of the probability model.

[0066] The determination result recording unit 29 outputs the determination result (i.e., the inference result) or records it in a storage device. The determination result recording unit 29 can record an index value (e.g., a numerical value indicating the degree of wobble) that is a result of estimating the state of the equipment 30b as the target object, in association with the time of the vibration operation period, the name of the equipment as the target object, etc.

[0067] Fig. 11 is a flowchart showing the operation of the inspection device 20a of Fig. 10. As shown in Fig. 11, in step S201, the force acquisition unit 21 acquires a force detection value D1b, which is a first input, during a vibration period, and the acceleration acquisition unit 22 acquires an acceleration detection value D2b, which is a second input.

[0068] In step S202, the force feature extraction unit 25 calculates the force detection value feature from the force detection value D1b, which is the first input during the vibration period, and the acceleration feature extraction unit 27 calculates the acceleration detection value feature from the acceleration detection value (second input), which is the second input during the vibration period.

[0069] In step S203, the user adaptation processing unit 18 reads the overall acceleration tendency parameters, the overall force tendency parameters, and the individual parameters from the learned model storage unit 15, and performs weighting processing on the input feature amounts.

[0070] In step S204, the determination unit 23 receives the weighted feature amount x, calculates a score g of the state of the device, and determines the fixed state of the device based on the score g. The score calculation is performed using, for example, the above formula (2) using a hierarchical Bayes model and regression analysis.

[0071] By using the inspection device 20a, the inspection method, and the inspection program according to the modified example of the second embodiment, the state of the object can be judged with high accuracy since the judgment is not influenced by the subjective judgment of the operator. [Explanation of symbols]

[0072] 10, 10a Learning device, 11 Learning data generation device, 12 Learning data storage unit, 13 Data acquisition unit, 14 Model generation unit, 15 Learned model storage unit, 16 Vibration period extraction unit, 17 Feature extraction unit, 18 User adaptation processing unit, 19 Recording unit, D1a Force detection value (first data), D2a Acceleration detection value (second data), D3a Status information (third data), U User information, 20, 20a Inspection device, 21 Force acquisition unit, 22 Acceleration acquisition unit, 23 Judgment unit, 24 Learned model storage unit, 25 Force feature extraction unit, 26 Vibration period extraction unit, 27 Acceleration feature extraction unit, 28 User adaptation processing unit, 29 Judgment result recording unit, 30a, 30b Equipment (target object), 31a, 31b Support member, 40a, 40b hands, 51a, 51b force sensors, 52a, 52b acceleration sensors, D1b force detection value (first input), D2b acceleration detection value (second input).

Claims

1. An inspection device for inspecting a state of an object, comprising: a force acquisition unit that acquires a first input indicating a force with which the user is gripping the object, the force being detected by a force sensor attached to a finger of the user's hand; an acceleration acquisition unit that acquires a second input indicating an acceleration detected by an acceleration sensor attached to the hand; a determination unit that determines a state of the object from the first input and the second input acquired during a period of a vibration operation of the object performed by the user, using a trained model for determining a state of the object, the trained model being generated using user information related to the user, from the first input and the second input acquired during the period; An inspection device comprising:

2. the state of the object is a fixed state of the object relative to a support member, The determination unit determines whether the fixed state of the object is good or bad.

2. The inspection apparatus according to claim 1 .

3. The vibration period extracting unit extracts the period of the vibration operation based on at least one of the first input and the second input.

3. The inspection apparatus according to claim 1 or 2.

4. The method further includes a user adaptation processing unit that selects the learned model to be used by the determination unit from a plurality of learned models stored in advance according to the user information including at least one of the user's specific information and the user's attributes.

3. The inspection device according to claim 1, wherein the inspection device is a semiconductor laser.

5. 5. The inspection apparatus according to claim 4, wherein the user attributes include the number of years of experience of the user in inspection work.

6. 5. The inspection device according to claim 4, wherein the user attributes include physical information of the user.

7. a feature extraction unit configured to extract a first feature from the first input; an acceleration feature amount extraction unit that extracts a second feature amount from the second input; Further comprising: The user adaptation processing unit selects the trained model based on the first feature amount, the second feature amount, and the user information.

5. The inspection apparatus according to claim 4.

8. An inspection method performed by an inspection device that inspects a state of an object, comprising: obtaining a first input indicative of a force with which the user is gripping the object, the force being detected by a force sensor attached to a finger of the user's hand; obtaining a second input indicative of acceleration detected by an acceleration sensor attached to the hand; determining a state of the object from the first input and the second input acquired during a period of a vibration operation of the object performed by the user, using a trained model for determining a state of the object, the trained model being generated using user information related to the user; and An inspection method comprising the steps of:

9. An inspection program for causing a computer to execute the inspection method according to claim 8.

10. A learning device that generates a trained model for determining a state of an object, a data acquisition unit that acquires learning data including first data indicating a force with which the object is gripped, the force being detected by a force sensor attached to the fingers of the user's hand, second data indicating acceleration detected by an acceleration sensor attached to the hand, third data indicating a state of the object with respect to a combination of the first data and the second data during a period in which the user performs a vibration motion on the object, and user information on the user; a model generation unit that generates the trained model for determining a state of the object using the training data; A learning device comprising:

11. the state of the object is a fixed state of the object relative to a support member, The model generation unit generates the trained model for determining whether a fixed state of the object is good or bad. The learning device according to claim 10 .

12. The vibration period extracting unit extracts the period of the vibration operation based on at least one of the first data and the second data.

12. The learning device according to claim 10 or 11.

13. The model generation unit generates a plurality of the trained models corresponding to the user information including at least one of the user's specific information and the user's attributes, and stores the trained models in a storage unit.

12. The learning device according to claim 10 or 11.

14. 14. The learning device according to claim 13, wherein the user attributes include the number of years of experience of the user in inspection work.

15. 14. The learning device according to claim 13, wherein the user attributes include physical information of the user.

16. 14. The learning device according to claim 13, wherein the user attributes include physical ability information of the user.

17. A learning method executed by a learning device that generates a trained model for determining a state of an object, acquiring learning data including first data indicating a force with which the object is grasped, the force being detected by a force sensor attached to the fingers of the user's hand; second data indicating acceleration detected by an acceleration sensor attached to the hand; third data indicating a state of the object with respect to a combination of the first data and the second data during a period in which the user performs a vibration motion of the object; and user information regarding the user; generating the trained model for determining the state of the object using the training data; A learning method comprising:

18. A learning program for causing a computer to execute the learning method according to claim 17.

Images