Inspection assistance device, inspection assistance system, inspection assistance method, and program

Abstract

This inspection assistance device (10) outputs information pertaining to a gripping and shaking method by which an object (50) is gripped and shaken, the information being outputted using determination reference data that has been stored upon being acquired by gripping and shaking an object (50) using a hand on which a pressure sensor (21) and an acceleration sensor (22) are worn. This inspection assistance device (10) comprises: a similarity evaluation unit (12) that, in an actual inspection, calculates a first pressure similarity (S1p) between pressure data and first reference pressure data (R1p), a first acceleration similarity (S1a) between acceleration data and first reference acceleration data (R1a), a second pressure similarity (S2p) between pressure data and second reference pressure data (R2p), and a second acceleration similarity (S2a) between acceleration data and second reference acceleration data (R2a); a gripping and shaking determination unit (13) that performs a determination with regards to the gripping and shaking method on the basis of the similarities; and a presentation information generation unit (14).

Description

Inspection support device, inspection support system, inspection support method, and program

[0001] The present disclosure relates to an inspection support device, an inspection support system, an inspection support method, and a program.

[0002] A device has been proposed that applies force to an object using a hand wearing a glove-type wearable sensor equipped with a pressure sensor, causing it to vibrate, and then determines the state of engagement of the object based on the visual, auditory (e.g., sound), and tactile (e.g., vibration) information obtained at that time (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2021-1410

[0004] However, the above-mentioned conventional technology does not determine whether the worker's gripping and vibrating of the object (also referred to as "gripping vibration") is appropriate, so there is a problem in that when a less skilled worker grips and vibrates the object, the accuracy of determining the fixed state of the object is low.

[0005] The present disclosure aims to provide an inspection support device, an inspection support system, an inspection support method, and a program that enable improving the accuracy of determining the fixation state of an object by determining the appropriateness of grasping and vibrating the object.

[0006] The inspection support device of the present disclosure prepares, as judgment criteria data acquired and stored by grasping and vibrating an object with the hands to which the pressure sensor and acceleration sensor are attached, first reference pressure data and first reference acceleration data acquired from the pressure sensor and the acceleration sensor when the hand to which the pressure sensor and the acceleration sensor are attached intentionally performs inappropriate grasping and vibration on the object, and second reference pressure data and second reference acceleration data acquired from the pressure sensor and the acceleration sensor when the hand to which the pressure sensor and the acceleration sensor are attached performs appropriate grasping and vibration on the object, and when an operator grasps and vibrates the object in an actual inspection, outputs information regarding the method of grasping and vibrating the object using the judgment criteria data. The inspection support device is characterized by having: a similarity evaluation unit that, during the actual inspection, calculates a first pressure similarity that is the similarity between the pressure data output from the pressure sensor and the first reference pressure data, calculates a first acceleration similarity that is the similarity between the acceleration data output from the acceleration sensor and the first reference acceleration data, calculates a second pressure similarity that is the similarity between the pressure data output from the pressure sensor and the second reference pressure data, and calculates a second acceleration similarity that is the similarity between the acceleration data output from the acceleration sensor and the second reference acceleration data; a grip vibration determination unit that makes a determination regarding the grip vibration method based on the first pressure similarity, the first acceleration similarity, the second pressure similarity, and the second acceleration similarity; and a presentation information generation unit that generates presentation information regarding the grip vibration method based on a result of the determination.

[0007] The inspection support method disclosed herein is a method executed by an inspection support device that prepares, as judgment criteria data acquired and stored by grasping and vibrating an object with hands having pressure sensors and acceleration sensors attached, first reference pressure data and first reference acceleration data acquired from the pressure sensors and acceleration sensors when an intentionally inappropriate grasping and vibration is performed on the object with the hands having the pressure sensors and acceleration sensors attached, and second reference pressure data and second reference acceleration data acquired from the pressure sensors and acceleration sensors when an appropriate grasping and vibration is performed on the object with the hands having the pressure sensors and acceleration sensors attached, and outputs information regarding a method of grasping and vibrating the object using the judgment criteria data when an operator grasps and vibrates the object in an actual inspection. The inspection support method includes the steps of, during the actual inspection, calculating a first pressure similarity that is a similarity between the pressure data output from the pressure sensor and the first reference pressure data, calculating a first acceleration similarity that is a similarity between the acceleration data output from the acceleration sensor and the first reference acceleration data, calculating a second pressure similarity that is a similarity between the pressure data output from the pressure sensor and the second reference pressure data, and calculating a second acceleration similarity that is a similarity between the acceleration data output from the acceleration sensor and the second reference acceleration data; making a determination regarding the grip vibration method based on the first pressure similarity, the first acceleration similarity, the second pressure similarity, and the second acceleration similarity; and generating presentation information regarding the grip vibration method based on a result of the determination.

[0008] According to the present disclosure, the accuracy of determining the fixed state of an object can be improved by determining the appropriateness of grasping and vibrating the object.

[0009] 1 is a functional block diagram illustrating an outline of the configuration of an inspection support device and an inspection support system according to an embodiment; FIG. 2 is a schematic diagram illustrating an example of the sensor glove illustrated in FIG. 1; FIG. 3 is a side view illustrating an example of grip vibration excitation of an object; FIG. 4 is a side view illustrating another example of grip vibration excitation of an object; FIG. 5 is a side view illustrating another example of grip vibration excitation of an object; FIG. 6 is a diagram illustrating an example of the hardware configuration of an inspection support device and an inspection support system according to an embodiment; FIG. 7 is a functional block diagram illustrating an outline of the configuration of a sensor data collection device; FIG. 8 is a diagram illustrating an example of the hardware configuration of a sensor data collection device; FIG. 9 is a flowchart illustrating the process of collecting sensor data during appropriate grip vibration excitation; FIG. 10 is a flowchart illustrating the process of collecting sensor data during inappropriate grip vibration excitation; FIG. 11 is a flowchart illustrating the operation of an inspection support device and an inspection support system according to an embodiment; FIG. 12 is a flowchart illustrating another operation of an inspection support device and an inspection support system according to an embodiment;

[0010] The following describes an examination support device, an examination support system, an examination support method, and a program according to embodiments, with reference to the drawings. The following embodiments are merely examples, and the embodiments can be combined as appropriate and each embodiment can be modified as appropriate.

[0011] 1 is a functional block diagram showing the schematic configuration of an examination support device 10 and an examination support system 1 according to an embodiment. The examination support device 10 is a device capable of implementing an examination support method according to an embodiment. The examination support device 10 is, for example, a computer that executes a program as software for implementing the examination support method.

[0012] The inspection support system 1 includes an inspection support device 10, a pressure sensor 21 and an acceleration sensor 22 attached to the hand of an operator, and an information presentation unit 30 that presents information to the operator. The pressure sensor 21 is located at the tip of the finger of the hand and detects the force applied to an object 50 by the finger and outputs a pressure signal as an electrical signal. The acceleration sensor 22 detects acceleration in orthogonal X, Y, and Z directions (i.e., three orthogonal axis directions) and outputs an acceleration signal as an electrical signal.

[0013] When an operator grasps and vibrates the object 50 with the hands to which the pressure sensor 21 and the acceleration sensor 22 are attached during an actual inspection, the inspection support device 10 outputs information about a method of grasping and vibrating the object 50 using judgment reference data that has been acquired and stored in advance by grasping and vibrating the object 50 with the hands to which the pressure sensor 21 and the acceleration sensor 22 are attached. The judgment reference data includes, for example, first reference pressure data (inappropriate reference pressure data) R1p and first reference acceleration data (inappropriate reference acceleration data) R1a acquired from the pressure sensor 21 and the acceleration sensor 22 when an operator intentionally performs inappropriate grasping and vibration with the hands to which the pressure sensor 21 and the acceleration sensor 22 are attached, and second reference pressure data (appropriate reference pressure data) R2p and second reference acceleration data (appropriate reference acceleration data) R2a acquired from the pressure sensor 21 and the acceleration sensor 22 when an appropriate grasping and vibration is performed on the object with the hands to which the pressure sensor 21 and the acceleration sensor 22 are attached. Appropriate grip vibration is grip vibration that satisfies the grip vibration conditions, such as the grip position, grip strength, vibration direction, vibration strength, vibration speed, and vibration stroke during vibration, that are predetermined for each object. Inappropriate grip vibration is grip vibration that does not satisfy the grip vibration conditions, such as the grip position, grip strength, vibration direction, vibration strength, vibration speed, and vibration stroke during vibration, that are predetermined for each object.

[0014] The inspection support device 10 includes a similarity evaluation unit 12, a grip vibration determination unit 13, a presentation information generation unit 14, and storage devices 11, 41, and 42. The storage devices 11, 41, and 42 do not need to be part of the inspection support device 10, but may be storage devices provided in an external device (e.g., a server computer on a network) connected to the inspection support device 10 so as to be able to communicate with the inspection support device 10.

[0015] In an actual inspection, the similarity evaluation unit 12 calculates a first pressure similarity S1p, which is the similarity between the pressure data output from the pressure sensor 21 and the first reference pressure data R1p, and calculates a first acceleration similarity S1a, which is the similarity between the acceleration data output from the acceleration sensor 22 and the first reference acceleration data R1a. In addition, the similarity evaluation unit 12 calculates a second pressure similarity S2p, which is the similarity between the pressure data output from the pressure sensor 21 and the second reference pressure data R2p, and calculates a second acceleration similarity S2a, which is the similarity between the acceleration data output from the acceleration sensor 22 and the second reference acceleration data R2a.

[0016] The grip vibration determination unit 13 determines the grip vibration method used by the worker based on the first pressure similarity S1p and first acceleration similarity S1a, which are similarities with the determination reference data at the time of inappropriate grip vibration, and the second pressure similarity S2p and second acceleration similarity (S2a), which are similarities with the determination reference data at the time of appropriate grip vibration. The determination of the grip vibration method is, for example, a determination of one or more of the grip position, grip strength, vibration direction, vibration strength, vibration speed, and vibration stroke at the time of vibration.

[0017] For example, the grip vibration determination unit 13 can determine that the grip vibration method is inappropriate when at least one of the conditions that the first pressure similarity S1p is greater than or equal to the second pressure similarity S2p and the first acceleration similarity S1a is greater than or equal to the second acceleration similarity S2a is satisfied.

[0018] Alternatively, the grip vibration determination unit 13 can determine whether the grip vibration method is appropriate or inappropriate based on whether the first pressure similarity S1p is equal to or greater than a predetermined first pressure threshold T1p, whether the first acceleration similarity S1a is equal to or greater than a predetermined first acceleration threshold T1a, whether the second pressure similarity S2p is equal to or greater than a predetermined second pressure threshold T2p, and whether the second acceleration similarity S2a is equal to or greater than a predetermined second acceleration threshold T2a.

[0019] Alternatively, the grip vibration determination unit 13 can determine that the grip vibration method is inappropriate when at least one of the conditions that the first pressure similarity S1p is greater than or equal to a predetermined first pressure threshold T1p and the first acceleration similarity S1a is greater than or equal to a predetermined first acceleration threshold T1a is satisfied.

[0020] The presentation information generator 14 generates presentation information related to the gripping and vibration method based on the determination result. The presentation information may include, for example, one or more pieces of information such as an appropriate gripping position, an appropriate gripping strength, an appropriate vibration direction, an appropriate vibration strength, an appropriate vibration speed, and an appropriate vibration stroke during vibration.

[0021] Specifically, the presentation information generation unit 14 may include, as presentation information, "The gripping position is inappropriate," "Please grip the surface, not the corner," "The hand is facing backwards," "The vibration direction is incorrect," "The vibration force is too strong," "The vibration force is too weak," and the like.

[0022] FIG. 2 is a schematic diagram illustrating an example of the sensor glove 20 shown in FIG. 1 . The sensor glove 20 is a glove-type device worn on the hand of a worker. Pressure sensors 21 are provided on the gripping surfaces of the fingertips of the sensor glove 20. In the example of FIG. 2 , a pressure sensor 21 is provided on each of the gripping surfaces of the thumb and index finger of the sensor glove 20. The gripping surfaces are surfaces that face the object 50 when the object 50 is gripped with two fingers. When gripping the object 50, each pressure sensor 21 is disposed between the finger and the object 50. Although two pressure sensors 21 are shown in FIG. 2 , a single pressure sensor 21 may be used. Furthermore, in the example of FIG. 2 , acceleration sensors 22 are provided on the thumb and index finger of the sensor glove 20. Although two acceleration sensors 22 are shown in FIG. 2 , a single acceleration sensor 22 may be used. The acceleration sensors 22 may be attached to a position other than the fingertips. The pressure sensor 21 and the acceleration sensor 22 may also be attached directly to the worker's finger or hand.

[0023] 3 is a side view showing an example of a gripping and vibrating operation on an object 50 by a worker wearing the sensor glove 20 on his / her hand. Fig. 3 shows the state in which the object 50 (e.g., a device) fixed to the upper surface of a support member 51 by screws 53 and 54 as fastening parts is gripped from both above and below and vibrated in the D1 direction. While gripping and vibrating in this way, the worker determines whether the object 50 is properly fixed based on sound information, tactile information transmitted to the hand, etc. The determination of whether the object 50 is properly fixed can also be made based on the waveforms of the detection signals from the pressure sensor 21 and the acceleration sensor 22.

[0024] 4 is a side view showing another example (an example of appropriate gripping) of a gripping and vibrating operation on an object 50 by a worker wearing the sensor glove 20 on his / her hand. Fig. 4 shows the state in which the object 50, which is fixed to the side of a support member 51 by screws 52, 53, and 54 as fastening parts, is gripped from both above and below and vibrated in the D2 direction. While gripping and vibrating in this way, the worker determines whether the object 50 is properly fixed based on sound information, tactile information transmitted to the hand, and the like. The determination of whether the object 50 is properly fixed can also be made based on the waveforms of the detection signals from the pressure sensor 21 and the acceleration sensor 22.

[0025] 5 is a side view showing another example of a vibration application task (an example of improper gripping) to an object 50 by a worker wearing the sensor glove 20 on his hand. Fig. 5 shows the situation in which two corners of the object 50, which is fixed to the side surface of a support member 51 by screws 52, 53, and 54 as fastening parts, are grasped from both sides and vibration is applied in the D3 direction. While applying vibration by gripping in this way, the worker determines whether the object 50 is properly fixed based on sound information, tactile information transmitted to the hand, etc.

[0026] FIG. 6 is a diagram illustrating an example of the hardware configuration of the test support device 10 and the test support system 1 according to an embodiment. As illustrated in FIG. 6, the test support device 10 includes a processor 101 such as a central processing unit (CPU), a memory 102 serving as a storage device, a non-volatile storage device 103 such as a solid-state drive (SSD), and an interface 104. The memory 102 is, for example, a semiconductor memory such as a random access memory (RAM). Each function of the test support device 10 illustrated in FIG. 6 can be realized by a processing circuit. The processing circuit may be dedicated hardware, or may be a processor 101 that executes a program (including a program according to an embodiment) stored in the memory 102. The processing circuit may be hardware, software, firmware, or a combination of these. The program may be provided by downloading via a network or by a storage medium such as an optical disc on which the program is stored.

[0027] The information presentation unit 30 may be, for example, a display device that presents video. The information presentation unit 30 may include an audio output unit (such as a speaker) that presents sound information. The information presentation unit 30 may include a lamp and a drive circuit that turns the lamp on, off, or blinks. The information presentation unit 30 may include a vibration generator that notifies the worker of improper grip vibration by vibrating the worker's body or hand. The information presentation unit 30 may be a laptop computer, tablet terminal, or smartphone that can communicate with the inspection support device 10. The information presentation unit 30 may also include smart glasses or a smartphone that utilizes augmented reality (AR) to display virtual images overlaid on the real world to indicate the grip position and vibration direction. In this case, it is possible to reduce even slight deviations from the grip vibration method of a skilled worker.

[0028] 7 is a functional block diagram illustrating a schematic configuration of the sensor data collection device 10a. The sensor data collection device 10a is a computer that collects data for creating a criterion for determining inappropriate grip vibration, which is pre-stored in the storage device 41 of the inspection support device 10, and a criterion for determining appropriate grip vibration, which is pre-stored in the storage device 42. The sensor data collection device 10a collects criterion data using the same pressure sensor 21 and acceleration sensor 22 (i.e., physically the same or having the same functions) as the pressure sensor 21 and acceleration sensor 22 connected to the inspection support device 10. The sensor data collection device 10a may be the same computer as the inspection support device 10. In other words, the computer serving as the inspection support device 10 may have the function of collecting and creating criterion data.

[0029] An appropriate determination criterion is created, for example, based on the pressure data and acceleration data acquired when a skilled worker grips and applies vibration to a plurality of objects in turn.

[0030] The inappropriateness determination criteria are created based on the pressure data and acceleration data acquired at the time, for example, when an operator intentionally adopts an inappropriate grip and applies vibration, when an operator intentionally adopts an inappropriate grip and applies inappropriate vibration, or when an operator adopts an appropriate grip and applies inappropriate vibration intentionally.

[0031] The sensor data collecting unit 15 stores the pressure data serving as a criterion for determining inappropriateness as first reference pressure data R1p and the acceleration data as first reference acceleration data R1a in the storage device 41. The sensor data collecting unit 15 may include a storage device 16 that stores label information indicating the cause of the inappropriate grip vibration and countermeasures (instructions to the worker) in advance. In this case, the sensor data collecting unit 15 may classify and store each of the first reference pressure data R1p and the first reference acceleration data R1a into categories corresponding to the causes of the inappropriate grip vibration method. The sensor data collecting unit 15 may also select and assign label information for each classified category. Alternatively, label information (e.g., text data representing presentation information) may be assigned to each of the first reference pressure data R1p and each of the first reference acceleration data R1a, rather than to each category.

[0032] The sensor data collecting unit 15 also stores the pressure data serving as a criterion for determining inappropriateness as first reference pressure data R1p and the acceleration data as first reference acceleration data R1a in the storage device 41. The sensor data collecting unit 15 may include a storage device 16 that stores label information indicating the cause of the inappropriate grip vibration and countermeasures (instructions to the worker) in advance. In this case, the sensor data collecting unit 15 may classify and store each of the first reference pressure data R1p and the first reference acceleration data R1a into categories corresponding to the causes of the inappropriate grip vibration method. The sensor data collecting unit 15 may also select and assign label information for each classified category. Alternatively, label information may be assigned to each of the first reference pressure data R1p and each of the first reference acceleration data R1a, rather than to each category.

[0033] FIG. 8 is a diagram illustrating an example of the hardware configuration of the sensor data collecting device 10a. As illustrated in FIG. 8, the sensor data collecting device 10a includes a processor 101a such as a CPU, a memory 102a serving as a storage device, a non-volatile storage device 103a such as an SSD, and an interface 104a. Each function of the sensor data collecting device 10a illustrated in FIG. 8 can be realized by a processing circuit. The processing circuit may be dedicated hardware, or may be a processor 101a that executes a program stored in the memory 102a. The processing circuit may realize each of the above-described functions by hardware, software, firmware, or any combination thereof.

[0034] 9 is a flowchart showing the process of collecting sensor data during appropriate grip vibration application. The sensor data during appropriate grip vibration application (or the judgment reference data created based on the sensor data) that is stored in the storage device 42 before the actual inspection is collected and stored as follows: A skilled worker wears the sensor glove 20 connected to the sensor data collection device 10a shown in FIGS. 7 and 8, applies appropriate grip vibration application to each object, and stores the pressure data and acceleration data obtained at that time as second reference pressure data R2p and second reference acceleration data R2a in the storage device 42 (steps S11 to S13).

[0035] FIG. 10 is a flowchart showing the process of collecting sensor data during improper grip vibration application. The sensor data (or criteria data created based on the sensor data) during improper grip vibration application, which is stored in the storage device 41 before the actual inspection, is collected and stored as follows: An operator wears the sensor glove 20 connected to the sensor data collection device 10a shown in FIGS. 7 and 8 , intentionally applies improper grip vibration application to each object, and the resulting pressure and acceleration data are stored in the storage device 42 as first reference pressure data R1p and first reference acceleration data R1a (steps S21 to S24). In step S23, the sensor data collection device 10a may also perform labeling to assign a label to each piece of first reference pressure data R1p, each piece of first reference acceleration data R1a, or each category into which these data are classified. The label information is output as presentation information to the information presentation unit 30.

[0036] 11 is a flowchart showing the operation of the test support device 10 and the test support system 1 according to the embodiment. In this case, the similarity evaluation unit 12 calculates a first pressure similarity S1p, which is the similarity between the pressure data output from the pressure sensor 21 and the first reference pressure data (inappropriate reference pressure data) R1p, and calculates a first acceleration similarity S1a, which is the similarity between the acceleration data output from the acceleration sensor 22 and the first reference acceleration data (inappropriate reference acceleration data) R1a (steps S31 and S32). Each of the first pressure similarity S1p and the first acceleration similarity S1a is also referred to as a first similarity.

[0037] Next, the similarity evaluation unit 12 calculates a second pressure similarity S2p, which is the similarity between the pressure data output from the pressure sensor 21 and the second reference pressure data (appropriate reference pressure data) R2p, and calculates a second acceleration similarity S2a, which is the similarity between the acceleration data output from the acceleration sensor 22 and the second reference acceleration data (appropriate reference acceleration data) R2a, in the actual inspection (step S33). Each of the second pressure similarity S2p and the second acceleration similarity S2a is also referred to as a second similarity.

[0038] The grip vibration determination unit 13 determines that the grip vibration method is inappropriate if at least one of the conditions that the first pressure similarity S1p is greater than or equal to the second pressure similarity S2p and the first acceleration similarity S1a is greater than or equal to the second acceleration similarity S2a is met (step S34).

[0039] If the determination in step S34 is NO, the grip vibration is appropriate, and the presentation information generation unit 14 outputs the determination result indicating that the grip vibration is appropriate as presentation information, and ends the inspection (steps S37, S38).

[0040] If the determination in step S34 is YES, the grip vibration is inappropriate, and the presentation information generator 14 outputs a determination result indicating that the grip vibration is inappropriate as presentation information and instructs the worker to perform a re-examination (steps S35 and S36). At this time, the presentation information generator 14 may output instruction information regarding the grip vibration to the worker based on the label information.

[0041] 12 is a flowchart showing another operation of the test support device 10 and the test support system 1 according to the embodiment. In this case, the similarity evaluation unit 12 calculates a first pressure similarity S1p, which is the similarity between the pressure data output from the pressure sensor 21 and the first reference pressure data (inappropriate reference pressure data) R1p, and calculates a first acceleration similarity S1a, which is the similarity between the acceleration data output from the acceleration sensor 22 and the first reference acceleration data (inappropriate reference acceleration data) R1a (steps S41 and S42). Each of the first pressure similarity S1p and the first acceleration similarity S1a is also referred to as a first similarity.

[0042] The grip vibration determination unit 13 determines that the grip vibration method is inappropriate when at least one of the conditions that the first pressure similarity S1p is equal to or greater than the first pressure threshold T1p and the first acceleration similarity S1a is equal to or greater than the first acceleration threshold T1a is satisfied (step S43). Each of the first pressure threshold T1p and the first acceleration threshold T1a is also referred to as a first threshold.

[0043] If the determination in step S43 is YES, the grip vibration is inappropriate, and therefore the presentation information generator 14 outputs a determination result indicating that the grip vibration is inappropriate as presentation information and instructs the worker to perform a re-examination (steps S44 and S45). At this time, the presentation information generator 14 may output instruction information regarding the grip vibration to the worker based on the label information.

[0044] If the determination in step S43 is NO, the similarity evaluation unit 12 calculates a second pressure similarity S2p, which is the similarity between the pressure data output from the pressure sensor 21 and the second reference pressure data (appropriate reference pressure data) R2p, and calculates a second acceleration similarity S2a, which is the similarity between the acceleration data output from the acceleration sensor 22 and the second reference acceleration data (appropriate reference acceleration data) R2a (step S46). Each of the second pressure similarity S2p and the second acceleration similarity S2a is also referred to as a second similarity.

[0045] The grip vibration determination unit 13 determines that the grip vibration method is inappropriate if at least one of the conditions that the second pressure similarity S2p is greater than or equal to the second pressure threshold T2p and the second acceleration similarity S2a is greater than or equal to the second acceleration similarity S2a is not satisfied (step S47).

[0046] If the determination in step S47 is NO, the grip vibration is inappropriate, and therefore the presentation information generator 14 outputs the determination result indicating that the grip vibration is inappropriate as presentation information and instructs the worker to perform a re-examination (steps S44 and S45). At this time, the presentation information generator 14 may output instruction information regarding the grip vibration to the worker based on the label information.

[0047] If the determination in step S47 is YES, the grip vibration is appropriate, and the presentation information generation unit 14 outputs the determination result indicating that the grip vibration is appropriate as presentation information, and ends the inspection (steps S48, S49).

[0048] 13A and 13B are explanatory diagrams showing examples of methods for calculating similarity. FIG. 13A shows vectorization processing for obtaining time-series data by sampling from a pressure signal waveform (or acceleration signal waveform). When the pressure signal (or acceleration signal) data and the judgment reference data are expressed as vector data, Euclidean distance can be used as the similarity. Furthermore, when the pressure signal (or acceleration signal) data and the judgment reference data are expressed as vector data, a correlation coefficient that can be obtained from a scatter plot can be used as the similarity. FIG. 13A shows an example of calculating distance on a waveform (solid line) whose time length is adjusted by dynamic time warping (DTW) from the pressure signal waveform (or acceleration signal waveform) shown by the dashed line.

[0049] 14(A) and (B) are explanatory diagrams showing an example of a method for determining grip vibration based on the absolute value of the pressure difference. Fig. 14(A) shows a case where the absolute value of the pressure difference does not exceed a predetermined determination threshold, in which case it is determined that grip vibration is inappropriate. Fig. 14(B) shows a case where the absolute value of the pressure difference exceeds a predetermined determination threshold, in which case it is determined that grip vibration is appropriate. Note that the method for determining similarity is not limited to the above example, and other known similarity measures may also be used.

[0050] As described above, according to this embodiment, it is possible to know on-site whether the grip vibration applied by the operator's hand is appropriate. Therefore, if the grip vibration is inappropriate, the operator can apply the grip vibration again on-site and re-inspect the object, eliminating the need to leave the site and then return to it for re-inspection. Furthermore, because the grip vibration can be applied appropriately, the accuracy of the inspection of the fixation state of the object 50 can be improved.

[0051] In addition, when the pressure data (or acceleration data) acquired in an actual inspection is similar to multiple inappropriate judgment criteria data (or similar to multiple categories of judgment criteria data), the inspection support device 10 can set the inappropriate factors of the gripping vibration method in natural language.

[0052] In addition, when the pressure data (or acceleration data) is similar to multiple categories of inappropriate judgment criteria data, the inspection support device 10 may individually calculate the similarity with each inappropriate category, extract categories having a similarity equal to or greater than a predetermined threshold, and use the extracted categories as presentation information.

[0053] In addition, when the pressure data (or acceleration data) is similar to multiple categories of inappropriate judgment criteria data, and the presentation information for each individual inappropriate category contains information with opposite meanings (e.g., opposite concepts or antonyms), the presentation information generation unit 14 of the inspection support device 10 may preferentially display the presentation information for the category with the higher similarity (i.e., delete the presentation information for the category with the lower similarity).

[0054] 1 Inspection support system, 10 Inspection support device, 11 Memory device, 12 Similarity evaluation unit, 13 Grasping vibration determination unit, 14 Presentation information generation unit, 20 Sensor glove, 21 Pressure sensor, 22 Acceleration sensor, 30 Information presentation unit, 41, 42 Memory device, 50 Object, R1p First reference pressure data, R1a First reference acceleration data, R2p Second reference pressure data, R2a Second reference acceleration data, S1p First pressure similarity, S1a First acceleration similarity, S2p Second pressure similarity, S2a Second acceleration similarity.

Claims

1. An inspection support device which prepares, as criteria data acquired and stored by gripping and vibrating an object with a hand having a pressure sensor and an acceleration sensor attached thereto, first reference pressure data and first reference acceleration data acquired from the pressure sensor and the acceleration sensor when an inappropriate gripping and vibration is intentionally performed on the object with the hand having the pressure sensor and the acceleration sensor attached thereto, and second reference pressure data and second reference acceleration data acquired from the pressure sensor and the acceleration sensor when an appropriate gripping and vibration is performed on the object with the hand having the pressure sensor and the acceleration sensor attached thereto, and outputs information regarding a gripping and vibration method for the object using the criteria data when an operator grips and vibrates the object in an actual inspection, an acceleration similarity evaluation unit that calculates, in the actual inspection, a first pressure similarity which is a similarity between the pressure data output from the pressure sensor and the first reference pressure data, a first acceleration similarity which is a similarity between the acceleration data output from the acceleration sensor and the first reference acceleration data, a second pressure similarity which is a similarity between the pressure data output from the pressure sensor and the second reference pressure data, and a second acceleration similarity which is a similarity between the acceleration data output from the acceleration sensor and the second reference acceleration data; a grip vibration determination unit that makes a determination regarding the grip vibration method based on the first pressure similarity, the first acceleration similarity, the second pressure similarity, and the second acceleration similarity; and a presentation information generation unit that generates presentation information regarding the grip vibration method based on a result of the determination.

2. The inspection support device described in claim 1, characterized in that the grip vibration judgment unit judges that the grip vibration method is inappropriate when at least one of the conditions that the first pressure similarity is greater than or equal to the second pressure similarity and that the first acceleration similarity is greater than or equal to the second acceleration similarity is satisfied.

3. The inspection support device described in claim 1, characterized in that the grip vibration judgment unit judges whether the grip vibration method is appropriate or inappropriate based on whether the first pressure similarity is equal to or greater than a predetermined first pressure threshold, whether the first acceleration similarity is equal to or greater than a predetermined first acceleration threshold, whether the second pressure similarity is equal to or greater than a predetermined second pressure threshold, and whether the second acceleration similarity is equal to or greater than a predetermined second acceleration threshold.

4. The inspection support device described in claim 1, characterized in that the grip vibration judgment unit judges that the grip vibration method is inappropriate when at least one of the conditions that the first pressure similarity is equal to or greater than a predetermined first pressure threshold and that the first acceleration similarity is equal to or greater than a predetermined first acceleration threshold is satisfied.

5. An inspection support device as described in any one of claims 1 to 4, characterized in that each of the first reference pressure data and the first reference acceleration data is classified into a category based on the cause of inappropriateness of the gripping vibration method and is assigned a label that is information indicating the cause.

6. The examination support device according to claim 5, characterized in that, when the presentation information for each individual inappropriate category contains information with contradictory meanings, the presentation information generation unit preferentially displays the presentation information for the category with a higher similarity.

7. An inspection support system comprising: the inspection support device according to any one of claims 1 to 6; the pressure sensor; and the acceleration sensor.

8. An inspection support method executed by an inspection support device, comprising: preparing, as judgment reference data acquired and stored by grasping and vibrating an object with a hand having a pressure sensor and an acceleration sensor attached thereto, first reference pressure data and first reference acceleration data acquired from the pressure sensor and the acceleration sensor when an inappropriate grasping and vibration is intentionally performed on the object with the hand having the pressure sensor and the acceleration sensor attached thereto; and second reference pressure data and second reference acceleration data acquired from the pressure sensor and the acceleration sensor when an appropriate grasping and vibration is performed on the object with the hand having the pressure sensor and the acceleration sensor attached thereto; and outputting information regarding a method of grasping and vibrating the object using the judgment reference data when an operator grasps and vibrates the object in an actual inspection, the method comprising:

13. An inspection support method comprising: a step of calculating, in the actual inspection, a first pressure similarity which is a similarity between the pressure data output from the pressure sensor and the first reference pressure data, a first acceleration similarity which is a similarity between the acceleration data output from the acceleration sensor and the first reference acceleration data, a second pressure similarity which is a similarity between the pressure data output from the pressure sensor and the second reference pressure data, and a second acceleration similarity which is a similarity between the acceleration data output from the acceleration sensor and the second reference acceleration data; a step of making a determination regarding the grip vibration method based on the first pressure similarity, the first acceleration similarity, the second pressure similarity, and the second acceleration similarity; and a step of generating presentation information regarding the grip vibration method based on a result of the determination.

9. A computer that is an inspection support device that prepares, as criteria data acquired and stored by gripping and vibrating an object with a hand having a pressure sensor and an acceleration sensor attached thereto, first reference pressure data and first reference acceleration data acquired from the pressure sensor and the acceleration sensor when an inappropriate gripping and vibrating operation is intentionally performed on the object with the hand having the pressure sensor and the acceleration sensor attached thereto, and second reference pressure data and second reference acceleration data acquired from the pressure sensor and the acceleration sensor when an appropriate gripping and vibrating operation is performed on the object with the hand having the pressure sensor and the acceleration sensor attached thereto, and outputs information regarding a gripping and vibrating method for the object using the criteria data when an operator grips and vibrates the object in an actual inspection.

23. The method of claim 22, further comprising: calculating a first pressure similarity which is a similarity between pressure data output from the pressure sensor and the first reference pressure data, calculating a first acceleration similarity which is a similarity between acceleration data output from the acceleration sensor and the first reference acceleration data, calculating a second pressure similarity which is a similarity between pressure data output from the pressure sensor and the second reference pressure data, and calculating a second acceleration similarity which is a similarity between acceleration data output from the acceleration sensor and the second reference acceleration data, in the actual inspection; making a determination regarding the grip vibration method based on the first pressure similarity, the first acceleration similarity, the second pressure similarity, and the second acceleration similarity; and generating presentation information regarding the grip vibration method based on a result of the determination.

Images