Mating detection and judgment system and method

Abstract

To make it possible to perform connector mating determination more efficiently.SOLUTION: A mating detection determination system includes: a first sensor that is attached to a hand or an arm of a worker who performs mating work of mating a pair of connectors and detects vibration; a second sensor that collects sound generated from the pair of connectors; and a terminal device that can communicate with each other between the first sensor and the second sensor, and determines whether the connectors are mated using a first vibration waveform based on the vibration and a second vibration waveform based on the sound. When the terminal device determines that a predetermined vibration waveform is detected from the first vibration waveform, the terminal device starts determining whether the connectors are mated.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a mating detection and determination system and a mating detection and determination method. [Background technology]

[0002] Patent Document 1 discloses a connector fitting state confirmation device that detects vibrations that occur when a connector is fitted, converts the detected connector vibrations into vibration waveform data, generates judgment data by removing data necessary for determining the connector's fitting state from the vibration waveform data, and compares the generated judgment data with pre-stored reference data to determine whether the connector's fitting state is good or bad. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-221971 A Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure has been devised in view of the above-mentioned conventional circumstances, and has an object to provide a mating detection and determination system and a mating detection and determination method that can more efficiently perform mating determination of a connector. [Means for solving the problem]

[0005] The present disclosure provides a terminal device that is attached to the hand or arm of a worker performing a fitting operation of fitting a pair of connectors, the terminal device comprising a first sensor that detects vibrations, a second sensor that picks up sounds generated from the pair of connectors, and a terminal device that is capable of communicating with the first sensor and the second sensor and that determines whether the connectors are fitted using a first vibration waveform based on the vibrations and a second vibration waveform based on the sounds, the terminal device determining whether the connectors are fitted from the first vibration waveform. Before mating, which occurs when the pair of connectors come into contact with each otherWhen it is determined that a vibration waveform is detected, a determination is made as to whether the connector is mated or not. and determining whether or not the connectors are mated based on a mating vibration waveform generated when the pair of connectors are mated, which is detected from the first vibration waveform after a timing when it is determined that the pre-mating vibration waveform has been detected from the first vibration waveform, and a mating vibration waveform generated when the pair of connectors are mated, which is detected from the second vibration waveform. The present invention provides a fit detection and determination system.

[0006] The present disclosure also provides a mating detection and determination method performed by a terminal device for determining whether a pair of connectors are mated, the method comprising: detecting vibrations with a first sensor attached to a hand or arm of a worker performing a mating operation of mating the pair of connectors; collecting sounds generated from the pair of connectors with a second sensor; and calculating a first vibration waveform based on the vibrations. Before mating, which occurs when the pair of connectors come into contact with each other When it is determined that a vibration waveform is detected, a determination is made as to whether the connector is mated or not. and determining whether or not the connectors are mated based on a mating vibration waveform generated when the pair of connectors are mated, which is detected from the first vibration waveform after the timing at which it is determined that the pre-mating vibration waveform has been detected from the first vibration waveform, and a mating vibration waveform generated when the pair of connectors are mated, which is detected from a second vibration waveform based on the sound. The present invention provides a method for detecting and determining a fit.

[0007] The present disclosure also provides a first sensor that is attached to the hand or arm of a worker performing a fitting operation of fitting a pair of connectors and detects vibrations; a second sensor that picks up sounds generated from the pair of connectors; and a terminal device that is capable of communicating with the first sensor and the second sensor and that determines whether the connectors are fitted using a first vibration waveform based on the vibration and a second vibration waveform based on the sound, wherein the terminal device starts determining whether the connectors are fitted when it determines that a predetermined vibration waveform has been detected from the first vibration waveform, measures a first detection time at which a fitting vibration waveform generated when the pair of connectors are fitted is detected from the first vibration waveform and a second detection time at which the fitting vibration waveform is detected from the second vibration waveform, and determines that the connectors are fitted when it determines that a time difference between the first detection time and the second detection time is within a predetermined time. to provide.

[0008] The present disclosure also provides A mating detection and determination method performed by a terminal device that determines whether a pair of connectors are mated, the method comprising: detecting vibrations with a first sensor attached to a hand or arm of a worker performing a mating operation of mating the pair of connectors; collecting sound generated from the pair of connectors with a second sensor; and, when it is determined that a predetermined vibration waveform is detected from a first vibration waveform based on the vibration, starting a determination of whether the connectors are mated; measuring a first detection time at which a mating vibration waveform generated when the pair of connectors are mated is detected from the first vibration waveform and a second detection time at which the mating vibration waveform is detected from a second vibration waveform based on the sound; and, when it is determined that a time difference between the first detection time and the second detection time is within a predetermined time, determining that the connectors are mated. to provide. Effect of the Invention

[0009] According to the present disclosure, connector mating determination can be performed more efficiently. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing an example of the overall configuration of a fit detection and determination system according to a first embodiment. [Diagram 2] Diagram explaining the connector mating operation [Diagram 3] A flowchart showing an example of an operation procedure of a terminal device according to the first embodiment. [Figure 4] FIG. 13 is a diagram showing an example of a fitting sound waveform and a vibration waveform. [Diagram 5] Enlarged view of mating sound and vibration waveforms before and after the start of mating judgment [Figure 6] FIG. 11 is a block diagram showing an example of the overall configuration of a fitting detection and determination system according to a second embodiment. [Figure 7]A flowchart showing an example of an operation procedure of a terminal device according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] (Background to this disclosure) Connectors used in vehicles have a pair of mechanisms (e.g., a male connector and a female connector) that can electrically connect the various electrical systems within the vehicle by mating with each other. In recent years, with the electronicization of devices mounted on vehicles, the number of connectors used in the manufacture of each vehicle has increased, and there is a demand for more efficient connector mating operations.

[0012] After such connectors are mated by an operator, the operator checks whether the connectors are properly mated by checking for the presence or absence of a mating sound that is emitted when the connectors are mated, by visually checking, etc. However, even though the locking mechanism (mechanism for maintaining the mated state) of the connector is in the locked state, there are cases where the connectors are not properly mated, that is, in a semi-mated state, and therefore the operator has to perform the mating check multiple times, which is very time-consuming.

[0013] Conventionally, as a method for determining whether a connector is properly mated (hereinafter referred to as a "mating determination method"), there is a method in which the sound (mating sound) of the connector being mated is picked up by a microphone, and whether the connector is properly mated is determined based on the mating sound that is picked up. However, the connector mating operation, in which the vibrations of the connector being mated are obtained by an acceleration sensor attached to the worker's hand, may detect various noises (vibrations) generated on the production line, which may reduce the accuracy of the mating state determination.

[0014] Another method for judging whether a connector is properly mated involves acquiring vibrations when the connector is mated, and comparing judgment data generated using the vibrations with pre-stored reference data to judge whether the connector is properly mated (Patent Document 1).In such cases, the acceleration sensor attached to the worker's hand detects various vibrations, such as when the worker's hand comes into contact with surrounding jigs, equipment, products, etc., and therefore there is a problem in that it is difficult to detect only the vibrations when the connector is mated.

[0015] Hereinafter, with reference to the drawings as appropriate, each embodiment that specifically discloses the configuration and operation of the mating detection and determination system and mating detection and determination method according to the present disclosure will be described in detail. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters and duplicate explanation of substantially the same configuration may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the attached drawings and the following explanation are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0016] (Embodiment 1) The overall configuration of a fit detection and determination system 100 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the overall configuration of the fit detection and determination system 100 according to the first embodiment.

[0017] The mating detection and determination system 100 is a system for determining whether or not a pair of a male connector 13A and a female connector 13B are mated by a mating operation. The mating detection and determination system 100 includes a vibration sensor 11, a transmitter 12, a microphone 14, and a terminal device P1.

[0018] Here, the term "fitting" means that a pair of mechanical parts are at least physically in contact with each other and fixed. The term "connector" means a part used to physically or electrically connect devices mounted on a vehicle or the like, and includes a connection part using a mechanism such as a latch. Specifically, the connector in the first and second embodiments may include mechanical parts such as a door opening and closing mechanism and a general locking mechanism. Note that, as an example, the fitting detection and determination system 100, 100A according to the first and second embodiments described below will be described as a case in which, in a fitting operation between a male connector 13A and a female connector 13B, the pair of connectors are determined to be properly fitted together. Note that proper fitting refers to a state in which the pair of connectors are fitted together so as to be electrically connectable, and the mechanical parts are locked so as not to fall off due to vibration or the like.

[0019] The vibration sensor 11, which is an example of a sensor and a first sensor, is a piezoelectric sensor that converts a force applied to a piezoelectric body into a voltage, and is connected to the transmitter 12 so as to be capable of wired communication. The vibration sensor 11 is attached to one of the hands, fingers, arms, etc. of a worker who performs the fitting operation of the connector 13, and detects vibrations when the connector 13 comes into contact, vibrations when the connector is fitted, etc. The vibration sensor 11 converts the detected vibrations into a voltage using the piezoelectric sensor, generates vibration data based on the converted voltage, and transmits it to the transmitter 12.

[0020] The vibration data referred to here is data indicating a time-series change in voltage value obtained by conversion by the piezoelectric sensor, and is an amplitude waveform of vibration (vibration waveform).

[0021] The transmitter 12 is connected to the vibration sensor 11 so as to be able to communicate by wire, and is connected to the terminal device P1 so as to be able to communicate by wireless. The transmitter 12 acquires vibration data transmitted from the vibration sensor 11 and transmits it to the terminal device P1. Note that the wireless communication referred to here is, for example, short-range wireless communication such as Bluetooth (registered trademark) or NFC (registered trademark), or communication via a wireless LAN (Local Area Network) such as Wi-Fi (registered trademark).

[0022] Connector 13 is composed of a pair of male connector 13A (see FIG. 2) and female connector 13B (see FIG. 2). When male connector 13A and female connector 13B are mated by an operator, a predetermined mating sound is generated during mating.

[0023] Microphone 14, as an example of a second sensor, is connected to terminal device P1 so as to be capable of wired or wireless communication, picks up sounds such as the mating sound when male connector 13A and female connector 13B are mated, converts the picked up sounds into audio signals, and transmits them to terminal device P1.

[0024] The terminal device P1 is connected to the transmitter 12 so as to be able to communicate wirelessly, and is connected to the microphone 14 so as to be able to communicate by wire or wirelessly. The terminal device P1 determines the timing to start a fitting determination of the connector 13 based on the vibration data transmitted from the transmitter 12. The terminal device P1 also determines whether the connector 13 is properly fitted based on the vibration data transmitted from the transmitter 12 and the audio signal transmitted from the microphone 14, and outputs the fitting determination result. The terminal device P1 includes a communication unit 20, a processor 21, a memory 22, and a database DB1.

[0025] The communication unit 20 is connected to the transmitter 12 so as to be able to communicate wirelessly, and is connected to the microphone 14 so as to be able to communicate by wire or wirelessly, and executes transmission and reception of data. The communication unit 20 outputs the vibration data transmitted from the transmitter 12 and the audio signal transmitted from the microphone 14 to the processor 21.

[0026] The processor 21 is configured using, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor) or an FPGA (Field Programmable Gate Array), and controls the operation of each part of the terminal device P1. The processor 21 performs various processes and controls in cooperation with the memory 22. Specifically, the processor 21 refers to the programs and data stored in the memory 22 and executes the programs to realize the functions of each part. Note that the parts referred to here are the pre-fitting vibration determination part 21A and the fitting determination part 21B.

[0027] Based on the vibration data (vibration waveform) transmitted from the transmitter 12, the pre-mating vibration determination unit 21A determines whether or not a vibration corresponding to a contact vibration when the tip ends of the male connector 13A and the female connector 13B come into contact, or a friction vibration from the contact to the completion of mating, has been detected. In the following description, the contact vibration, friction vibration, and other vibrations that occur until the completion of mating between the male connector 13A and the female connector 13B described above are referred to as "pre-mating vibrations." When the pre-mating vibration determination unit 21A analyzes the vibration data and determines that a pre-mating vibration waveform corresponding to the pre-mating vibration has been detected, it generates a control command requesting the start of mating determination and outputs it to the mating determination unit 21B.

[0028] Based on the control command output from the pre-fitting vibration determination unit 21A, the fitting determination unit 21B starts a fitting determination process from the timing when the pre-fitting vibration determination unit 21A determines that a pre-fitting vibration waveform has been detected. The fitting determination unit 21B executes a first fitting determination process based on the vibration data (vibration waveform) transmitted from the transmitter 12, and a second fitting determination process based on the sound signal (fitting sound waveform) transmitted from the microphone 14. Based on the results of the first and second fitting determination processes, the fitting determination unit 21B executes a fitting determination of the connector 13 and outputs a fitting determination result.

[0029] The memory 22 has, for example, a RAM (Random Access Memory) as a work memory used when executing each process of the processor 21, and a ROM (Read Only Memory) that stores programs and data that define the operation of the processor 21. The RAM temporarily stores data or information generated or acquired by the processor 21. The ROM has written therein programs that define the operation of the processor 21. The memory 22 records various data used in the first and second fitting determination processes (fitting vibration waveform data, threshold values, etc. used to determine pre-fitting vibration of the connector), learned data, etc.

[0030] The database DB1 is configured using a storage device such as a hard disk drive (HDD) or a solid state drive (SSD), etc. The database DB1 records the vibration data transmitted from the transmitter 12, the audio signal transmitted from the microphone 14, etc. in association with each other.

[0031] The mating operation of connector 13 will be described with reference to Fig. 2. Fig. 2 is a diagram illustrating the mating operation of connector 13. It goes without saying that the structure of connector 13 shown in Fig. 2 is an example and is not limited to this.

[0032] The connector 13 in the first and second embodiments may have a structure in which a pair of connectors having a fitting structure, such as male connector 13A and female connector 13B, can be fitted together to electrically connect electrical devices that are the connection sources of the pair of connectors to each other. The locking mechanisms for maintaining the fitted state of male connector 13A and female connector 13B are merely examples and are not intended to be limiting. The locking mechanism may be, for example, a mechanism having a recess and a protrusion.

[0033] Convex portion 135A at the tip of male connector 13A fits into concave portion 132B of female connector 13B, thereby electrically connecting an electrical device (not shown) to which female connector 13B is connected and an electrical device (not shown) to which male connector 13A is connected. Male connector 13A has a locking mechanism 131A for maintaining a mated state with female connector 13B.

[0034] The lock mechanism 131A is integrally formed of a latch protrusion 132A, a latch beam 133A, and a latch operation part 134A. The latch protrusion 132A engages with a protrusion 131B of the female connector 13B to maintain the mated state between the male connector 13A and the female connector 13B. The latch beam 133A has a latch protrusion 132A on one side and a latch operation part 134A on the other side, and connects the latch protrusion 132A and the latch operation part 134A. When the latch operation part 134A is pressed down by an operator, it lifts the latch protrusion 132A located on the opposite side through the latch beam 133A so that it is positioned above the protrusion 131B (upper side of the paper), making it possible to engage or release the latch protrusion 132A and the protrusion 131B.

[0035] Female connector 13B electrically connects an electrical device (not shown) to which male connector 13A is connected and an electrical device (not shown) to which female connector 13B is connected, by fitting recess 132B at the tip end into protrusion 135A of male connector 13A. Female connector 13B has protrusion 131B for maintaining a fitted state with male connector 13A.

[0036] Protrusion 131B is an engagement protrusion for retaining the mated state of male connector 13A and female connector 13B by engaging with latch protrusion 132A.

[0037] The fitting operation performed by the worker will now be described.

[0038] First, the worker picks up the pair of male connector 13A and female connector 13B to be fitted together, and brings the tip portions (i.e., convex portion 135A and concave portion 132B) of the pair of male connector 13A and female connector 13B into contact with each other. Note that in this contact state, the pair of male connector 13A and female connector 13B do not fit together because latch protrusion 132A of male connector 13A interferes with protrusion 131B of female connector 13B.

[0039] Next, the worker presses down latch operation portion 134A so that latch protrusion 132A is raised to a position where it does not interfere with protrusion 131B (i.e., above protrusion 131B), and while keeping latch operation portion 134A pressed down, moves male connector 13A and female connector 13B closer together. As a result, the worker moves latch operation portion 134A over protrusion 131B, and brings convex portion 135A of male connector 13A into a recessed portion 132B of female connector 13B into an engaged state. When the worker finishes pressing down latch operation portion 134A, latch protrusion 132A is engaged with protrusion 131B at a position where convex portion 135A engages with recessed portion 132B, and the engaged state of male connector 13A and female connector 13B is maintained.

[0040] An operation procedure of the terminal device P1 in the first embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of an operation procedure of the terminal device P1 in the first embodiment.

[0041] The terminal device P1 acquires vibration data acquired by the vibration sensor 11 and transmitted from the transmitter 12 (St10).

[0042] The terminal device P1 analyzes the acquired vibration data and determines whether or not a contact vibration when the operator brings the tip portions (i.e., convex portion 135A and concave portion 132B) of the pair of male connector 13A and female connector 13B into contact with each other, or a pre-mating vibration waveform (e.g., pre-mating vibration waveform WV21 shown in Figure 4) indicating frictional vibration caused by friction between convex portion 135A of male connector 13A and concave portion 132B of female connector 13B when mating of the pair of male connector 13A and female connector 13B is started has been detected (St11).

[0043] When the terminal device P1 determines in the process of step St11 that the pre-fitting vibration waveform is detected (St11, YES), the terminal device P1 proceeds to the process of steps St12A and St12B and starts the first fitting determination process and the second fitting determination process. The terminal device P1 continues to acquire the vibration data (i.e., the vibration data of the fitting vibration) acquired by the vibration sensor 11 and transmitted from the transmitter 12 (St12A). In addition, the terminal device P1 starts acquiring the audio signal (i.e., the audio signal based on the fitting sound of the connector 13) transmitted from the microphone 14 (St12B).

[0044] On the other hand, if the terminal device P1 determines in the processing of step St11 that the pre-fitting vibration waveform is not detected (St11, NO), it proceeds to the processing of step St10 and again acquires the vibration data transmitted from the transmitter 12 (St10).

[0045] As a first fitting determination process, the terminal device P1 analyzes the vibration data and detects a fitting vibration waveform (for example, the fitting vibration waveform WV22 shown in FIG. 4 and FIG. 5) when the male connector 13A and the female connector 13B are fitted together (St13A).

[0046] As a second mating determination process, the terminal device P1 analyzes the audio signal and detects a mating vibration waveform (for example, the mating vibration waveform WV11 shown in Figures 4 and 5) corresponding to the mating sound when the male connector 13A and the female connector 13B are mated (St13B).

[0047] The terminal device P1 executes a comprehensive fitting judgment based on the judgment result of the first fitting judgment process and the judgment result of the second fitting judgment process (St14). The terminal device P1 judges whether or not a fitting vibration waveform is detected in each of the first fitting judgment process and the second fitting judgment process. The terminal device P1 also compares the time when the fitting vibration waveform is detected in the first fitting judgment process with the time when the fitting vibration waveform is detected in the second fitting judgment process, and if it judges that the time difference between them is within a predetermined time (e.g., 0.5 seconds, 1 second), it judges that the connector 13 is normally fitted.

[0048] When the terminal device P1 determines, as a result of the overall fitting judgment, that a fitting vibration waveform was detected in both the first fitting judgment process and the second fitting judgment process, and that the time difference between them is within a predetermined time, it generates and outputs a notice that the connector 13 is normally fitted (St15). On the other hand, when the terminal device P1 determines, as a result of the overall fitting judgment, that a fitting vibration waveform was not detected in either the first fitting judgment process or the second fitting judgment process, or when it determines that the time difference between them is not within a predetermined time, it generates and outputs a notice that the connector 13 is not normally fitted (St15).

[0049] As described above, the terminal device P1 in the first embodiment performs the mating judgment (i.e., the first mating judgment and the second mating judgment) from the timing when the pre-mating vibration waveform is detected, thereby more effectively suppressing the mating judgment of the connector 13 based on the environmental sounds around the worker and the vibration of the worker's hand due to other factors (for example, contact between the worker's hand and surrounding obstacles, etc.), and improving the accuracy of the mating judgment of the connector 13. This allows the terminal device P1 to support the worker in more efficiently performing the mating judgment of the connector 13.

[0050] The fitting judgment of the connector 13 executed by the terminal device P1 will be described with reference to Fig. 4 and Fig. 5. Fig. 4 is a diagram showing an example of a fitting sound waveform WV1 and a vibration waveform WV2. Fig. 5 is an enlarged view of the fitting sound waveform WV1 and the vibration waveform WV2 before and after the start of the fitting judgment.

[0051] The fitting sound waveform WV1 is generated by the terminal device P1 using a voice signal based on the voice picked up by the microphone 14. The vibration waveform WV2 is generated by the terminal device P1 using vibration data based on the vibration detected by the vibration sensor 11.

[0052] In the process of step St11, the terminal device P1 detects the pre-fitting vibration waveform WV21 from the vibration waveform WV2. The detection method of the pre-fitting vibration waveform WV21 may be a known method, and may be performed by, for example, comparing with a pre-fitting vibration waveform for each connector registered in advance, by determining whether a voltage value, a sound pressure value (sound pressure level), or the like indicating the magnitude of vibration of the fitting sound waveform is equal to or greater than a predetermined value, or by waveform analysis using a trained model or the like.

[0053] The terminal device P1 starts a fitting determination process for the connector 13 from the timing (time t10 shown in FIG. 4) when the pre-fitting vibration waveform WV21 is detected from the vibration waveform WV2. The terminal device P1 executes a first fitting determination process for detecting a fitting vibration waveform WV22 from the vibration waveform WV2 after time t10, and a second fitting determination process for detecting a fitting vibration waveform WV11 from the fitting sound waveform WV1 after time t10.

[0054] When the terminal device P1 determines that the time difference between the time when the fitting vibration waveform WV22 is detected in the first fitting judgment process and the time when the fitting vibration waveform WV11 is detected in the second fitting judgment process is within a predetermined time (e.g., 0.5 seconds, 1 second), the terminal device P1 determines that the connector 13 is normally fitted. In the example shown in Fig. 4 and Fig. 5, the terminal device P1 detects the fitting vibration waveform at time t11 in each of the first fitting judgment process and the second fitting judgment process. Also, Fig. 5 shows an enlarged view of each of the fitting sound waveform WV1 and the vibration waveform WV2 at this time t11.

[0055] As described above, the terminal device P1 in embodiment 1 determines that the connector 13 is properly mated if the mating vibration waveform is detected at approximately the same time (i.e., within a predetermined time) in the first mating determination process and the second mating determination process, thereby efficiently suppressing erroneous determinations and deterioration in mating determination accuracy due to environmental noise generated around the worker or other vibrations when the worker's hand comes into contact with an obstacle.

[0056] As described above, the mating detection and determination system 100 according to the first embodiment includes a vibration sensor 11 (an example of a first sensor) that is attached to the hand or arm of a worker who performs a mating operation of mating a pair of connectors (i.e., a male connector 13A and a female connector 13B) and detects vibrations, a microphone 14 (an example of a second sensor) that collects sounds generated from the pair of connectors, and a terminal device P1 that is capable of communicating with the vibration sensor 11 and the microphone 14 and determines whether the connectors are mated or not using a vibration waveform WV2 (an example of a first vibration waveform) based on vibration and a mating sound waveform WV1 (an example of a second vibration waveform) based on sound. When the terminal device P1 determines that a pre-mating vibration waveform WV21 (an example of a predetermined vibration waveform) has been detected from the vibration waveform WV2 (first vibration waveform), it starts determining whether the connector 13 is mated or not.

[0057] As a result, the mating detection and determination system 100 according to the first embodiment can detect a pre-mating vibration waveform WV21 corresponding to contact vibration when a pair of connectors (i.e., the male connector 13A and the female connector 13B) come into contact, or frictional vibration occurring between contact and mating, from the vibration waveform WV2 based on the vibration detected by the vibration sensor 11. Therefore, the mating detection and determination system 100 according to the first embodiment can start mating determination of the pair of connectors at the timing when the pre-mating vibration waveform WV21 is detected, thereby shortening the time required for performing the mating determination process, effectively suppressing the processing load required for the mating determination process, and more effectively suppressing detection of noise (vibration) unnecessary for the mating determination process, such as environmental sounds occurring around the operator and mating sounds of other connectors.

[0058] As described above, the terminal device P1 in the mating detection and determination system 100 according to the first embodiment determines whether the connector 13 is mated or not based on the vibration waveform WV2 (first vibration waveform) and the mating sound waveform WV1 (second vibration waveform) after the timing at which it is determined that the pre-mating vibration waveform WV21 (predetermined vibration waveform) is detected from the vibration waveform WV2 (first vibration waveform). As a result, the mating detection and determination system 100 according to the first embodiment can start mating determination of the pair of connectors at the timing at which the pre-mating vibration waveform WV21 is detected, and therefore can more efficiently suppress detection of noise (vibration) unnecessary for the mating determination process, such as environmental sounds occurring around the operator and mating sounds of other connectors, thereby improving the accuracy of the mating determination.

[0059] As described above, the predetermined vibration waveform detected by the terminal device P1 in the mating detection and determination system 100 according to the first embodiment is the pre-mating vibration waveform WV21 that occurs when a pair of connectors (i.e., male connector 13A and female connector 13B) come into contact with each other. As a result, the mating detection and determination system 100 according to the first embodiment can detect the pre-mating vibration waveform WV21 that corresponds to the contact vibration of the pair of connectors (i.e., male connector 13A and female connector 13B) coming into contact with each other, which indicates that the worker has started the mating operation of the connectors 13, or the frictional vibration occurring between the contact and the mating.

[0060] Further, as described above, the terminal device P1 in the mating detection and determination system 100 according to the first embodiment measures a first detection time (for example, time t11 shown in FIG. 4) at which the mating vibration waveform WV22 generated when the pair of connectors are mated is detected from the vibration waveform WV2 (first vibration waveform) and a second detection time (for example, time t11 shown in FIG. 4) at which the mating vibration waveform WV11 is detected from the mating sound waveform WV1 (second vibration waveform), and determines that the connector 13 is mated when it is determined that the time difference between the first detection time and the second detection time is within a predetermined time (for example, 0.5 seconds, 1 second). As a result, the mating detection and determination system 100 according to the first embodiment determines that the connector 13 is normally mated when the mating vibration waveforms are detected at approximately the same time (that is, within a predetermined time), so that it is possible to efficiently suppress erroneous determinations due to environmental sounds generated around the operator, other vibrations when the operator's hand touches an obstacle, and a decrease in the accuracy of mating determination.

[0061] (Embodiment 2) The mating detection and determination system 100 according to the first embodiment has been described as an example in which the timing for an operator to start mating determination of the connector 13 is determined based on vibration data acquired by the vibration sensor 11. The mating detection and determination system 100A according to the second embodiment will be described as an example in which the timing for an operator to start mating determination of the connector 13 is determined based on an image captured by the camera 16.

[0062] The overall configuration of the mating detection and determination system 100A according to the second embodiment will be described with reference to Fig. 6. Fig. 6 is a block diagram showing an example of the overall configuration of the mating detection and determination system 100A according to the second embodiment. Note that in the overall configuration of the mating detection and determination system 100A according to the second embodiment shown in Fig. 6, the same components as those in the mating detection and determination system 100 according to the first embodiment are denoted by the same reference numerals, and description thereof will be omitted.

[0063] The fit detection and determination system 100A according to the second embodiment includes a vibration sensor 11, a transmitter 12, gloves 15A and 15B, a camera 16, and a terminal device P2.

[0064] The gloves 15A and 15B are different in color and are worn on both hands by a worker performing a fitting operation. Each of the gloves 15A and 15B may have a marker that allows the two gloves 15A and 15B to be identified.

[0065] The camera 16 is connected to the terminal device P2 so as to be able to communicate with it via wired or wireless communication. The camera 16 is configured to have at least a lens (not shown) and an image sensor (not shown). The image sensor is a solid-state imaging element such as a CCD (Charged-Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor), and converts an optical image formed on an imaging surface into an electrical signal. The camera 16 transmits the captured image to the terminal device P2.

[0066] The terminal device P2 is connected to the transmitter 12 so as to be able to communicate wirelessly, and is connected to the camera 16 so as to be able to communicate by wire or wirelessly. The terminal device P2 determines the timing to start a fitting determination of the connector 13 based on the captured image transmitted from the camera 16. The terminal device P2 also performs a fitting determination as to whether the connector 13 is properly fitted or not based on the vibration data transmitted from the transmitter 12, or the vibration data and the captured image transmitted from the camera 16, and outputs the fitting determination result. The terminal device P2 includes a communication unit 30, a processor 31, a memory 32, and a database DB2.

[0067] The communication unit 30 is connected to the transmitter 12 so as to be able to communicate wirelessly, and is connected to the camera 16 so as to be able to communicate by wire or wirelessly, and executes transmission and reception of data. The communication unit 30 outputs the vibration data transmitted from the transmitter 12 and the captured image (captured video) transmitted from the camera 16 to the processor 31.

[0068] The processor 31 is configured using, for example, a CPU, a DSP, or an FPGA, and controls the operation of each part of the terminal device P2. The processor 31 performs various processes and controls in cooperation with the memory 32. Specifically, the processor 31 refers to the programs and data stored in the memory 32 and executes the programs to realize the functions of each part. Note that the parts referred to here are the pre-mating operation determination part 31A and the mating determination part 31B.

[0069] The pre-fitting operation determination unit 31A performs image recognition processing on the captured image transmitted from the camera 16, and measures the distance between the gloves 15A and 15B worn on both hands of the worker. When the pre-fitting operation determination unit 31A determines that the measured distance is equal to or less than a predetermined distance (e.g., several cm), it generates a control command requesting the start of fitting determination and outputs it to the fitting determination unit 31B. Note that the pre-fitting operation determination unit 31A may recognize both hands of the worker by detecting the respective colors of the gloves 15A and 15B by image recognition processing, may recognize both hands of the worker by detecting markers attached to the gloves 15A and 15B by executing image recognition processing, or may recognize both hands of the worker by detecting the gloves 15A and 15B by executing image recognition processing.

[0070] Based on the control command output from the pre-mating operation determination unit 31A, the mating determination unit 31B starts a mating determination process from the timing when the pre-mating operation determination unit 31A determines that the distance between the glove 15A and the glove 15B is equal to or less than a predetermined distance. The mating determination unit 31B executes a third mating determination process based on the vibration data (vibration waveform) transmitted from the transmitter 12, and a fourth mating determination process based on the captured image transmitted from the camera 16. Note that the mating determination unit 31B may execute only the third mating determination process to determine the mating state of the connector 13.

[0071] The memory 32 has, for example, a RAM as a work memory used when executing each process of the processor 31, and a ROM for storing programs and data that define the operation of the processor 31. The RAM temporarily stores data or information generated or acquired by the processor 31. The ROM has written therein programs that define the operation of the processor 31. The memory 32 records various data used in the third and fourth fitting determination processes (fitting vibration waveform data, threshold values, etc. used to determine the fitting of the connector), learned data, etc.

[0072] The database DB2 is configured using a storage device such as a HDD or SSD, etc. The database DB2 records the vibration data transmitted from the transmitter 12 and the captured image (captured video) transmitted from the camera 16 in association with each other.

[0073] An operation procedure of the terminal device P2 in the second embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of an operation procedure of the terminal device P2 in the second embodiment.

[0074] The terminal device P2 acquires the captured image captured by the camera 16 and transmitted, and performs image recognition processing to recognize each of the gloves 15A and 15B worn on both hands of the worker based on the acquired captured image (St20).

[0075] The terminal device P2 measures the distance between the glove 15A and the glove 15B (that is, the distance between the worker's hands) based on the image recognition processing result (St21).

[0076] The terminal device P2 determines whether or not the measured distance between the glove 15A and the glove 15B is equal to or less than a predetermined distance (for example, a few centimeters) (St22).

[0077] When the terminal device P2 determines in the process of step St22 that the measured distance between the glove 15A and the glove 15B is equal to or shorter than a predetermined distance (St22, YES), the terminal device P2 proceeds to the process of steps St23A and St23B and starts the third fitting determination process and the fourth fitting determination process. The terminal device P2 starts acquiring vibration data (i.e., vibration data of fitting vibration) acquired by the vibration sensor 11 and transmitted from the transmitter 12 (St23A). In addition, the terminal device P2 continues acquiring the captured image transmitted from the camera 16 (St23B).

[0078] On the other hand, if the terminal device P2 determines in the processing of step St22 that the measured distance between glove 15A and glove 15B is not equal to or less than the predetermined distance (St22, NO), it proceeds to the processing of step St20, again acquires the captured image transmitted from the camera 16, and performs image recognition processing (St20).

[0079] As a third fitting determination process, terminal device P2 analyzes the vibration data and detects a fitting vibration waveform (for example, fitting vibration waveform WV22 shown in FIG. 4 and FIG. 5) when male connector 13A and female connector 13B are fitted together (St24A).

[0080] The terminal device P2 executes an image recognition process as a fourth fitting determination process, and detects a fitting operation for fitting the male connector 13A and the female connector 13B together (St24B). The fitting operation here may be detected (determined) based on whether or not the distance between the gloves 15A and 15B is smaller than the distance between the gloves 15A and 15B used in the determination process of step St22. The fitting operation may also be detected (determined) based on whether or not the movement (flow line data) of both hands of the worker during the fitting operation of the connector 13 is detected from the movement (flow line data) of both hands of the worker recognized by the image recognition process.

[0081] The terminal device P2 executes a comprehensive fitting judgment based on the judgment result of the third fitting judgment process and the judgment result of the fourth fitting judgment process (St25). The terminal device P2 judges whether or not a fitting vibration waveform is detected in the third fitting judgment process, and whether or not a fitting operation is detected in the fourth fitting judgment process. More specifically, the terminal device P2 compares the time when the fitting vibration waveform is detected in the third fitting judgment process with the time when the fitting operation is detected in the fourth fitting judgment process. If the terminal device P2 judges as a result of the comparison that the time difference between these is within a predetermined time (e.g., 0.5 seconds, 1 second), it judges that the connector 13 is normally fitted.

[0082] When the terminal device P2 determines that the fitting vibration waveform is detected in the third fitting judgment process and the fitting operation is detected in the fourth fitting judgment process as a result of the overall fitting judgment, and further determines that the time at which these were detected is within a predetermined time, the terminal device P2 generates and outputs a notice that the connector 13 is normally fitted (St26). On the other hand, when the terminal device P2 determines that the fitting vibration waveform is not detected in the third fitting judgment process or the fitting operation is not detected in the fourth fitting judgment process as a result of the overall fitting judgment, the terminal device P2 generates and outputs a notice that the connector 13 is not normally fitted (St26). Also, when the terminal device P2 determines that the fitting operation is detected in the third and fourth fitting judgment processes and the time difference between them is not within a predetermined time, the terminal device P2 generates and outputs a notice that the connector 13 is not normally fitted (St26).

[0083] As described above, the terminal device P2 in the second embodiment performs the mating judgment (i.e., the third mating judgment and the fourth mating judgment) from the timing when the pre-mating operation of the connector 13 by the worker is detected, thereby more effectively suppressing the mating judgment of the connector 13 based on the environmental sounds around the worker and the vibration of the worker's hand due to other factors (for example, contact between the worker's hand and surrounding obstacles, etc.), and improving the accuracy of the mating judgment of the connector 13. This allows the terminal device P2 to support the worker in more efficiently performing the mating judgment of the connector 13.

[0084] The terminal device P2 in the second embodiment may omit the processes of steps St23B, St24B, and St25. In such a case, the terminal device P2 executes only the third fitting determination process in step St23A, and determines whether the connector 13 is fitted or not based on the result of the third fitting determination process. Hereinafter, a procedure for performing a fitting determination of the connector 13 by executing only the third fitting determination process will be described.

[0085] When the terminal device P2 determines in the process of step St22 that the measured distance between the glove 15A and the glove 15B is equal to or shorter than a predetermined distance (St22, YES), the terminal device P2 proceeds to the process of step St23A and starts the third fitting determination process. The terminal device P2 starts acquiring the vibration data (i.e., the vibration data of the fitting vibration) acquired by the vibration sensor 11 and transmitted from the transmitter 12 (St23A).

[0086] On the other hand, if the terminal device P2 determines in the processing of step St22 that the measured distance between glove 15A and glove 15B is not equal to or less than the predetermined distance (St22, NO), it proceeds to the processing of step St20, again acquires the captured image transmitted from the camera 16, and performs image recognition processing (St20).

[0087] As a third fitting determination process, terminal device P2 analyzes the vibration data and detects a fitting vibration waveform (for example, fitting vibration waveform WV22 shown in FIG. 4 and FIG. 5) when male connector 13A and female connector 13B are fitted together (St24A).

[0088] When the terminal device P2 determines, as a result of the overall fitting judgment, that a fitting vibration waveform has been detected in the third fitting judgment process, it generates and outputs a notice that the connector 13 is normally fitted (St26). On the other hand, when the terminal device P2 determines, as a result of the overall fitting judgment, that a fitting vibration waveform has not been detected in the third fitting judgment process, it generates and outputs a notice that the connector 13 is not normally fitted (St26).

[0089] As described above, the terminal device P2 in the second embodiment can perform the fitting determination of the connector 13 without the fourth fitting process determination.

[0090] As described above, the mating detection and determination system 100A according to the second embodiment includes a vibration sensor 11 (one example of a sensor) that is attached to the hand or arm of a worker who performs a mating operation of mating a pair of connectors (i.e., male connector 13A and female connector 13B) and detects vibrations, a camera 16 that can capture images of both hands of the worker, and a terminal device P2 that can communicate with the vibration sensor 11 and the camera 16 and determines whether or not the connector 13 is mated using a vibration waveform based on the vibration (for example, the vibration waveform WV2 shown in FIG. 4). The terminal device P2 detects both hands based on the captured image captured by the camera 16, measures the distance between one hand and the other hand of the worker, and starts determining whether or not the connector 13 is mated when it determines that the measured distance is equal to or less than a predetermined distance (for example, several cm).

[0091] As a result, the mating detection and determination system 100A according to the second embodiment can detect both hands of the worker based on the captured image captured by the camera 16, and can determine whether or not the hands of the worker are approaching each other to within a predetermined distance based on the distance between the detected hands of the worker (i.e., whether or not the worker has started the mating operation of the pair of connectors). Therefore, the mating detection and determination system 100A can start mating determination of the pair of connectors at the timing when it is determined that the worker has started the mating operation of the pair of connectors, and therefore can shorten the time required for performing the mating determination process, effectively suppress the processing load required for the mating determination process, and more effectively suppress the detection of noise (vibration) unnecessary for the mating determination process, such as environmental sounds occurring around the worker and mating sounds of other connectors.

[0092] As described above, the terminal device P2 in the mating detection and determination system 100A according to the second embodiment determines whether the connector 13 is mated or not based on the vibration waveform (for example, the vibration waveform WV2 shown in FIG. 4) after the timing at which it is determined that the measured distance is equal to or less than a predetermined distance (for example, several centimeters). As a result, the mating detection and determination system 100A according to the second embodiment can start mating determination of the pair of connectors at the timing when both hands of the worker approach each other to within the predetermined distance (i.e., when the worker starts the mating operation of the pair of connectors), and therefore can more efficiently suppress detection of noise (vibrations) that are unnecessary for the mating determination process, such as environmental sounds occurring around the worker and mating sounds of other connectors, and can improve the accuracy of the mating determination.

[0093] As described above, the terminal device P2 in the mating detection and determination system 100A according to the second embodiment detects the mating operation of the pair of connectors by the worker based on the captured image, and determines whether the connectors 13 are mated or not based on the vibration waveform (e.g., the vibration waveform WV2 shown in FIG. 4) after the timing when it is determined that the measured distance is equal to or less than a predetermined distance (e.g., several cm) and the presence or absence of the mating operation based on the captured image. As a result, the mating detection and determination system 100A according to the second embodiment can perform mating determination of the pair of connectors using not only the vibration waveform (e.g., the vibration waveform WV2 shown in FIG. 4) but also the result of the analysis of the worker's operation (i.e., the presence or absence of the mating operation), thereby improving the accuracy of the mating determination.

[0094] As described above, the terminal device P2 in the mating detection and determination system 100A according to the second embodiment measures a first detection time (e.g., time t11 shown in FIG. 4) at which a mating vibration waveform (e.g., mating vibration waveform WV22 shown in FIG. 4) generated when a pair of connectors are mated is detected from a vibration waveform (e.g., vibration waveform WV2 shown in FIG. 4) and a second detection time (not shown) at which a mating operation is detected, and determines that the connector 13 is mated when it is determined that the time difference between the first detection time and the second detection time is within a predetermined time (e.g., 0.5 seconds, 1 second). As a result, the mating detection and determination system 100A according to the second embodiment determines that the connector 13 is normally mated when the mating vibration waveform and the mating operation are detected at approximately the same time (i.e., within a predetermined time), so that it is possible to efficiently suppress erroneous determinations due to environmental sounds generated around the operator, other vibrations when the operator's hand touches an obstacle, and a decrease in mating determination accuracy.

[0095] As described above, the camera 16 in the mating detection and determination system 100A according to the second embodiment captures images of the gloves 15A and 15B of different colors worn on both hands of the worker. The terminal device P2 detects the gloves 15A and 15B of different colors from the captured images and measures the distance. This allows the terminal device P2 in the mating detection and determination system 100A according to the second embodiment to more accurately detect each of the worker's hands based on the colors of the gloves 15A and 15B. Based on the detected distance between the gloves 15A and 15B, the terminal device P2 can determine whether the worker's hands are approaching each other to within a predetermined distance (i.e., whether the worker has started the mating operation of a pair of connectors).

[0096] As described above, the camera 16 in the mating detection and determination system 100A according to the second embodiment captures the gloves 15A and 15B, which are worn on both hands of the worker and have a predetermined marker (not shown). The terminal device P2 detects the predetermined marker from the captured image captured by the camera 16 and measures the distance. As a result, the terminal device P2 in the mating detection and determination system 100A according to the second embodiment can more accurately detect both hands of the worker based on the predetermined markers of each of the gloves 15A and 15B. Based on the detected distance between the gloves 15A and 15B, the terminal device P2 can determine whether the hands of the worker are approaching each other to within a predetermined distance (i.e., whether the worker has started the mating operation of a pair of connectors). The markers may be the same for the gloves 15A and 15B, or may be different for each. The markers may be determined for each worker so that the worker can be identified.

[0097] As described above, the camera 16 in the mating detection and determination system 100A according to the second embodiment captures images of the gloves 15A and 15B worn on both hands of the worker. The terminal device P2 performs image recognition using the captured images, detects the gloves 15A and 15B, and measures the distance. As a result, the terminal device P2 in the mating detection and determination system 100A according to the second embodiment can detect the gloves 15A and 15B worn on both hands of the worker based on the captured images captured by the camera 16, and determine whether the hands of the worker are approaching each other to within a predetermined distance based on the detected distance between the gloves 15A and 15B (i.e., whether the worker has started the mating operation of a pair of connectors).

[0098] Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, corrections, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also belong to the technical scope of the present disclosure. In addition, the components in the various embodiments described above may be arbitrarily combined within the scope of the invention. [Industrial Applicability]

[0099] The present disclosure is useful as providing a mating detection and determination system and a mating detection and determination method that can more efficiently perform mating determination of a connector. [Explanation of symbols]

[0100] 11 Vibration Sensor 12 Transmitter 13 Connectors 13A Male Connector 13B Female Connector 14. Mike 15A,15B Gloves 16 Cameras 20,30 Communications Department 21,31 Processor 21A Vibration judgment section before mating 21B, 31B Mating judgment part 22,32 Memory 31A Pre-mating operation determination section 100,100A Mating Detection Judgment System DB1,DB2 databases P1, P2 terminal device WV1 Mating sound waveform WV2 vibration waveform WV11, WV22 Mating vibration waveform WV21 Pre-mating vibration waveform

Claims

1. a first sensor that is attached to a hand or arm of a worker performing a fitting operation of fitting a pair of connectors and detects vibrations; a second sensor that picks up a sound generated from the pair of connectors; a terminal device capable of communicating with the first sensor and the second sensor, and determining whether or not the connector is fitted using a first vibration waveform based on the vibration and a second vibration waveform based on the sound, The terminal device when it is determined that a pre-mating vibration waveform that occurs when the pair of connectors contact each other is detected from the first vibration waveform, a determination is started as to whether or not the connectors are mated; determining whether or not the connectors are mated based on a mating vibration waveform generated when the pair of connectors are mated, which is detected from the first vibration waveform after the timing at which it is determined that the pre-mating vibration waveform has been detected from the first vibration waveform, and a mating vibration waveform generated when the pair of connectors are mated, which is detected from the second vibration waveform; Mating detection and judgment system.

2. The terminal device measuring a first detection time at which the fitting vibration waveform is detected from the first vibration waveform and a second detection time at which the fitting vibration waveform is detected from the second vibration waveform; if it is determined that the time difference between the first detection time and the second detection time is within a predetermined time, it is determined that the connector is mated. The fit detection and determination system according to claim 1 .

3. A method for detecting and determining whether a pair of connectors is mated, the method being performed by a terminal device, comprising: detecting vibrations with a first sensor attached to a hand or arm of a worker performing a fitting operation of fitting the pair of connectors; A sound generated from the pair of connectors is collected by a second sensor; when it is determined that a pre-mating vibration waveform that occurs when the pair of connectors come into contact with each other is detected from a first vibration waveform based on the vibration, a determination is started as to whether or not the connectors are mated; determining whether or not the connectors are mated based on a mating vibration waveform generated when the pair of connectors are mated, which is detected from the first vibration waveform after the timing at which it is determined that the pre-mating vibration waveform has been detected from the first vibration waveform, and a mating vibration waveform generated when the pair of connectors are mated, which is detected from a second vibration waveform based on the sound; Mating detection judgment method.

4. A first sensor that is attached to the hand or arm of a worker performing a fitting operation to fit a pair of connectors and detects vibrations; a second sensor that picks up a sound generated from the pair of connectors; a terminal device capable of communicating with the first sensor and the second sensor, and determining whether or not the connector is fitted using a first vibration waveform based on the vibration and a second vibration waveform based on the sound, The terminal device when it is determined that a predetermined vibration waveform is detected from the first vibration waveform, a determination is started as to whether or not the connector is mated; measuring a first detection time at which a mating vibration waveform generated when the pair of connectors are mated is detected from the first vibration waveform, and a second detection time at which the mating vibration waveform is detected from the second vibration waveform; if it is determined that the time difference between the first detection time and the second detection time is within a predetermined time, it is determined that the connector is mated. Mating detection and judgment system.

5. A method for detecting and determining whether a pair of connectors are mated, the method being performed by a terminal device, comprising: detecting vibrations with a first sensor attached to a hand or arm of a worker performing a fitting operation of fitting the pair of connectors; A sound generated from the pair of connectors is collected by a second sensor; when it is determined that a predetermined vibration waveform is detected from the first vibration waveform based on the vibration, a determination is started as to whether or not the connector is mated; measuring a first detection time at which a mating vibration waveform generated when the pair of connectors are mated is detected from the first vibration waveform, and a second detection time at which the mating vibration waveform is detected from a second vibration waveform based on the sound; if it is determined that the time difference between the first detection time and the second detection time is within a predetermined time, it is determined that the connector is mated. Mating detection judgment method.

Images