Plug-in detection method and detection device of connecting piece, storage medium and program product

Abstract

The invention provides an insertion detection method and detection device of a connecting piece, a storage medium and a program product. The method comprises the following steps: acquiring a sound signal acquired by a sound acquisition module arranged in a preset distance range of an operation part and a pressure signal acquired by a pressure acquisition module arranged at a force application position of the operation part; the operation part is a manipulator for performing insertion operation or a hand of an operator; and performing insertion analysis based on the sound signal and the pressure signal to obtain a detection result of whether the connecting piece is inserted in place or not. According to the embodiment of the invention, the sound signal and the pressure signal are collected, analysis is carried out in combination with the collected sound signal and pressure signal, the detection result of whether the connecting piece is plugged in place is obtained, and the plugging detection efficiency of the connecting piece is improved through an automatic data collection and analysis method.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic detection, in particular to a connection piece plugging detection method, a detection device, a storage medium and a program product. BACKGROUND

[0002] The communication and / or power supply between various functional components in an electrical equipment are mostly in the form of wiring harness connection, for example, signal communication of multiple board cards, connection of a battery and a vehicle in an electric vehicle, etc., all of which need to be plugged by connecting a connection piece on a wiring harness from an A component to a connection piece of a B component. In actual operation, after the connection piece is plugged, there may be a problem of not being plugged in place. Therefore, in the prior art, manual re-inspection is often used to ensure that the connection piece is plugged in place and reduce the probability of failure. However, the manual re-inspection method has the problem of low efficiency. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide a connection piece plugging detection method, a detection device, a storage medium and a program product to improve the detection efficiency of connection piece plugging.

[0004] In a first aspect, the embodiments of the present application provide a connection piece plugging detection method, comprising: acquiring a sound signal collected by a sound collection module arranged within a preset distance range of an operation part and a pressure signal collected by a pressure collection module arranged at a force application position of the operation part; the operation part being a mechanical hand or a hand of an operator for plugging operation; performing plugging analysis based on the sound signal and the pressure signal to obtain a detection result of whether the connection piece is plugged in place.

[0005] The embodiments of the present application acquire the sound signal and the pressure signal, analyze the acquired sound signal and pressure signal, and obtain the detection result of whether the connection piece is plugged in place. The efficiency of connection piece plugging detection is improved by the method of automatic data acquisition and analysis.

[0006] In a possible implementation manner of the first aspect, the plugging analysis based on the sound signal and the pressure signal to obtain the detection result of whether the connection piece is plugged in place comprises: inputting the sound signal into a first sound recognition model to obtain a sound recognition result; inputting the pressure signal into a pressure recognition model to obtain a pressure recognition result; selecting the sound recognition result and the pressure recognition result in the same time interval, if both the sound recognition result and the pressure recognition result represent plugging in place, it is determined that the connection piece is plugged in place; otherwise, it is determined that the connection piece is not plugged in place; wherein, the time interval corresponds to one sound recognition result and one pressure recognition result.

[0007] In the embodiment of the application, since the collected sound signal may contain interference signals similar to the sound during plugging, in order to filter out the interference signals, pressure recognition is combined, because the pinch force of the handheld connector during plugging is greater than the force during plugging, so it can be obtained by a pressure sensor, therefore, the sound signal and the pressure signal can be recognized respectively, and then the sound recognition result and the pressure recognition result are aligned in time, so that the detection result corresponding to the sound during plugging can be analyzed, and the detection accuracy is improved.

[0008] In a possible implementation of the first aspect, the plugging analysis based on the sound signal and the pressure signal to obtain the detection result of whether the connector is plugged in place includes: extracting a single sound sample from the sound signal, and extracting a single pressure sample from the pressure signal; for each single sound sample, obtaining a corresponding sound time interval; for each single pressure sample, obtaining a corresponding pressure time interval; matching the sound time interval and the pressure time interval, and taking the single sound sample corresponding to the sound time interval matched successfully as a target single sound sample; analyzing the target single sound sample by using the second sound recognition model to obtain the detection result of whether the connector is plugged in place.

[0009] In the embodiment of the application, since the collected sound signal may contain interference signals similar to the sound during plugging, in order to filter out the interference signals, pressure recognition is combined, because the pinch force of the handheld connector during plugging is greater than the force during plugging, so it can be obtained by a pressure sensor, therefore, the sound signal and the pressure signal can be recognized respectively, and then the sound recognition result and the pressure recognition result are aligned in time, so that the detection result corresponding to the sound during plugging can be analyzed, and the detection accuracy is improved.

[0010] In a possible implementation of the first aspect, the sound time interval and the pressure time interval satisfy any one of the following conditions: the sound time interval and the pressure time interval at least partially overlap; the time interval between the start time of the sound time interval and the end time of the pressure time interval is less than a first preset time interval; the time interval between the end time of the sound time interval and the start time of the pressure time interval is less than a second preset time interval.

[0011] The embodiment of the present application can filter single sound samples belonging to the plug connector from the sound signal through the pressure signal by matching the sound time interval with the pressure time interval, reduce the influence of external interference noise, and improve the detection accuracy.

[0012] In a possible implementation of the first aspect, the single sound sample is extracted from the sound signal, including: performing frequency domain signal enhancement processing on the sound signal to obtain an increased sound signal; converting the enhanced sound signal from the frequency domain to a time domain sound signal; extracting a continuous signal greater than a sound threshold from the time domain sound signal as the single sound sample.

[0013] In the embodiment of the present application, the frequency domain signal enhancement processing is performed on the sound signal, the enhanced sound signal is converted from the frequency domain to the time domain sound signal, and then the single sound sample is extracted from the time domain sound signal, so that the phase and timing information of the sound can be retained, and a data basis for subsequent analysis of the synchronization of the plug-in action is provided.

[0014] In a possible implementation of the first aspect, the single pressure sample is extracted from the pressure signal, including: obtaining a basic pressure when the hand-held material is held and a plug-in pressure when the plug connector is connected; determining a pressure threshold according to the basic pressure and the plug-in pressure; extracting a continuous pressure signal exceeding the pressure threshold from the pressure signal as the single pressure sample.

[0015] In the embodiment of the present application, the pressure threshold is determined according to the basic pressure and the plug-in pressure, so that the single pressure sample can be more accurately extracted from the pressure signal according to the pressure threshold.

[0016] In a possible implementation of the first aspect, the method further includes: collecting historical pressure of a preset time length before the plug connector at at least one historical moment; determining the basic pressure based on the historical pressure.

[0017] In the embodiment of the present application, the basic pressure is determined based on the historical pressure of the preset time length before the plug connector, so that the pressure generated by the operator or the mechanical hand when holding the connector without plugging can be more accurately extracted, and a data basis for subsequent determination of the pressure threshold is provided.

[0018] In a second aspect, the embodiments of the present application provide another method for detecting plugging of a connecting piece. The method is applied to a detection module in a detection device, and the detection device comprises a pressure acquisition module, a sound acquisition module and the detection module. The pressure acquisition module and the sound acquisition module are in communication connection with the detection module. The pressure acquisition module and the sound acquisition module are arranged on a glove worn by an operator, and the pressure acquisition module is arranged at a position corresponding to a force application position of an operating part. The method comprises the following steps: acquiring a sound signal acquired by the sound acquisition module arranged within a preset distance range of the operating part and a pressure signal acquired by the pressure acquisition module, wherein the operating part is a mechanical hand or a hand of an operator for plugging operation, and the sound signal and the pressure signal are acquired synchronously; extracting a single sound sample from the sound signal and a single pressure sample from the pressure signal; performing plugging analysis on the single sound sample by using a second sound recognition module to obtain a sound recognition result; performing plugging analysis on the single pressure sample by using a pressure recognition model to obtain a pressure recognition result; obtaining a detection result of whether the connecting piece is plugged in place based on the sound recognition result and the pressure recognition result.

[0019] The embodiments of the present application acquire the sound signal and the pressure signal, and analyze the acquired sound signal and the pressure signal to obtain the detection result of whether the connecting piece is plugged in place. The efficiency of plugging detection of the connecting piece is improved by the method of automatic data acquisition and analysis.

[0020] In a third aspect, the embodiments of the present application provide a detection device for plugging of a connecting piece, which comprises a pressure acquisition module, a sound acquisition module and a detection module. The pressure acquisition module and the sound acquisition module are in communication connection with the detection module. The pressure acquisition module is arranged at a position where the mechanical hand or the operator holds the connecting piece, and is used for acquiring a pressure signal and sending the pressure signal to the detection module. The sound acquisition module is arranged within a preset range from the connecting piece, and is used for acquiring a sound signal and sending the sound signal to the detection module. The detection module is used for executing the method of the first aspect or the second aspect.

[0021] The embodiments of the present application acquire the pressure signal by the pressure acquisition module, acquire the sound signal by the sound acquisition module, and perform plugging analysis on the acquired pressure signal and sound signal by the detection module to determine whether the connecting piece is plugged in place each time. The efficiency of plugging detection of the connecting piece is improved by the method of automatic data acquisition and analysis.

[0022] In a possible implementation manner of the third aspect, the detection device further comprises a hand wearing device; the hand wearing device is configured to be worn on a hand of the worker; The sound collecting module is arranged on a wearing body of the hand wearing device. The pressure collecting module is arranged on a finger sleeve of the hand wearing device.

[0023] The sound collecting module and the pressure collecting module are arranged on the hand wearing device, so that the sound signal and the pressure signal can be conveniently collected.

[0024] In a possible implementation manner of the third aspect, the wearing body comprises a glove and a sensor base; one side of the sensor base is detachably connected with the glove. The pressure collecting module and the sound collecting module are arranged on the other side of the sensor base.

[0025] The pressure collecting module and the sound collecting module are arranged on the sensor base, and the sensor base is detachably connected with the glove, so that the glove can be conveniently replaced when the glove is damaged or contaminated.

[0026] In a possible implementation manner of the third aspect, the sound collecting module is arranged at a first finger gap region of the glove.

[0027] The sound collecting module is arranged at the finger gap region of the glove, so that the sound signal of plugging can be collected, and the plugging operation is not affected.

[0028] In a possible implementation manner of the third aspect, the detection device further comprises a power supply module, a control module and a wireless transmission module. The power supply module is configured to supply power to the pressure collecting module, the sound collecting module, the control module and the wireless transmission module. The control module is configured to receive the pressure signal collected by the pressure collecting module and the sound signal collected by the sound collecting module, and transmit the pressure signal and the sound signal to the detection module through the wireless transmission module.

[0029] The power supply module supplies power, the control module collects the pressure signal and the sound signal, and the wireless transmission module transmits the pressure signal and the sound signal to the detection module, so that the detection module can analyze the plugging.

[0030] In a possible implementation manner of the third aspect, the pressure collecting module is a flexible thin film pressure sensor.

[0031] The flexible thin film pressure sensor is adopted, so that the limitation on the plugging operation is reduced.

[0032] In a fourth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium comprising: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause a computer to execute the method in each possible implementation manner of the first aspect.

[0033] In a fifth aspect, an embodiment of the present application provides a computer program product comprising computer program instructions, which, when read and executed by a processor, execute the method in each possible implementation manner of the first aspect.

[0034] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be learned by practice of the application. The objects and other advantages of the present application will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0036] Figure 1 A schematic structural diagram of a connection plug-in detection device provided by an embodiment of the present application; Figure 2 A schematic structural diagram of a detection device provided by an embodiment of the present application; Figure 3 A schematic diagram of a detection device provided by an embodiment of the present application arranged on a hand; Figure 4 A schematic flow diagram of a connection plug-in detection method provided by an embodiment of the present application; Figure 5 A schematic diagram of a time-domain waveform of a sound signal provided by an embodiment of the present application; Figure 6 A schematic diagram of a time-domain waveform of a pressure signal provided by an embodiment of the present application; Figure 7 A schematic flow diagram of another connection plug-in detection method provided by an embodiment of the present application; Figure 8 A schematic diagram of a signal acquisition process provided by an embodiment of the present application; Figure 9 A schematic structural diagram of another connection plug-in detection device provided by an embodiment of the present application; Figure 10The detection module structure schematic diagram provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0037] The embodiments of the technical solutions of the present application will be described in detail below with reference to the drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and therefore only serve as examples, and cannot limit the protection scope of the present application.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "include" and "have" and any variations thereof in the specification and claims of the present application and the above description of drawings are intended to cover non-exclusive inclusion.

[0039] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise explicitly and specifically limited.

[0040] In this paper, the reference to "embodiments" means that the specific features, structures or properties described in conjunction with the embodiments can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor is it an independent or alternative embodiment to other embodiments. The skilled person in the art explicitly and implicitly understands that the embodiments described herein can be combined with other embodiments.

[0041] In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " in this paper generally represents a "or" relationship between the front and rear associated objects.

[0042] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two), and similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0043] In the description of the embodiments of the present application, the orientations or positional relationships indicated by the technical terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the embodiments of the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the embodiments of the present application.

[0044] In the description of the embodiments of the present application, unless otherwise explicitly specified and limited, the technical terms "mounting", "connection", "connection", "fixing" and the like should be understood in a broad sense, for example, can be fixed connection, can also be detachable connection, or integral; can be mechanical connection, can also be electrical connection; can be directly connected, can also be indirectly connected through an intermediate medium, can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0045] The communication and / or power supply between various functional components in the electrical equipment are mostly in the form of wiring harness connection, for example: signal communication of multiple board cards, connection of battery and vehicle in electric vehicle, etc., all of which need to be connected by inserting the connecting piece on the wiring harness from A component into the connecting piece of B component.

[0046] Taking the insertion of the connector on the vehicle as an example, the current wiring harness connector installation process is: insertion, listening and pulling out (one insertion, two listening and three pulling out) → self-checking by marking → manual pressing by another person for rechecking → rechecking by marking → photographing and archiving. Therefore, the insertion of the wiring harness connector to the correct position in the production site is mainly guaranteed by manual work. When the insertion is not in place or virtual insertion occurs, the process detection cannot identify the virtual insertion, which causes the virtual insertion to be pulled out after the product is transported or the vehicle is running, resulting in connection failure, affecting the function of the whole vehicle and even the driving safety. In the above wiring harness detection, the labor accounts for 50% of the total labor of wiring harness installation, the rechecking cost is high, the efficiency is low, and the pulling out of the wiring harness may cause the terminal buckle of the single-sided wiring harness to be pulled out, causing defects.

[0047] In order to solve this technical problem, the embodiments of the present application provide a connection piece insertion detection method, which obtains the sound signal collected by the sound collection module arranged within the preset distance range of the operation part and the pressure signal collected by the pressure collection module arranged at the force applying position of the operation part, performs insertion analysis based on the sound signal and the pressure signal, and obtains the detection result of whether the connection piece is inserted in place. Compared with manual detection, the embodiments of the present application can more accurately detect whether the connection piece is inserted in place through sensor data.

[0048] It is understood that the connector insertion detection method provided in this application embodiment can be applied to various scenarios such as wire harness connectors and water cooling pipe connections. Among them, the insertion structures on lithium batteries include heating film connectors, low-voltage connectors, cabinet fan connectors, high-voltage connectors, heating connectors, CSC connectors, BMU connectors, FUSE connectors, and various wire harness terminal connections.

[0049] Figure 1 This is a schematic diagram of the structure of a connector insertion detection device provided in an embodiment of this application, as shown below. Figure 1 As shown, the detection device includes a pressure acquisition module 101, a sound acquisition module 102, and a detection module 103; both the pressure acquisition module 101 and the sound acquisition module 102 are communicatively connected to the detection module 103. The pressure acquisition module 101 and the sound acquisition module 102 can communicate with the detection module 103 via wired connection, wireless connection, or indirect connection, for example, through an intermediate data storage / processing device.

[0050] The pressure acquisition module 101 is installed on the part of the robot arm or operator that grips the connector, and is used to collect pressure signals and send them to the detection module 103. The pressure acquisition module 101 can be a flexible thin-film pressure sensor, a micro-fused force-sensitive resistor, or a fiber Bragg grating (FBG), etc. The pressure acquisition module 101 can be attached to the inner surface of the fingers of the robot arm or operator gripping the connector; that is, one side of the pressure acquisition module 101 contacts the fingers of the robot arm or operator, and the other side contacts the connector, to detect the force applied by the robot arm or operator when gripping the connector. When the robot arm or operator is inserting the connector, the force applied to the connector is often greater than when no insertion operation is performed. Therefore, by collecting the pressure signal, it can be determined whether the robot arm or operator has performed an insertion operation.

[0051] The sound collection module 102 is arranged within a preset range of the connecting piece, is configured to collect a sound signal, and sends the sound signal to the detection module 103. The sound collection module 102 can be a microphone sensor, configured to collect sound in a working process. Considering environmental noise and other interference sound in the working process, the sound collection module 102 can be an integrated band-pass filter, configured to retain only sound in a specific frequency range and avoid part of process interference sound. The sound collection module 102 can be integrated on a bracelet, worn on a wrist of a worker, integrated on a glove used by the worker, or fixed on a workbench, without affecting normal work of the worker and can normally collect a sound signal.

[0052] The detection module 103 receives the pressure signal and the sound signal collected by the pressure collection module 101 and the sound collection module 102 respectively, and performs plug-in analysis on the pressure signal and the sound signal to determine whether the connecting piece is plugged in. The detection module 103 can be an industrial computer or other electronic equipment running a detection algorithm, as long as it can communicate with the pressure collection module 101 and the sound collection module 102 and perform plug-in analysis based on the received pressure signal and sound signal. The detection module 103 can receive the pressure signal and the sound signal through a wireless communication mode. The wireless communication technology can include but is not limited to standard Bluetooth technology, Bluetooth low power consumption technology, wireless compatibility authentication (Wireless Fidelity, WIFI) technology, infrared technology, Zigbee technology, ultra-wideband (Ultra Wide Bande, UWB) technology or near field communication (Near Field Communication, NFC) technology, and the present application does not make special limitations on this.

[0053] It should be noted that the specific process of plug-in analysis of the detection module 103 based on the pressure signal and the sound signal is described in subsequent embodiments.

[0054] The present application collects a pressure signal through a pressure collection module, collects a sound signal through a sound collection module, and performs plug-in analysis on the collected pressure signal and sound signal through a detection module to determine whether the connecting piece is plugged in each time. Through the method of automatic data collection and analysis, the efficiency of connecting piece plug-in detection is improved.

[0055] On the basis of the above-mentioned embodiments, Figure 2 A detection device entity structure schematic diagram provided by the present application embodiment is provided. Figure 3 A detection device provided by the present application embodiment is arranged on a hand. It should be noted that,Figure 2 In actual production process, the detection device can be packaged to be better in performance and appearance. The detection device further comprises a hand wearing device, which is used for wearing on the hand of the operator, for example, a glove. The sound collecting module 102 can be arranged on the wearing body of the hand wearing device, i.e., the glove, and specifically can be arranged at the first finger web gap area (the tiger mouth) of the glove or at the wrist of the glove. On the one hand, the sound signal of the plugging can be collected, and on the other hand, the plugging operation is not affected. The sound collecting module 102 can be a microphone sound sensor, which is connected to the control module through two physical signal lines along the outer surface of the glove. A wire harness buckle is arranged at the tiger mouth to prevent the microphone sensor element from changing the position during the operation of the staff, so that the collected plugging signal of the wire harness terminal is more stable and is not affected by the actual collection distance of the process.

[0056] The pressure collecting module 101 is arranged on the finger sleeve of the hand wearing device. For example, if the operator holds the connecting piece by the index finger and the thumb, the pressure collecting module can be arranged on the finger pulp of the index finger and the thumb and covers the whole area of the first joint of the end of the index finger and the thumb. The pressure collecting module 101 can be a pressure-sensitive force sensor or a flexible thin-film pressure sensor, which is connected to the integrated control module through two physical signal lines along the outer surface of the glove. Thus, the operator can stably collect the plugging force signal during normal operation. The control module can be arranged at the wrist of the glove, which can be arranged in the form of a flat cylinder of a watch, the diameter of which is within 35 mm and the height of which is within 10 mm, so as to minimize the influence of wearing the glove on the actual operation of the operator. The control module is used for receiving the pressure signal collected by the pressure collecting module and the sound signal collected by the sound collecting module and sending the pressure signal and the sound signal to the detection module.

[0057] The embodiment of the application sets the sound collecting module and the pressure collecting module on the hand wearing device, which is integrated and convenient for the operator to wear and take off and is convenient for collecting the sound signal and the pressure signal.

[0058] On the basis of the above embodiment, the wearing body comprises a glove and a sensor base; one side of the sensor base is detachably connected with the glove; for example, a hook surface or a hair surface of a magic tape or a male buckle or a female buckle of a snap fastener is arranged on the glove, which is used for detachably connecting the sensor base with the glove.

[0059] The pressure collecting module and the sound collecting module are arranged on the other side of the sensor base. The sensor base can be a flexible film, such as a piece of cloth, etc. If the glove is provided with a hook surface of a magic tape, then the sensor base can be provided with a loop surface of the magic tape; conversely, if the glove is provided with a loop surface of the magic tape, then the sensor base can be provided with a hook surface of the magic tape. Similarly, one of the glove and the sensor base can be provided with a male buckle, and the other can be provided with a female buckle.

[0060] The pressure collecting module and the sound collecting module are arranged on the other side of the sensor base. The sensor base can be a flexible film, such as a piece of cloth, etc. If the glove is provided with a hook surface of a magic tape, then the sensor base can be provided with a loop surface of the magic tape; conversely, if the glove is provided with a loop surface of the magic tape, then the sensor base can be provided with a hook surface of the magic tape. Similarly, one of the glove and the sensor base can be provided with a male buckle, and the other can be provided with a female buckle.

[0061] On the basis of the above-mentioned embodiment, the detection device further comprises a power supply module, a control module and a wireless transmission module. The power supply module is used for supplying power to the pressure collecting module, the sound collecting module, the control module and the wireless transmission module; the power supply module can be a rechargeable battery, which can be a cylindrical battery, a square shell battery or a button battery.

[0062] The control module is used for receiving the pressure signal collected by the pressure collecting module and the sound signal collected by the sound collecting module, and sending the pressure signal and the sound signal to the detection module through the wireless transmission module.

[0063] It can be understood that the detection device can further comprise a charging / power supply module, a data storage module for storing the collected pressure signal and sound signal, and a data conversion module for converting the collected pressure signal and sound signal in data format, such as converting the collected analog signal into digital signal.

[0064] Based on the detection device of the above-mentioned embodiment, the application provides a plug-in detection method of a connecting piece, as shown in Figure 4 The method comprises: Step 401: obtaining the sound signal collected by the sound collecting module arranged within the preset distance range of the operating part, and the pressure signal collected by the pressure collecting module arranged at the force applying position of the operating part; the operating part is a mechanical hand or a hand of an operator for plug-in operation; Step 402: performing plug-in analysis based on the sound signal and the pressure signal to obtain a detection result of whether the connecting piece is plugged in place.

[0065] In a specific implementation process, the preset distance range refers to a distance at which the sound emitted by the plug connector can be collected. Of course, the closer to the plug connector, the better, but it should not affect the plug operation of the operator or the mechanical hand. Therefore, the setting of the preset distance range needs to consider the decibel value of the environmental noise in the workshop, the decibel value of the "click" sound when the wire harness connector is plugged, and the sensitivity of the microphone. For example, in a scenario where the environmental noise is 60-80 dB, the wire harness plug sound is 55-70 db, and the operating part is a hand, the preset distance range can be set to 5-20 cm.

[0066] The sound signal and the pressure signal can be signals continuously collected during the operation process, which includes plug operation and non-plug operation. Therefore, the sound signal includes the sound signal corresponding to the plug operation and the sound signal corresponding to the non-plug operation. Similarly, the pressure signal includes the pressure signal corresponding to the plug operation and the pressure signal corresponding to the non-plug operation.

[0067] The operating part refers to an execution component for plug operation of the connector, for example, a mechanical hand or a hand of an operator. The operating part needs to hold or grip the connector when plugging the connector. Therefore, the pressure signal refers to the pressure applied to the connector by the operating component.

[0068] In the sound signal, there can be interference sound similar to the plug sound. In order to reduce the influence of the interference sound on the plug analysis, the pressure signal can be used for correction to identify which time period in the sound signal belongs to the sound signal generated by the plug connector. In addition, when the connector is plugged in place, the characteristics reflected on the sound have certain identifiability. Therefore, the embodiment of the present application can identify whether the connector is plugged in place based on the sound signal, thereby obtaining a detection result.

[0069] The embodiment of the present application collects the sound signal and the pressure signal, and analyzes the collected sound signal and the pressure signal to obtain a detection result of whether the connector is plugged in place. Through the method of automatic data collection and analysis, the efficiency of the connector plug detection is improved.

[0070] Based on the above embodiment, the sound signal and the pressure signal are used for plug analysis to obtain a detection result of whether the connector is plugged in place, which includes: inputting the sound signal into a first sound recognition model to obtain a sound recognition result; inputting the pressure signal into a pressure recognition model to obtain a pressure recognition result; The sound recognition result and the pressure recognition result in the same time interval are selected, and if the sound recognition result and the pressure recognition result both represent that the connector is plugged in place, it is determined that the connector is plugged in place; otherwise, it is determined that the connector is not plugged in place; wherein the time interval corresponds to one sound recognition result and one pressure recognition result.

[0071] In the specific implementation process, the first sound recognition model is obtained by pre-construction and training, wherein the first sound recognition model can be a convolutional neural network model, and can also be other network models capable of processing sound signals and recognizing them, and the present application embodiment does not make specific limitation thereto. The training samples can include positive samples and negative samples, wherein the positive samples refer to sound signals of the connector being plugged in place, and are marked as being plugged in place. The negative samples refer to sound signals of the connector not being plugged in place, and are marked as not being plugged in place; the negative samples can also include sound signals that are not insertion operations, and are marked as non-insertion sound. After obtaining the training samples, the sound signals in the training samples are input into the to-be-trained model to obtain the prediction result output by the to-be-trained model, which is used to represent whether the sound signal in the training sample is a sound signal of being plugged in place. The prediction result is compared with the label of the training sample, and the internal parameters of the to-be-trained model are optimized in reverse according to the comparison result. The above training process is repeatedly executed until the performance of the to-be-trained model meets the requirements, and the trained first sound recognition model is obtained.

[0072] After obtaining the trained first sound recognition model, the sound signal is input into the first sound recognition model to obtain the sound recognition result output by the first sound recognition model. It should be noted that the sound signal input into the first sound recognition model can be a signal containing multiple insertion operations, or a signal containing a single insertion operation. For the signal containing multiple insertion operations, the first sound recognition model can divide the features of the signal into time periods and generate a sound recognition result corresponding to each time period. For the signal containing a single insertion operation, the first sound recognition model inputs the sound recognition result corresponding to the insertion operation.

[0073] The pressure recognition model is obtained by pre-construction and training. The pressure recognition model can be a convolutional neural network model, and can also be other network models capable of processing pressure signals and identifying them. The embodiments of the present application do not make specific limitations thereto. The training samples can include positive samples and negative samples. The positive sample refers to a pressure signal of a connector inserted in place, and is marked as inserted in place. The negative sample refers to a pressure signal of a connector not inserted in place, and is marked as not inserted in place. The negative sample can also include a pressure signal that is not an insertion operation, and is marked as a non-insertion pressure. After obtaining the training samples, the pressure signals in the training samples are input into the to-be-trained model to obtain a prediction result output by the to-be-trained model. The prediction result is used to represent whether the pressure signal in the training sample is an insertion-in-place pressure signal. The prediction result is compared with the label of the training sample, and the internal parameters of the to-be-trained model are optimized in reverse according to the comparison result. The above training process is repeatedly executed until the performance of the to-be-trained model meets the requirements, and a trained pressure recognition model is obtained.

[0074] After obtaining the trained pressure recognition model, the pressure signal is input into the pressure recognition model to obtain a pressure recognition result output by the pressure recognition model. It should be noted that the pressure signal input into the pressure recognition model can be a signal containing multiple insertion operations, or a signal containing a single insertion operation. For a signal containing multiple insertion operations, the pressure recognition model can divide the features of the signal into time periods and generate a pressure recognition result corresponding to each time period. For a signal containing a single insertion operation, the pressure recognition model inputs a pressure recognition result corresponding to the insertion operation.

[0075] It should be noted that for a single insertion operation, the time period of the insertion operation of the sound signal and the pressure signal is the same. For a plurality of insertion operations, the first sound recognition model can perform signal segmentation on the sound signal to obtain a sound signal corresponding to a single insertion operation; the pressure recognition model can perform signal segmentation on the pressure signal to obtain a pressure signal corresponding to a single insertion operation.

[0076] The sound recognition result and the pressure recognition result in the same time interval are selected. If the sound recognition result and the pressure recognition result both represent that the plugging operation corresponding to the time interval is plugged in place, it is determined that the connector is plugged in place in this plugging. If the sound recognition result represents that the plugging is in place, but the pressure recognition result represents that the plugging is not in place, it is indicated that the sound signal may be an interference signal from the outside, but the signal characteristics of the interference signal are very similar to the signal characteristics of the plugging in place, which causes the first sound recognition model to be misdetected as the plugging in place. If the sound recognition result represents that the plugging is not in place, and the pressure recognition result represents that the plugging is in place, it is indicated that the plugging is not in place, and the reason why the pressure signal is recognized as the plugging in place may be that the operation part is misoperated. If the sound recognition result represents that the plugging is not in place, and the pressure recognition result also represents that the plugging is not in place, it is indicated that the plugging is not in place.

[0077] In the embodiment of the application, there may be interference signals similar to the sound during plugging in the collected sound signal. In order to filter out the interference signals, the pressure recognition is combined. Because the pinch force of the handheld connector during plugging is greater than the force during no plugging, the pressure sensor can be used to collect and obtain, so that the sound signal and the pressure signal can be recognized respectively, and then the sound recognition result and the pressure recognition result are aligned in time, so that the detection result corresponding to the sound belonging to plugging can be analyzed, and the detection accuracy is improved.

[0078] On the basis of the above embodiment, the sound signal and the pressure signal are used for plugging analysis to obtain the detection result of whether the connector is plugged in place, including: extracting a single sound sample from the sound signal, and extracting a single pressure sample from the pressure signal; for each single sound sample, obtaining a corresponding sound time interval; for each single pressure sample, obtaining a corresponding pressure time interval; matching the sound time interval and the pressure time interval, and taking the single sound sample corresponding to the matched sound time interval as a target single sound sample; using a second sound recognition model to analyze the target single sound sample to obtain the detection result of whether the connector is plugged in place.

[0079] In the specific implementation process, the main role of the pressure signal is to assist in recognizing the interference signal in the sound signal which does not belong to the plugging operation, but the sound signal characteristics are similar to the sound signal characteristics of the plugging operation. Therefore, a plurality of single sound samples can be extracted from the sound signal containing a plurality of plugging operations, and a plurality of single pressure samples can be extracted from the pressure signal containing a plurality of plugging operations.

[0080] The collected sound signal can be first processed by fast Fourier transform for data signal enhancement, and the frequency domain enhancement processing is converted back to time domain, Figure 5 A time domain waveform diagram of a sound signal provided by an embodiment of the present application is shown in Figure 5 The change trend of the sound signal is: environmental sound→insertion sound→environmental sound, so the sound signal can be segmented into single sound samples according to a sound threshold, for example, a signal segment greater than the sound threshold is taken as a single sound sample. The time corresponding to the signal segment is the sound time interval. It can be understood that in addition to the segmentation of single sound samples according to the sound threshold, segmentation can also be performed according to the change rate of the sound signal, and the specific segmentation manner is not limited by the embodiments of the present application.

[0081] The change trend of the pressure signal is: small pressure signal→insertion pressure signal→small pressure signal, Figure 6 A time domain waveform diagram of a pressure signal provided by an embodiment of the present application is shown in Figure 6 The segmentation of single pressure samples can be based on a pressure threshold, and a continuous signal segment greater than the pressure threshold is taken as a single pressure sample. The time period corresponding to the signal segment is the pressure time interval.

[0082] Among them, the single sound sample refers to the sound signal that may belong to one insertion operation extracted from the sound signal. The single pressure sample refers to the pressure signal that may belong to one insertion operation extracted from the pressure signal.

[0083] For each single sound sample, the corresponding sound time interval is obtained, and for each single pressure sample, the corresponding pressure time interval is obtained. For each sound time interval, each pressure time interval is matched, if the matching is successful, it means that the single sound sample is the sound caused by the insertion connector. If the matching is not successful, it means that the single sound sample is non-insertion sound.

[0084] It should be noted that whether the matching is successful can be determined by any of the following ways: The sound time interval and the pressure time interval at least partially overlap; for example, the sound time interval is 10:10:10-10:10:12, and the pressure time interval is 10:10:9-10:10:12, and it is determined that the matching is successful.

[0085] The time interval between the start time of the sound time interval and the end time of the pressure time interval is less than a first preset time interval; assuming that the first preset time interval is 2 seconds, the sound time interval is 10:10:10-10:10:12, and the pressure time interval is 10:10:8-10:10:11, it is determined that the matching is successful.

[0086] The time interval between the end time of the sound time interval and the start time of the pressure time interval is less than a second preset time interval. Assuming that the first preset time interval is 2 seconds, the sound time interval is 10:10:10-10:10:12, and the pressure time interval is 10:10:13-10:10:14, it is determined that the matching is successful.

[0087] In the embodiments of the present application, interference signals similar to the sound during plugging may exist in the collected sound signals. In order to filter out the interference signals, pressure recognition is combined. Because the pinching force of the handheld connecting piece during plugging is greater than the force during plugging, the pressure sensor can be used to collect and obtain, so that the signals in the sound signal that do not belong to the plugging sound can be filtered out through the pressure signal, and then the signals belonging to the plugging sound are analyzed, so that the detection result corresponding to the plugging sound can be obtained, and the detection accuracy is improved.

[0088] On the basis of the above-mentioned embodiments, when the single pressure sample is extracted from the pressure signal, the pressure threshold value can be used for extraction, and the determination method of the pressure threshold value is as follows: The basic pressure when the handheld material is obtained and the plugging pressure when the connecting piece is plugged are obtained. The pressure threshold value is determined according to the basic pressure and the plugging pressure.

[0089] In the specific implementation process, the mechanical hand or the worker holds the connecting piece, but has not started the plugging operation, the pressure sensor collects the pressure when not plugging, and the pressure when not plugging corresponding to different workers is different. The pressure when not plugging is taken as the basic pressure in the embodiments of the present application. The basic pressure can be the pressure collected by the pressure sensor when the mechanical hand or the worker holds the material (i.e. the connecting piece) in the historical time period, but does not perform the plugging operation. The average value of the pressure collected in the historical time period can be calculated as the basic pressure. The average value of the pressure when the mechanical hand or the worker plugs the connecting piece in the historical time period can also be collected as the plugging pressure.

[0090] After obtaining the basic pressure and the plugging pressure, the pressure threshold value can be determined according to the basic pressure and the plugging pressure. For example, the average value of the basic pressure and the plugging pressure can be taken as the pressure threshold value, or the pressure threshold value can be calculated according to the formula . Wherein, is the pressure threshold value; The value represents the average pressure value of the preset number of basic pressure sample data in the natural non-plugging state, and the preset number can be 30 or more, of course, it can also be less than 30, and the specific value can be adjusted according to the actual situation; The value represents the average pressure value of a preset number of sample data within a preset time period before the sound signal triggers in the terminal plug-in state of the wire harness, wherein the preset time period can be 0.5 seconds, and can also be a longer or shorter time period, and the preset number can be 30 or more, and can also be less than 30, and the specific value can be adjusted according to actual conditions; and is a preset parameter, for example, which can be 3, or can be adjusted according to actual conditions; is the standard deviation of the basic pressure sample data; is the standard deviation of the plug-in pressure sample data.

[0091] After obtaining the pressure threshold, the continuous pressure signal exceeding the pressure threshold can be extracted from the pressure signal as a single pressure sample.

[0092] On the basis of the above embodiment, in order to accurately set the pressure threshold to extract the plug-in pressure from the pressure signal, the basic pressure can be determined by the following method: Collecting historical pressure within a preset time period before the plug-in connector at at least one historical time; Determining the basic pressure based on the historical pressure.

[0093] In the specific implementation process, the preset time period can be 0.5 seconds, the pressure signal in the historical time period can be obtained, the pressure signal segment corresponding to the plug-in operation can be manually marked, and then the historical pressure of 0.5 seconds before each plug-in operation corresponding pressure signal segment can be extracted, and then the average value of the extracted historical pressure can be taken as the basic pressure.

[0094] The embodiment of the application determines the basic pressure based on the historical pressure within a preset time period before the plug-in connector, so that the pressure generated by the operator or the mechanical hand when holding the connector without plugging can be more accurately extracted, thereby providing a data basis for subsequent determination of the pressure threshold.

[0095] In another embodiment, the embodiment of the application provides a work flow for detecting the plug-in of the connector, which is specifically as follows: 1. Work preparation: turn on the signal acquisition device, industrial computer and other devices.

[0096] 2. Start signal acquisition: after receiving the product in-place signal, start continuous acquisition of work signals.

[0097] 3. Assembly work and signal acquisition: assemble according to the normal work process, and synchronously acquire the corresponding signals in the work process.

[0098] 4. Signal analysis: the industrial computer analyzes the acquired work signals to determine whether the work result of this time is “OK” or “NG”.

[0099] 5. The determination result transmission application: the industrial computer transmits the determination result to the pull-in PLC, which can determine and prompt the detection result of this time through the display screen, sound and light alarm, and associated MES.

[0100] 6. Product circulation: based on the detection result, the qualified products are released normally, and the unqualified products are recorded.

[0101] Figure 7 Another connection plug-in detection method flowchart provided by the embodiment of the application is shown in FIG. 6. Figure 7 The detection module in the detection device provided in the above embodiment runs an AI model. The AI model training is mainly applied to two scenes: (1) target sample cutting; and (2) correct threshold setting.

[0102] In the target sample cutting process, the value range of the target sample is continuously shortened by continuously collecting operation process signal data (correct and NG data) and fusing sound signals and pressure signals in the time domain, so that the target sample is more accurate.

[0103] In the correct threshold setting process, the threshold range formulated in the early stage is continuously improved by a large amount of operation process data, especially the “overkill” and “missed kill” samples, so that the determination of the test result tends to be more accurate.

[0104] The plug-in detection method comprises the following steps. Step 701: acquiring a sound signal collected by a sound collection module arranged within a preset distance range of an operation part and a pressure signal collected by a pressure collection module arranged at a force application position of the operation part; the operation part is a mechanical hand or a hand of an operator for plug-in operation. The detection module can receive the sound signal and the pressure signal in a wireless manner, which can be a Bluetooth transmission manner. The pressure collection module and the sound collection module are arranged on a glove worn by the operator. Specifically, the pressure collection module can be arranged on the finger pulp of the glove for gripping the connection piece, and the sound collection module can be arranged in the first finger web gap area of the glove. The pressure collection module and the sound collection module synchronously collect signals. Figure 8 A signal collection process diagram provided by the embodiment of the application is shown in FIG. 6. Figure 8 The signal collection module comprises a sound collection module and a pressure collection module. During the plug-in process of the connection piece, the sound collection module collects a sound signal, and the pressure collection module collects a pressure signal. Then, the collected sound signal and pressure signal are sent to the detection module in a wireless manner.

[0105] Step 702: extracting a single sound sample from the sound signal and a single pressure sample from the pressure signal. It can be understood that the manner of extracting the single sound sample and the single pressure sample can be referred to the above embodiment, which will not be described here.

[0106] Step 703: performing splice analysis on the single sound sample by using the second sound recognition module to obtain a sound recognition result; Step 704: performing splice analysis on the single pressure sample by using the pressure recognition model to obtain a pressure recognition result; The specific implementation of splice analysis of step 703 and step 704 can refer to the above embodiments, which will not be repeated here.

[0107] Step 705: obtaining a detection result of whether the connector splice is in place based on the sound recognition result and the pressure recognition result. Selecting the sound recognition result and the pressure recognition result in the same time interval, if both the sound recognition result and the pressure recognition result represent that the connector splice is in place, it is determined that the connector splice is in place.

[0108] The embodiments of the present application obtain the detection result of whether the connector splice is in place by collecting sound signals and pressure signals, and analyzing the collected sound signals and pressure signals. Through the method of automatic data collection and analysis, the efficiency of connector splice detection is improved.

[0109] Figure 9 A structure diagram of a connector splice detection device provided by the embodiments of the present application is shown. The device can be a module, a program segment or code on an electronic device. It should be understood that the device corresponds to the above-mentioned Figure 4 method embodiments, and can perform each step involved in the Figure 4 method embodiments. The specific functions of the device can be referred to the description in the above, and the detailed description is appropriately omitted here to avoid repetition. The device includes a signal acquisition module 901 and a splice analysis module 902, wherein: The signal acquisition module 901 is configured to acquire sound signals collected by a sound collection module arranged within a preset distance range of an operating part, and pressure signals collected by a pressure collection module arranged at a force application position of the operating part. The operating part is a mechanical hand or a hand of an operator for splice operation; The splice analysis module 902 is configured to perform splice analysis based on the sound signals and the pressure signals to obtain a detection result of whether the connector splice is in place.

[0110] On the basis of the above embodiments, the splice analysis module 902 is specifically configured to: input the sound signals into a first sound recognition model to obtain a sound recognition result; input the pressure signals into a pressure recognition model to obtain a pressure recognition result; The sound recognition result and the pressure recognition result in the same time interval are selected, if the sound recognition result and the pressure recognition result both represent that the connector is plugged in place, it is determined that the connector is plugged in place, otherwise, it is determined that the connector is not plugged in place, wherein the time interval corresponds to one sound recognition result and one pressure recognition result.

[0111] On the basis of the above embodiment, the plugging analysis module 902 is specifically configured to: extract a single sound sample from the sound signal, and extract a single pressure sample from the pressure signal; for each single sound sample, obtain a corresponding sound time interval; for each single pressure sample, obtain a corresponding pressure time interval; match the sound time interval and the pressure time interval, and take the single sound sample corresponding to the matched sound time interval as a target single sound sample; analyze the target single sound sample by using the second sound recognition model to obtain a detection result of whether the connector is plugged in place.

[0112] On the basis of the above embodiment, the sound time interval and the pressure time interval satisfy any one of the following conditions: the sound time interval at least partially overlaps with the pressure time interval; a time interval between a start time of the sound time interval and an end time of the pressure time interval is less than a first preset time interval; a time interval between an end time of the sound time interval and a start time of the pressure time interval is less than a second preset time interval.

[0113] On the basis of the above embodiment, the single sound sample is extracted from the sound signal, including: perform frequency domain signal enhancement processing on the sound signal to obtain an increased sound signal; convert the enhanced sound signal from the frequency domain to a time domain sound signal; extract a continuous signal greater than a sound threshold from the time domain sound signal as a single sound sample.

[0114] On the basis of the above embodiment, the single pressure sample is extracted from the pressure signal, including: obtain a basic pressure when holding the material and a plugging pressure when plugging the connector; determine a pressure threshold according to the basic pressure and the plugging pressure; extract a continuous pressure signal exceeding the pressure threshold from the pressure signal as a single pressure sample.

[0115] On the basis of the above embodiment, the device further includes a basic pressure determination module configured to: Collect historical pressure data within a preset time period prior to the insertion of the connector at at least one historical moment; Determine the underlying pressure based on historical pressures.

[0116] Figure 10 This is a schematic diagram of the detection module structure provided in an embodiment of this application, as shown below. Figure 10 As shown, the detection module includes: a processor 1001, a memory 1002, and a bus 1003; wherein: The processor 1001 and the memory 1002 communicate with each other through the bus 1003; The processor 1001 is used to call the program instructions in the memory 1002 to execute the methods provided in the above-described method embodiments, such as: acquiring sound signals collected by a sound acquisition module set within a preset distance range of the operating part, and pressure signals collected by a pressure acquisition module set at the force application position of the operating part; the operating part is a robotic arm or the hand of an operator performing the insertion operation; and performing insertion analysis based on the sound signal and pressure signal to obtain a detection result of whether the connector is inserted in place.

[0117] The processor 1001 can be an integrated circuit chip with signal processing capabilities. The processor 1001 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0118] The memory 1002 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0119] The embodiment discloses a computer program product, the computer program product comprises a computer program stored on a non-transitory computer readable storage medium, the computer program comprises program instructions, when the program instructions are executed by a computer, the computer can execute the method provided by each method embodiment, for example, comprising: acquiring a sound signal collected by a sound collection module arranged within a preset distance range of an operating part, and a pressure signal collected by a pressure collection module arranged at a force application position of the operating part; the operating part is a mechanical hand or a hand of a worker for performing a plugging operation; performing plugging analysis based on the sound signal and the pressure signal to obtain a detection result of whether a connecting piece is plugged in place.

[0120] The embodiment provides a non-transitory computer readable storage medium, the non-transitory computer readable storage medium stores computer instructions, the computer instructions enable the computer to execute the method provided by each method embodiment, for example, comprising: acquiring a sound signal collected by a sound collection module arranged within a preset distance range of an operating part, and a pressure signal collected by a pressure collection module arranged at a force application position of the operating part; the operating part is a mechanical hand or a hand of a worker for performing a plugging operation; performing plugging analysis based on the sound signal and the pressure signal to obtain a detection result of whether a connecting piece is plugged in place.

[0121] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other manners. The described device embodiments are merely illustrative. For example, the division of the units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0122] In addition, the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0123] Furthermore, each functional module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0124] In this document, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.

[0125] The above description is merely illustrative of the application and not in limitation of the principles of the application. Numerous modifications and adaptations thereof will be readily apparent to those skilled in the art without departing from the spirit and scope of the application as defined in the following claims.

Claims

1. A method of detecting plugging of a connection, characterized in that The method comprises the following steps: acquiring a sound signal collected by a sound collection module arranged within a preset distance range of an operating part, and a pressure signal collected by a pressure collection module arranged at a force application position of the operating part; the operating part is a mechanical hand or a hand of an operator for plugging operation; performing plugging analysis based on the sound signal and the pressure signal to obtain a detection result of whether the connector is plugged in place.

2. The method of claim 1, wherein, The method of performing plugging analysis based on the sound signal and the pressure signal to obtain a detection result of whether the connector is plugged in place comprises: inputting the sound signal into a first sound recognition model to obtain a sound recognition result; inputting the pressure signal into a pressure recognition model to obtain a pressure recognition result; selecting the sound recognition result and the pressure recognition result in the same time interval, and if both the sound recognition result and the pressure recognition result represent plugging in place, it is determined that the connector is plugged in place; otherwise, it is determined that the connector is not plugged in place; wherein the time interval corresponds to one sound recognition result and one pressure recognition result.

3. The method of claim 1, wherein, The method of performing plugging analysis based on the sound signal and the pressure signal to obtain a detection result of whether the connector is plugged in place comprises: extracting a single sound sample from the sound signal, and extracting a single pressure sample from the pressure signal; for each single sound sample, acquiring a corresponding sound time interval; for each single pressure sample, acquiring a corresponding pressure time interval; matching the sound time interval and the pressure time interval, and taking the single sound sample corresponding to the sound time interval matched successfully as a target single sound sample; analyzing the target single sound sample by using a second sound recognition model to obtain a detection result of whether the connector is plugged in place.

4. The method of claim 3, wherein, The sound time interval and the pressure time interval are matched successfully if any of the following conditions is met: the sound time interval and the pressure time interval at least partially overlap; the time interval between the start time of the sound time interval and the end time of the pressure time interval is less than a first preset time interval; the time interval between the end time of the sound time interval and the start time of the pressure time interval is less than a second preset time interval.

5. The method of claim 3, wherein, The method of extracting a single sound sample from the sound signal comprises: performing frequency domain signal enhancement processing on the sound signal to obtain an increased sound signal; converting the enhanced sound signal from the frequency domain to a time domain sound signal; extracting a continuous signal greater than a sound threshold value from the time domain sound signal as the single sound sample.

6. The method of claim 3, wherein, The method of extracting a single pressure sample from the pressure signal comprises: acquiring a basic pressure when holding a material and a plugging pressure when plugging a connector; determining a pressure threshold value according to the basic pressure and the plugging pressure; extracting a continuous pressure signal exceeding the pressure threshold value from the pressure signal as a single pressure sample.

7. The method of claim 6, wherein, The method further comprises: acquiring a historical pressure within a preset time length before plugging a connector at at least one historical time; determining the basic pressure based on the historical pressure.

8. A method of detecting insertion of a connector, characterized by, The detection module is applied to a detection device, the detection device comprising a pressure acquisition module, a sound acquisition module and the detection module; the pressure acquisition module and the sound acquisition module are in communication connection with the detection module; the pressure acquisition module and the sound acquisition module are arranged on a glove worn by an operator, and the pressure acquisition module is arranged on a position of the glove corresponding to a force application position of an operating part; the method comprises: acquiring a sound signal acquired by the sound acquisition module and a pressure signal acquired by the pressure acquisition module within a preset distance range of the operating part; the operating part is a hand of an operator performing a plugging operation; wherein the sound signal and the pressure signal are synchronously acquired; extracting a single sound sample from the sound signal and a single pressure sample from the pressure signal; performing plugging analysis on the single sound sample by using a second sound recognition module to obtain a sound recognition result; performing plugging analysis on the single pressure sample by using a pressure recognition model to obtain a pressure recognition result; obtaining a detection result of whether the connector is plugged in based on the sound recognition result and the pressure recognition result.

9. A connection plug-in detection device, characterized by comprise: a pressure acquisition module, a sound acquisition module and a detection module; the pressure acquisition module and the sound acquisition module are in communication connection with the detection module; the pressure acquisition module is arranged on a part of a mechanical hand or an operator gripping the connector, for acquiring a pressure signal and sending the pressure signal to the detection module; the sound acquisition module is arranged within a preset range from the connector, for acquiring a sound signal and sending the sound signal to the detection module; the detection module is used for executing the method according to any one of claims 1-8.

10. The detection device of claim 9, wherein, The detection device further comprises a hand wearing device; the hand wearing device is used for being worn on a hand of an operator; the sound acquisition module is arranged on a wearing body of the hand wearing device; the pressure acquisition module is arranged on a finger sleeve of the hand wearing device.

11. The detection device of claim 10, wherein, The wearing body comprises a glove and a sensor base; one side of the sensor base is detachably connected with the glove; the pressure acquisition module and the sound acquisition module are arranged on the other side of the sensor base.

12. The detection device of claim 11, wherein, The sound acquisition module is arranged in a first interdigital gap region of the glove.

13. The detection device of claim 9, wherein, The detection device further comprises a power supply module, a control module and a wireless transmission module; the power supply module is used for supplying power to the pressure acquisition module, the sound acquisition module, the control module and the wireless transmission module; the control module is used for receiving a pressure signal acquired by the pressure acquisition module and receiving a sound signal acquired by the sound acquisition module, and sending the pressure signal and the sound signal to the detection module through the wireless transmission module.

14. The detection device according to any one of claims 9 to 13, characterized in that The pressure acquisition module is a flexible thin film pressure sensor.

15. A non-transitory computer-readable storage medium, comprising: The non-transitory computer readable storage medium stores computer instructions, which, when executed by a computer, cause the computer to execute the method according to any one of claims 1-7. The non-transitory computer readable storage medium stores computer instructions, which, when executed by a computer, cause the computer to execute the method according to any one of claims 1-7.

16. A computer program product, characterised in that, comprising computer program instructions, which, when read and executed by a processor, perform the method according to any one of claims 1-7.

Images