System and method for connector assembly verification in a battery pack

Abstract

A method of verifying assembly of a connector, the method comprising capturing at least one audio signal from a footprint of an assembly line during assembly of the connector; and verifying the assembly of the connector based on the at least one audio signal.

Description

[0001] introduce

[0002] The information provided in this section is for the purpose of generally presenting the context of the present disclosure. From the description in this section, the work of the currently named inventors and the aspects of the description that may not constitute prior art at the time of filing should neither be explicitly nor implicitly admitted to be prior art of the present disclosure.

[0003] The present disclosure relates to assembly lines, and more particularly to systems and / or methods for verifying proper assembly of a connector within, for example, a battery pack.

[0004] Cooling hoses are essential components in battery electric vehicles (BEVs) as they play a key role in managing the thermal performance of the vehicle's battery pack and other critical components. However, after the battery pack is assembled, it may be difficult to access or replace any cooling components in the battery pack. Therefore, it is important to perform inline quality verification to ensure defect-free assembly of the battery pack.

[0005] The cooling hose may be assembled using one or more connectors, such as a quick connector (QC). The QC may be manually connected by an operator, and verification of proper engagement of the QC may be performed by visual inspection by the operator or another person on the assembly line. However, because the measurements of each connector may be small (e.g., approximately 10 mm), it may be difficult to verify proper seating of the connector using the naked eye. Summary of the invention

[0006] At least some example embodiments relate to a method of verifying assembly of a connector.

[0007] In some exemplary embodiments, the method includes capturing at least one audio signal from a footprint of an assembly line during assembly of the connector; and verifying the assembly of the connector based on the at least one audio signal.

[0008] In some exemplary embodiments, when a battery for an electric vehicle including a connector is assembled on an assembly line, the verification may verify the assembly in real time.

[0009] In some exemplary embodiments, the method further includes tracking a relative position of an operator and an audio sensing device in the occupied space, the audio sensing device being configured to capture at least one audio signal.

[0010] In some exemplary embodiments, tracking the relative position includes detecting a position of an operator in an occupied space; determining whether the position of the operator is aligned with a position of the audio sensing device; and, in response to determining that the position of the operator is misaligned with the position of the audio sensing device, moving the audio sensing device toward the position of the operator.

[0011] In some exemplary embodiments, detecting the position of the operator includes continuously detecting the position of the operator.

[0012] In some exemplary embodiments, determining whether the position of the operator is aligned with the position of the audio sensing device includes calculating an offset of a linear actuator associated with the audio sensing device based on the relative positions of the operator and the audio sensing device.

[0013] In some exemplary embodiments, capturing at least one audio signal includes capturing multiple audio signals from multiple audio sensing devices, respectively, during assembly of the connector, the multiple audio sensing devices being spaced apart from each other at different locations in the occupied space of the assembly line; and fusing the multiple audio signals to generate a final audio signal, wherein the verification is based on the final audio signal to verify the assembly of the connector.

[0014] In some exemplary embodiments, verifying the assembly of a connector includes determining at least one region of interest in at least one audio signal; extracting features from the at least one audio signal; and predicting whether the assembly of the connector is correctly assembled or incorrectly assembled based on the extracted features and training data.

[0015] In some exemplary embodiments, the extracting of features includes converting the at least one audio signal into a visual image that visually represents at least one region of interest in the at least one audio signal.

[0016] In some exemplary embodiments, predicting includes determining whether a connection matching signature is found in the extracted features by classifying the extracted features based on training data using artificial intelligence.

[0017] In some exemplary embodiments, the method further includes training a learning model based on features extracted from a training data set, the training data set including a signature of a properly mated connector and a signature of background noise.

[0018] In some exemplary embodiments, the method further includes outputting a verification signal indicating whether the assembly of the connector is correctly assembled or incorrectly assembled.

[0019] In some exemplary embodiments, the method further includes providing feedback to an operator based on the verification signal.

[0020] In some exemplary embodiments, providing feedback includes controlling an output device so that the output device outputs a first result in response to a verification signal indicating that the assembly of the connector is correctly assembled; and, in response to a verification signal indicating that the assembly of the connector is incorrectly assembled, controlling the output device so that the output device outputs a second result different from the first result.

[0021] In some exemplary embodiments, the method further includes controlling the collaborative robot to automatically perform assembly of the connector based on data captured from the guidance sensor; and, in response to the verification indicating that the assembly of the connector is incorrectly assembled, controlling the collaborative robot to reassemble the connector.

[0022] Other exemplary embodiments relate to a system configured to verify assembly of a connector.

[0023] In some exemplary embodiments, the system includes at least one audio capture device configured to capture at least one audio signal from an occupied space of an assembly line during assembly of the connector; and a controller configured to verify the assembly of the connector based on the at least one audio signal.

[0024] In some exemplary embodiments, the controller is configured to verify assembly of the connector in real time while a battery for an electric vehicle including the connector is assembled on an assembly line.

[0025] In some exemplary embodiments, the controller is configured to verify the assembly by determining at least one region of interest in at least one audio signal, extracting features from the at least one audio signal, and predicting whether the assembly of the connector is correctly assembled or incorrectly assembled based on the extracted features and training data.

[0026] In some exemplary embodiments, the controller is configured to extract the features by converting the at least one audio signal into a visual image that visually represents at least one region of interest in the at least one audio signal.

[0027] In some exemplary embodiments, the controller is configured to determine whether a connection mating signature is found in the extracted features by classifying the extracted features using artificial intelligence based on training data, thereby predicting whether the assembly of the connector is correctly assembled or incorrectly assembled.

[0028] Further areas of applicability of the present disclosure will become apparent from the detailed description, the claims and the accompanying drawings.The detailed description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present disclosure will become more fully understood from the detailed description and accompanying drawings, in which:

[0030] Figure 1 illustrates a battery assembly for a battery electric vehicle (BEV) according to some exemplary embodiments;

[0031] Figure 2A close-up view showing an engaged quick connector in a battery of a battery electric vehicle (BEV) according to some exemplary embodiments;

[0032] Figure 3 A system for verifying connector assembly according to a first exemplary embodiment is shown;

[0033] Figure 4 shows a hardware configuration of a controller of a system for verifying connector assembly according to some exemplary embodiments;

[0034] Figure 5 illustrates a software configuration of a controller of a system for verifying connector assembly according to some exemplary embodiments;

[0035] Figure 6 A method of verifying connector assembly according to some exemplary embodiments is shown;

[0036] Figure 7 A method of tracking an operator's location according to some exemplary embodiments is shown;

[0037] Figure 8 illustrates a verification module according to some exemplary embodiments;

[0038] Fig. 9 A method of performing a verification operation according to some exemplary embodiments is shown;

[0039] Fig.10 A system for verifying connector assembly according to a second exemplary embodiment is shown; and

[0040] Fig.11 A system for verifying connector assembly according to a third exemplary embodiment is shown.

[0041] Among the drawings, reference numerals may be repeated to identify similar and / or identical elements. DETAILED DESCRIPTION

[0042] Exemplary embodiments disclose a system and method for electronic verification of the engagement of a connector assembly. The verification can be performed in real time so that an operator can be provided with immediate feedback to correct any defects in the engagement of the connector before completing the assembly of the battery pack.

[0043] Figure 1 shows a battery assembly for a battery electric vehicle (BEV) according to some exemplary embodiments, Figure 2 A close-up view of an engaged quick connector in a battery of a battery electric vehicle (BEV) is shown, according to some exemplary embodiments.

[0044] refer to Figure 1 and Figure 2, current and future electric vehicle subsystems may increasingly rely on thermal regulation (e.g., heating and / or cooling) to achieve desired performance levels. Therefore, proper design and assembly of thermal management systems may be critical.

[0045] The vehicle may include one or more hoses 30 (or, alternatively, fluid lines) that circulate a liquid for heating and / or cooling, wherein each hose is formed of a plurality of interconnected segments. For example, the liquid may be a 50 / 50 mixture of water and an antifreeze agent such as ethylene glycol. However, exemplary embodiments are not limited thereto. The segments of each hose 30 may be formed by interconnecting the segments via connectors 10-1 to 10-n. The connectors 10-1 to 10-n may be quick connectors (QCs). QCs are a common solution for assembling these hoses 30 during vehicle integration and may help achieve an efficient and safe connection. As a result, the more hoses 30 a vehicle has, the more connectors 10-1 to 10-n may be needed to keep the vehicle operating safely and efficiently.

[0046] The connectors 10-1 to 10-n are typically manually installed during vehicle assembly by applying manual pressure from an operator. However, as the size of subsystems such as the battery 20 increases to increase the energy storage capacity and driving range of the vehicle, it may become more difficult to ensure that the connectors 10-1 to 10-n are properly mated due to the increased number of connectors 10-1 to 10-n and the reduced space available for operators to manually manipulate such connectors 10-1 to 10-n. In addition, in hybrid vehicles, the electric drive components must coexist with the internal combustion engine and its associated components including fuel lines, filters, and fuel tanks, which may require further reductions in component complexity and increased density, and in these complex environments, the correct connection of the various connectors 10-1 to 10-n may be particularly challenging to manufacture and verify. However, reliable connections are critical because any leakage caused by incorrect assembly may pose a significant safety threat. For example, improperly mated connectors 10 - 1 to 10 - n may be difficult to repair after the battery 20 is assembled, and improperly mated connectors 10 - 1 to 10 - n may cause fluid to leak from the hose 30 , which may cause an electrical fire in the battery 20 .

[0047] Figure 3 A system for verifying connector assembly according to a first exemplary embodiment is shown.

[0048] refer to Figure 3, a system for verifying connector assembly may include a position tracking sensor 110, a track 120, at least one actuator 130, an audio sensing device 140, an automated guided vehicle (AGV) 150, a tracking marker 160, a feedback device 170, and a controller 1000.

[0049] The position tracking sensor 110 may be configured to sense the position of the operator and output position data indicating the operator. For example, the position tracking sensor 110 may be installed above the occupied space of the assembly station, wherein the automated guided vehicle 150 is configured to be stationary and may be configured to sense the position of the operator based on data collected from the occupied space. In some exemplary embodiments, the position tracking sensor 110 is a light detection and ranging (LiDAR) sensor configured to detect the operator using a laser. For example, LiDAR can be used to identify the operator as a mobile obstacle at the assembly station based on laser scanning data obtained by the position tracking sensor 110. In other exemplary embodiments, the position tracking sensor 110 may be a camera configured to capture images in the occupied space and detect the position of the operator from the captured images. For example, the position tracking sensor 110 may generate image data that may be used to detect the operator using a computer vision algorithm such as a deep classification (simple online and real-time tracking) model.

[0050] The track 120 may be made of metal and may extend longitudinally along the length of the assembly station and may include a track that allows at least one actuator 130 to move linearly thereon. The at least one actuator 130 may include a pair of 2D linear actuators that move longitudinally along the track 120 in the X-axis direction and also extend or contract to move up and down along the Z-axis direction. The 2D linear actuator may convert rotation into linear motion. For example, the 2D linear actuator may have a rack and pinion design.

[0051] The audio sensing device 140 may be attached to one end of the at least one actuator 130. The audio sensing device 140 may include an audio sensor and a long distance amplifier.

[0052] The audio sensor may be a microphone, which may be a transducer that converts sound waves into electrical signals. The microphone may be a directional microphone in which the microphone sensitivity is in one direction, or may be an omnidirectional microphone in which the microphone sensitivity may be 360 ​​degrees. The long-distance amplifier may include a parabolic reflector that collects and focuses the sound waves onto the transducer to help pick up the sound waves at a distance, and may thus enable the audio sensing device 140 to be positioned away from the operator.

[0053] The automated guided vehicle (AGV) 150 may be equipped with a navigation and control system that allows the AGV 150 to move and perform tasks without human intervention. The AGV 150 may be used as a conveyor to carry the batteries 20 and other components of the vehicle along an assembly line.

[0054] The tracking marker 160 may be a tracker placed, for example, on a safety helmet worn by the operator. However, exemplary embodiments are not limited thereto, and the tracking marker 160 may also be placed at any location on the operator that can be detected by the position tracking sensor 110. In some exemplary embodiments, the tracking marker 160 may be an April tag or a QR code. In other exemplary embodiments, the tracking marker 160 may be omitted, and the position tracking sensor 110 may sense the position of the operator using, for example, a machine learning algorithm. For example, the position tracking sensor 110 may employ a deep classification (simple online and real-time tracking) model to track the operator's movement in the occupied space of the AVG 150.

[0055] Feedback device 170 may be a display device that outputs a visual display to an operator or other personnel associated with the assembly line. For example, feedback device 170 may output a green light or a red light based on a verification signal input thereto. In other exemplary embodiments, feedback device 170 may output other forms of feedback to an operator or other personnel associated with the assembly line. For example, feedback device 170 may output different sounds based on the verification signal, or may output different patterns of vibrations to an operator or other personnel associated with the assembly line based on a verification signal input thereto.

[0056] The controller 1000 may control at least a portion of a system for verifying connector assembly. For example, the controller 1000 may collect position data from the position tracking sensor 110 and control at least one actuator 130 based on the position data. In addition, as discussed in more detail below, the controller 1000 may also collect audio data from the audio sensing device 140 and may perform a verification operation based on the collected audio data. The controller 1000 may control the feedback device 170 based on the results of the verification operation. In some exemplary embodiments, the controller 1000 may not control the AGV 150, so that the AGV 150 may be controlled by a separate controller. In other exemplary embodiments, the controller 1000 may also control the AGV 150 to move the AGV 150 along the assembly line.

[0057] Figure 4 A hardware configuration of a controller of a system for verifying connector assembly according to an exemplary embodiment is shown.

[0058] refer to Figure 4, the controller 1000 may include a processing circuit 1100, such as at least one processor, at least one communication bus 1200, a memory 1300, and at least one network interface (I / F) 1400. In addition, the controller 1000 may further include at least one input / output (I / O) device 1500 (e.g., a keyboard, a monitor, a touch screen, a mouse, a microphone, a camera, a speaker, etc.), etc., but the exemplary embodiment is not limited thereto.

[0059] In at least one exemplary embodiment, the processing circuit 1100 may include a processor core, a distributed processor, or a networked processor. The processing circuit 1100 may be configured as one or more units of the controller 1000, and thereby enable the controller 1000 to perform various operations. The processing circuit 1100 may be configured to perform processing by extracting program code (e.g., computer-readable instructions) and data from the memory 1300 to process them, thereby performing dedicated control and functions of the controller 1000. Once the dedicated program instructions are loaded onto, for example, at least one processor, the processing circuit 1100 may execute the dedicated program instructions, thereby transforming the processing circuit 1100 into a dedicated processor.

[0060] In at least one exemplary embodiment, at least one communication bus 1200 may enable communication and / or data transfer between units of the controller 1000. The communication bus 1200 may be implemented using a high-speed serial bus, a parallel bus, and / or any other suitable communication technology. According to some exemplary embodiments, the controller 1000 may include multiple communication buses (not shown).

[0061] In at least one exemplary embodiment, memory 1300 may be a non-transitory computer-readable storage medium and may include random access memory (RAM), read-only memory (ROM), and / or a permanent mass storage device such as a disk drive or a solid-state drive. Stored in memory 1300 are program codes (i.e., computer-readable instructions) associated with verifying connector assembly and controlling at least one network interface 140 and / or at least one I / O device 1500, etc., as described herein.

[0062] Such software units may be loaded from a non-transitory computer-readable storage medium independent of the memory 1300 using a drive mechanism (not shown) connected to the controller 1000 or via at least one network interface 1400 and / or at least one I / O device 1500 or the like.

[0063] Figure 5 A software configuration of a controller of a system for verifying connector assembly according to an exemplary embodiment is shown.

[0064] refer to Figure 5 The processing circuit 1100 of the controller 1000 can execute program code (e.g., computer-readable instructions) extracted from, for example, the memory 1300 to convert the processing circuit 1100 into a dedicated processor, which includes a position tracking module 1110, a verification module 1120, and a feedback and control module 1130, which together perform a method of verifying the assembly of the connector. Figure 6 The operation of each of modules 1110-1130 is discussed.

[0065] Although the above functions are described using the above modules 1110-1130 to increase the clarity of the description, the processing circuit 1100 is not intended to be limited to only these functional units. For example, in one or more exemplary embodiments, various operations and / or functions of the functional units can be performed by other functional units in the functional units. In addition, the processing circuit 1100 can perform the operations and / or functions of various functional units without subdividing the operations and / or functions of the computer processing unit into these various functional units.

[0066] Figure 6 A method of verifying connector assembly according to some exemplary embodiments is shown.

[0067] refer to Figure 4-6 In operation S100, the controller 1000 may perform position tracking. For example, the position tracking module 1110 may receive position data from the position tracking sensor 110 and track the position of the operator based on the received position data.

[0068] Figure 7 A method of tracking a position of an operator according to an exemplary embodiment is shown.

[0069] refer to Figure 4-7 In operation S110, the location tracking module 1110 may determine whether a trigger has been detected. For example, in some exemplary embodiments, the trigger may occur when the AGV 150 has stopped at a preset (or, desired) position of the occupied space defining the assembly station, or may occur when the presence of an operator is detected in the occupied space of the AGV 150. However, exemplary embodiments are not limited thereto. If the location tracking module 1110 determines that the trigger has not occurred, the location tracking module 1110 may wait until a trigger is received.

[0070] In operation S120, when a trigger is detected, the position tracking module 1110 can continuously track the position of the operator. For example, the position tracking sensor 110 can continuously generate position data indicating the position of the operator based on data continuously collected from the occupied space of the AGV 150, and provide the generated position data to the controller 1000 (e.g., the position tracking module 1110).

[0071] In operation S130, the position tracking module 1110 can determine whether the audio sensing device 140 is aligned with the operator. For example, the position tracking module 1110 can determine the relative difference between the position of the audio sensing device 140 and the position of the operator, and determine whether the audio sensing device 140 is aligned with the operator based on the difference therebetween. If the position tracking module 1110 determines that the audio sensing device 140 is sufficiently aligned with the operator, the position tracking module 1110 can no longer move the audio sensor. Instead, the position tracking module 1110 returns to operation S120 and continuously monitors the position of the operator to determine whether the operator has moved position in the occupied space of the AGV 150. If the position tracking module 1110 determines that the position movement of the operator has exceeded a threshold, then the position tracking module 1110 can determine that the audio sensing device 140 has been misaligned with the operator and proceeds to operation S140.

[0072] Based on empirical measurements of audio data, a certain range of positions may be considered sufficiently aligned, wherein audio signals captured within that range of positions may accurately identify the signature of properly mated connectors 10-1 to 10-n using the same or similar type of audio sensing device 140.

[0073] In operation S140, if the position tracking module 1110 determines that the audio sensing device 140 is not sufficiently aligned with the operator, the position tracking module 1110 calculates an offset (eg, an X-axis offset and a Z-axis offset) between the position of the audio sensing device 140 and the position of the operator.

[0074] In operation S150, the position tracking module 1110 may instruct the at least one actuator 130 to move the audio sensing device 140 to align the audio sensing device 140 with the operator. For example, the position tracking module 1110 may instruct the at least one actuator 130 to move in one or more of the X direction (e.g., horizontally along the length of the occupied space of the AGV 150) or the Z direction (e.g., vertically above the occupied space of the AGV 150). Once the audio sensing device 140 is sufficiently aligned with the operator, the position tracking module 1110 may generate an alignment signal and may return to operation S120 and continue to track the position of the operator to determine whether the operator has shifted more than a threshold in the occupied space of the AGV 150.

[0075] Return to reference Figure 6 In operation S200, the controller 1000 may perform a verification operation. For example, the verification module 1120 may receive an alignment signal from the position tracking module 1110 indicating that the audio sensing device 140 is sufficiently aligned with the operator, and then may perform a verification operation.

[0076] Figure 8 A verification module according to an exemplary embodiment is shown. Fig. 9 A method of performing a verification operation according to an exemplary embodiment is shown.

[0077] refer to Figure 8 and Fig. 9 In operation S210 , the audio receiving module 1122 included in the verification module 1120 may buffer and / or process the audio signal received from the audio sensing device 140 .

[0078] In operation S220, the feature extraction module 1124 included in the verification module 1120 may receive the processed audio signal, may identify a region of interest in the received audio signal, may perform feature extraction on the identified region of interest in the audio signal, and output the extracted features.

[0079] The region of interest may include one or more frequency and / or amplitude ranges that have been previously identified through empirical research as containing the signature of the sound produced by connectors 10-1 to 10-n when properly mated, and the region of interest may therefore be identified by analyzing the energy distribution of audio signals captured when an operator mates connectors 10-1 to 10-n.

[0080] The verification module 1120 may perform feature extraction by inputting the identified region(s) of interest in the captured audio signal into a series of signal processing layers that extract features that may be used to predict whether the feature contains a connection fit signature. The verification module 1120 may generate a visual image of the features extracted from the region of interest in the captured audio signal, wherein the image may contain a connection fit signature.

[0081] In operation S230, the prediction module 1126 included in the verification module 1120 may predict, for example, whether one of the connectors 10-1 to 10-n is correctly assembled based on whether the connector mating signature is found in the features extracted based on the training data, and may output a verification signal based thereon. For example, the prediction module 1126 may analyze a visual image of the extracted features to determine whether the image contains a known visual representation of the connector mating signature.

[0082] For example, the prediction module 1126 can utilize artificial intelligence to perform classification using a convolutional neural network (CNN), which is based on a pre-trained deep learning model. The prediction module 1126 can utilize a tracking model to perform classification based on a classification score.

[0083] The training data may be assembled by the training module 1128. For example, the training module 1128 may receive a recorded data set, train a model based on the recorded data set, deploy the trained model, test the trained model, and monitor and record the results. The model may be trained based on features extracted from the data set, the data set being composed of features of multiple categories, including signatures of correctly mated connectors 10-1 to 10-n and signatures of non-connector categories (background noise, speech, or any other abnormal sound). When the model accurately predicts correctly assembled connectors 10-1 to 10-n, the recorded results may include true positives (TP), when the model fails to accurately predict correctly assembled connectors 10-1 to 10-n in response to other sounds, the recorded results may include false positives (FP), when the model accurately predicts incorrectly assembled connectors 10-1 to 10-n, the recorded results may include true negatives (TN), and when the model fails to accurately predict incorrectly assembled connectors 10-1 to 10-n, the recorded results may include false negatives (FN). The training module 1128 can retrain the model based on the recorded results by, for example, adding TP and FN to the connector data set and adding FP and TN to the non-connector data set.

[0084] Return to reference Figure 6In operation S300, the controller 1000 may provide feedback or perform other controls based on the verification signal. For example, the feedback and control module 1130 may receive the verification signal from the position tracking module 1110, and may control the feedback device 170 based on the verification signal. For example, when the verification signal indicates that one of the connectors 10-1 to 10-n assembled by the operator is a correctly assembled connector, the feedback and control module 1130 may control the feedback device 170 to provide positive feedback to the operator. On the contrary, when the verification signal indicates that one of the connectors 10-1 to 10-n assembled by the operator is an incorrectly assembled connector, the feedback and control module 1130 may control the feedback device 170 to provide negative feedback to the operator. In some exemplary embodiments, the feedback device 170 may output a first color (e.g., green) light as positive feedback, and may output a second color (e.g., red) light as negative feedback. However, exemplary embodiments are not limited thereto. In other exemplary embodiments, the feedback device 170 may also output a first sound as positive feedback, and may output a second sound as negative feedback. In other exemplary embodiments, the feedback device 170 may also provide tactile feedback, such as providing a vibration sensation to the operator. For example, the feedback device 170 may provide a first vibration pattern to the operator as positive feedback, and may provide a second vibration pattern to the operator as negative feedback. In other exemplary embodiments, the feedback device 170 may also output only a single pattern vibration, such as when the verification signal indicates a correctly assembled connector, and when the connector is an incorrectly assembled connector, no feedback may be output. However, exemplary embodiments are not limited to this.

[0085] The controller 1000 may perform the position tracking operation S100 and the verification operator S200 in real time, and may provide real-time feedback to the operator in operation S300 based on the result of the verification operation. Therefore, the controller 1000 may enable the operator to immediately correct the incorrectly assembled connectors 10-1 to 10-n before the final assembly of the battery 20. Therefore, the controller 1000 may ensure the correct connection of the respective connectors 10-1 to 10-n before the final assembly of the battery 20, and thus may avoid significant safety threats caused by incorrect mating of the connectors and significant costs associated with preparing the connectors after the final assembly of the battery 20.

[0086] Fig.10 A system for verifying connector assembly according to a second exemplary embodiment is shown.

[0087] refer to Fig.10According to another exemplary embodiment, a system for verifying connector assembly may include a fixing beam 210 instead of a track, and may include a plurality of audio sensing devices 140-1 to 140-n attached to the fixing beam 210. Each of the plurality of audio sensing devices 140-1 to 140-n may be substantially similar to the audio sensing device 140.

[0088] Instead of tracking the location of the operator and moving the sound sensor to align the sound sensor with the operator, sound signals can be collected from multiple sound sensing devices 140-1 to 140-n arranged along the fixed beam 210 in the occupied space of the AGV 150. The verification module 1120, such as the audio receiving module 1122, can fuse the audio data from the multiple audio sensing devices 140-1 to 140-n by synchronizing the audio signals from the multiple audio sensing devices 140-1 to 140-n and processing the synchronized signals into a single fused audio signal to detect a connection matching signature. When the connection matching signature is detected, the audio receiving module 1122 can calculate the sound that is delayed in time to arrive at each of the multiple audio sensing devices 140-1 to 140-n, and can use triangulation methods to identify the location of the sound source. For example, the audio receiving module 1122 may utilize time of flight and signal triangulation methods to determine the location and / or source of the received sound, thereby identifying that the connectors 10-1 to 10-n are being assembled and, therefore, identifying situations in which the connectors 10-1 to 10-n may be improperly mated.

[0089] The system for verifying connector assembly according to the second exemplary embodiment may be substantially the same as the system for verifying connector assembly according to the first exemplary embodiment, except that a plurality of fixed audio sensing devices 140-1 to 140-n are attached to the fixed beam 210 and the position tracking sensor 110 is omitted. Therefore, for the sake of brevity, repeated descriptions of common functions will be omitted.

[0090] Fig.11 A system for verifying connector assembly according to a third exemplary embodiment is shown.

[0091] refer to Fig.11, a system for verifying connector assembly according to a third exemplary embodiment may include a collaborative robot 300. The collaborative robot 300 may include a guidance sensor 310 and an audio sensing device 340 attached to an end effector of the collaborative robot 300. The guidance sensor 310 may be a 2D camera configured to perform visual guidance of the collaborative robot 300. The audio sensing device 340 may be substantially the same as the audio sensing device 140, however, the audio sensing device 340 may not include a long-range amplifier because it may not be necessary to position the audio sensing device 140 at a distance away from the collaborative robot 300.

[0092] The collaborative robot 300 may be controlled by the controller 1000 based on data provided by, for example, the feedback and control module 1130 , or may be controlled by a separate controller. The collaborative robot 300 may automatically assemble the connectors 10 - 1 to 10 - n based on data captured from the guide sensor 310 .

[0093] The system for verifying connector assembly according to the third exemplary embodiment may be substantially the same as the system for verifying connector assembly according to the first exemplary embodiment, except for having a collaborative robot 300 instead of a position tracking sensor 110, a track 120, at least one actuator 130, and an audio sensing device 140.

[0094] For example, the controller 1000 may predict whether the collaborative robot 300 has correctly assembled the connectors 10-1 to 10-n in substantially the same manner as discussed above. Therefore, for the sake of brevity, repeated descriptions of common functions will be omitted.

[0095] In some exemplary embodiments, when the controller 1000 (i.e., the verification module 1120) predicts that the collaborative robot 300 has incorrectly assembled one or more of the connectors, the feedback and control module 1130 can control the collaborative robot 300 to automatically reconnect any incorrectly assembled connectors 10-1 to 10-n.

[0096] In addition, in some exemplary embodiments, the controller 1000 (i.e., the feedback and control module 1130) can control the display device 170 in substantially the same manner as the system for verifying connector assembly according to the first exemplary embodiment. For example, the feedback and control module 1130 can provide visual, auditory, and / or tactile feedback to the operator of the collaborative robot 300 or other persons on the assembly line. However, the exemplary embodiments are not limited thereto, and in exemplary embodiments, the display device 170 can be omitted, and the feedback and control module 1130 can also only control the collaborative robot 300 to reconnect any incorrectly assembled connectors.

[0097] As described above, exemplary embodiments can help ensure the correct connection of each connector 10-1 to 10-n before the final assembly of the battery 20, and can therefore avoid leakage and significant safety threats caused by incorrect mating of the connector. In addition, exemplary embodiments can provide real-time feedback to operators or other people on the assembly line before assembling the battery 20.

[0098] Some of the aspects explained above are summarized below by means of numbered examples.

[0099] Example 1. A method of verifying assembly of a connector, the method comprising:

[0100] capturing at least one audio signal from a footprint of an assembly line during assembly of the connector; and

[0101] Assembly of the connector is verified based on the at least one audio signal.

[0102] Example 2. The method according to Example 1, wherein the verification verifies the assembly in real time while the battery for an electric vehicle including the connector is assembled on the assembly line.

[0103] Example 3. The method according to Example 1, further comprising:

[0104] A relative position of an operator and an audio sensing device in the occupied space is tracked, the audio sensing device being configured to capture the at least one audio signal.

[0105] Example 4. The method of Example 3, wherein tracking the relative position comprises:

[0106] Detecting the operator’s position in an occupied space;

[0107] determining whether the operator's position is aligned with the position of the audio sensing device; and

[0108] In response to determining that the position of the operator is misaligned with the position of the audio sensing device, the audio sensing device is moved toward the position of the operator.

[0109] Example 5. The method of Example 4, wherein detecting the position of the operator comprises:

[0110] The position of the operator is continuously detected.

[0111] Example 6. The method of Example 4, wherein determining whether the position of the operator is aligned with the position of the audio sensing device comprises:

[0112] An offset of a linear actuator associated with the audio sensing device is calculated based on the relative position of the operator and the audio sensing device.

[0113] Example 7. The method of Example 1, wherein capturing the at least one audio signal comprises:

[0114] During assembly of the connector, capturing a plurality of audio signals respectively from a plurality of audio sensing devices spaced apart from one another at different locations in a footprint of the assembly line; and

[0115] The multiple audio signals are fused to generate a final audio signal, wherein:

[0116] The verification verifies assembly of the connector based on the final audio signal.

[0117] Example 8. The method of Example 1, wherein verifying the assembly of the connector comprises:

[0118] determining at least one region of interest in the at least one audio signal;

[0119] extracting features from the at least one audio signal; and

[0120] Whether the assembly of the connector is correctly assembled or incorrectly assembled is predicted based on the extracted features and the training data.

[0121] Example 9. The method according to Example 8, wherein the extracting of the features comprises:

[0122] The at least one audio signal is converted into a visual image, the visual image visually representing at least one region of interest in the at least one audio signal.

[0123] Example 10. The method of Example 8, wherein the predicting comprises:

[0124] A determination is made whether a connection matching signature is found in the extracted features by classifying the extracted features based on the training data using artificial intelligence.

[0125] Example 11. The method according to Example 10, further comprising:

[0126] The learning model is trained based on features extracted from a training dataset that includes signatures of properly mated connectors and signatures of background noise.

[0127] Example 12. The method according to Example 8, further comprising:

[0128] A verification signal is outputted, the verification signal indicating whether the connector is correctly assembled or incorrectly assembled.

[0129] Example 13. The method according to Example 12, further comprising:

[0130] Feedback is provided to an operator based on the verification signal.

[0131] Example 14. The method of Example 13, wherein providing feedback comprises:

[0132] controlling an output device so that the output device outputs a first result in response to a verification signal indicating that the assembly of the connector is correctly assembled; and

[0133] The output device is controlled so that the output device outputs a second result different from the first result in response to a verification signal indicating that the assembly of the connector is incorrectly assembled.

[0134] Example 15. The method according to Example 1, further comprising:

[0135] controlling the collaborative robot to automatically perform assembly of the connector based on data captured from the guidance sensor; and

[0136] In response to the verifying indicating that the assembly of the connector is incorrectly assembled, the collaborative robot is controlled to reassemble the connector.

[0137] Example 16. A system configured to verify assembly of a connector, the system comprising:

[0138] at least one audio capture device configured to capture at least one audio signal from a footprint of an assembly line during assembly of the connector; and

[0139] A controller is configured to verify assembly of the connector based on the at least one audio signal.

[0140] Example 17. The system of Example 16, wherein the controller is configured to verify assembly of the connector in real time when a battery for an electric vehicle including the connector is assembled on the assembly line.

[0141] Example 18. The system of Example 16, wherein the controller is configured to verify the assembly by:

[0142] determining at least one region of interest in the at least one audio signal,

[0143] extracting features from at least one audio signal, and

[0144] Whether the assembly of the connector is correctly assembled or incorrectly assembled is predicted based on the extracted features and the training data.

[0145] Example 19. The system of Example 18, wherein the controller is configured to extract features by:

[0146] The at least one audio signal is converted into a visual image, the visual image visually representing at least one region of interest in the at least one audio signal.

[0147] Example 20. The system of Example 18, wherein the controller is configured to predict whether the assembly of the connector is correctly assembled or incorrectly assembled by:

[0148] A determination is made whether a connection matching signature is found in the extracted features by classifying the extracted features based on the training data using artificial intelligence.

[0149] The foregoing description is essentially illustrative only, and there is no intention to limit the present disclosure, its application or use. The broad teachings of the present disclosure can be implemented in various forms. Therefore, although the present disclosure includes specific examples, the true scope of the present disclosure should not be limited thereto, because after studying the drawings, the specification and the appended claims, other modifications can be made apparent. It should be understood that one or more steps in the method can also be performed in different orders (or simultaneously) without changing the principles of the present disclosure. In addition, although each of the embodiments is described above as having certain features, any one or more of those features described with reference to any embodiment of the present invention can also be implemented in the features of any of the other embodiments and / or combined with the features of any of the other embodiments, even if the combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and the permutation of one or more embodiments to each other remains within the scope of the present disclosure.

[0150] Various terms are used to describe the spatial and functional relationships between elements (e.g., between modules, circuit elements, semiconductor layers, etc.), including "connected," "engaged," "coupled," "adjacent," "next to," "on top," "above," "below," and "disposed." Unless explicitly described as "direct," when describing the relationship between a first and a second element in the above disclosure, the relationship can be a direct relationship in which there are no other intermediate elements between the first and second elements, but can also be an indirect relationship in which there are one or more intermediate elements (spatially or functionally) between the first and second elements. As used herein, the phrase at least one of A, B, and C should be interpreted as meaning a logical (A or B or C) using a non-exclusive logical "or," and should not be interpreted as meaning "at least one of A, at least one of B, and at least one of C."

[0151] In the drawings, the direction of the arrow, as indicated by the arrow, generally represents the flow of information (e.g., data or instructions) of interest to the illustration. For example, when component A and component B exchange various information, but the information transmitted from component A to component B is relevant to the illustration, the arrow may point from component A to component B. The unidirectional arrow does not imply that no other information is transmitted from component B to component A. In addition, for information sent from component A to component B, component B may also send a request for the information or a confirmation of receipt of the information to component A.

[0152] In this application, including the following definitions, the term "module" or the term "controller" may be replaced with the term "circuit". The term "module" may refer to, be part of, or include: an application specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor circuit (shared, dedicated, or group) that executes code; a memory circuit (shared, dedicated, or group) that stores code executed by the processor circuit; other suitable hardware components that provide the functionality; or a combination of some or all of the above, such as in a system on a chip.

[0153] The module may include one or more interface circuits. In some examples, the interface circuit may include a wired or wireless interface connected to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functions of any given module of the present disclosure may be distributed in multiple modules connected via interface circuits. For example, multiple modules may allow load balancing. In other examples, a server (also referred to as a remote or cloud) module may perform some functions on behalf of a client module.

[0154] The term "code" as used above may include software, firmware and / or microcode, and may refer to a program, a routine, a function, a class, a data structure and / or an object. The term "shared processor circuit" covers a single processor circuit that executes some or all of the code from multiple modules. The term "group processor circuit" covers a processor circuit that executes some or all of the code from one or more modules in combination with an additional processor circuit. References to multi-processor circuits cover multi-processor circuits on discrete dies, multi-processor circuits on a single die, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above. The term "shared memory circuit" covers a single memory circuit that stores some or all of the code from multiple modules. The term "group memory circuit" covers a memory circuit that stores some or all of the code from one or more modules in combination with additional memory.

[0155] The term "memory circuit" is a subset of the term "computer-readable medium". The term "computer-readable medium" as used herein does not encompass transient electrical or electromagnetic signals propagated through a medium (such as on a carrier wave). Therefore, the term "computer-readable medium" may be considered to be both tangible and non-transitory. Non-limiting examples of non-transitory, tangible computer-readable media are non-volatile memory circuits (such as flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile memory circuits (such as static random access memory circuits or dynamic random access memory circuits), magnetic storage media (such as analog or digital magnetic tapes or hard drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0156] The apparatus and methods described in this application may be implemented in part or in whole by a special purpose computer created by configuring a general purpose computer to perform one or more specific functions contained in a computer program. The above-mentioned function blocks, flow chart components and other elements are used as software specifications, which can be converted into a computer program by routine work of a skilled technician or programmer.

[0157] The computer program includes processor executable instructions stored on at least one non-transitory tangible computer readable medium. The computer program may also include or rely on stored data. The computer program may include a basic input / output system (BIOS) that interacts with the hardware of the special-purpose computer, device drivers that interact with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0158] A computer program may include: a descriptive text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation); (ii) assembly code; (iii) object code generated by a compiler from a source code; (iv) source code executed by an interpreter; (v) source code compiled and executed by a just-in-time compiler, etc. For example, a computer program may be used from languages ​​including C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, Fortran, Perl, Pascal, Culli, OCaml, HTML5 (Hypertext Markup Language, Revision 5), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Visual Lua, MATLAB, SIMULINK, and The syntax of the language is used to write source code.

Claims

1. A method for verifying the assembly of a connector, the method comprising: capturing at least one audio signal from a footprint of an assembly line during assembly of the connector; as well as Assembly of the connector is verified based on the at least one audio signal.

2. The method according to claim 1, wherein: The verification verifies the assembly in real time while the battery for an electric vehicle including the connector is assembled on the assembly line.

3. The method according to claim 1, further comprising: A relative position of an operator and an audio sensing device in the occupied space is tracked, the audio sensing device being configured to capture the at least one audio signal.

4. The method according to claim 3, wherein: Tracking the relative position includes: Detecting the operator’s position in an occupied space; determining whether the operator's position is aligned with the position of the audio sensing device; and In response to determining that the position of the operator is misaligned with the position of the audio sensing device, the audio sensing device is moved toward the position of the operator.

5. The method according to claim 4, wherein: Detecting the position of the operator includes: The position of the operator is continuously detected.

6. The method according to claim 4, wherein: Determining whether the position of the operator is aligned with the position of the audio sensing device includes: An offset of a linear actuator associated with the audio sensing device is calculated based on the relative position of the operator and the audio sensing device.

7. The method according to claim 1, wherein: Capturing the at least one audio signal comprises: During assembly of the connector, capturing a plurality of audio signals respectively from a plurality of audio sensing devices spaced apart from one another at different locations in a footprint of the assembly line; and The multiple audio signals are fused to generate a final audio signal, wherein: The verification verifies assembly of the connector based on the final audio signal.

8. The method according to claim 1, wherein: Verifying assembly of the connector includes: determining at least one region of interest in the at least one audio signal; extracting features from the at least one audio signal; and Whether the assembly of the connector is correctly assembled or incorrectly assembled is predicted based on the extracted features and the training data.

9. The method according to claim 8, wherein: The feature extraction includes: The at least one audio signal is converted into a visual image, the visual image visually representing at least one region of interest in the at least one audio signal.

10. The method according to claim 8, wherein: The forecasts include: A determination is made whether a connection matching signature is found in the extracted features by classifying the extracted features based on the training data using artificial intelligence.

Images