Joining state learning device and joining state inference device

Abstract

Provided is a device and the like with which it is possible to improve the accuracy of inference of the joining state of a plurality of workpieces. On the basis of a plurality of sets of data for learning, a learning model is caused to learn the correlation between the input data for learning and joining states of a plurality of workpieces. The input data for learning includes data representing a segmented spectrogram obtained by dividing, into each of a plurality of frequency regions and into each of a plurality of time regions, a spectrogram of a vibration signal generated during one joining process of a plurality of workpieces W1 and W2 performed by an ultrasonic joining device. Segmented spectral data at the time of joining the workpieces W1 and W2 for which the joining state is unknown is input to the trained model as input data for inference. Consequently, the result of the inference of the joining state of the plurality of workpieces is acquired as an output from the trained model.

Description

Bonding condition learning device and bonding condition estimation device

[0001] The present invention relates to a technique for estimating the bonding state of a plurality of workpieces made of various materials such as metals using ultrasonic vibrations.

[0002] The present applicant has proposed a technique for joining workpieces using ultrasonic complex vibration (see, for example, Patent Document 1).

[0003] Patent No. 7219495

[0004] It is important to estimate the bonding state of multiple workpieces.

[0005] Therefore, an object of the present invention is to provide an apparatus etc. that can improve the accuracy of estimating the bonding state of multiple workpieces.

[0006] The bonding state learning device of the present invention includes: a learning data acquisition unit that acquires, as learning input data, multiple sets of learning data including data representing divided spectrograms in which a spectrogram of a vibration signal generated during the bonding process of multiple workpieces using an ultrasonic bonding device is divided into each of multiple frequency domains and each of multiple time domains; and a model generation unit that generates a trained model by having a learning model learn the correlation between the learning input data and the bonding state of the multiple workpieces based on the multiple sets of learning data acquired by the learning data acquisition unit.

[0007] The bonding state estimation device of the present invention includes an estimation input data acquisition unit that acquires, as estimation input data, divided spectral data in which spectral data of a vibration signal generated during the bonding process of a plurality of workpieces by an ultrasonic bonding device is divided into each of a plurality of frequency domains and each of a plurality of time domains, and a bonding state estimation unit that inputs the input data acquired by the estimation input data acquisition unit into the trained model generated by the bonding state learning device, and thereby acquires an estimation result of the bonding state of the plurality of workpieces as an output from the trained model.

[0008] FIG. 1 is a diagram illustrating the configuration of an ultrasonic bonding device. FIG. 2 is a diagram illustrating the configuration of a bonded state estimation device according to one embodiment of the present invention. FIG. 3 is a flowchart relating to the functions of an ultrasonic bonding device. FIG. 4 is a flowchart illustrating the procedure of a bonded state learning method according to one embodiment of the present invention. FIG. 5 is a flowchart illustrating the procedure of a bonded state estimation method according to one embodiment of the present invention. FIG. 6 is an example of bonded acoustic data. FIG. 7 is an example of a division spectrogram.

[0009] (Configuration of Ultrasonic Complex Vibration Device) The estimation target of the bonding condition estimation device in one embodiment of the present invention is the bonding condition of workpieces W1 and W2 bonded using ultrasonic complex vibrations by an ultrasonic bonding device configured as shown in FIG. 1 . The ultrasonic complex vibration device 10 shown in FIG. 1 is a component of the ultrasonic bonding device that bonds workpieces W1 and W2, which are objects to be bonded such as metal plates, using ultrasonic complex vibrations, as described below. The ultrasonic bonding device is used, for example, to bond electrodes of lithium-ion batteries and / or semiconductor elements, or to bond metals of the same or different types. The ultrasonic bonding device may also be used to bond semiconductors, ceramics, and / or resins, etc.

[0010] 1 , the ultrasonic complex vibration device 10 includes a substantially cylindrical first vibration element 111, a substantially cylindrical, cylindrical, or bottomed cylindrical intermediate vibration element 110, and a substantially cylindrical or bottomed cylindrical second vibration element 112. The first vibration element 111, the intermediate vibration element 110, and the second vibration element 112 constitute "vibration elements (or vibrators)." The ultrasonic bonding device includes the ultrasonic complex vibration device 10, a horn tip 140 (ultrasonic bonding tip), and an anvil 118.

[0011] The first vibration element 111 and the intermediate vibration element 110 are coaxially connected by a mechanical connecting mechanism (such as a bolt and / or a clamp mechanism) at the middle or intermediate portion of the ultrasonic complex vibration device 10. The intermediate vibration element 110 and the second vibration element 112 are coaxially connected by a mechanical connecting mechanism at the middle portion of the ultrasonic complex vibration device 10. The first vibration element 111, the intermediate vibration element 110, and the second vibration element 112 may be integrally configured rather than being mechanically connected.

[0012] The intermediate vibration element 110 may be a component of the first vibration element 111. That is, the first vibration element 111 may be composed of two vibration elements. In this case, the first vibration element 111 and the intermediate vibration element 110 may be integrally configured rather than being mechanically connected. The intermediate vibration element 110 may be a component of the second vibration element 112. That is, the second vibration element 112 may be composed of two vibration elements. In this case, the second vibration element 112 and the intermediate vibration element 110 may be integrally configured rather than being mechanically connected.

[0013] As shown in FIG. 1, the first vibration element 111 is provided with a piezoelectric body 1112 whose axial direction is the piezoelectric polarization direction.

[0014] As shown in FIG. 1 , the intermediate vibration element 110 is formed with a substantially annular plate-shaped intermediate flange 102 that protrudes radially around its entire circumference at a central position in the axial direction. The intermediate vibration element 110 is configured to be clamped or supported around its entire circumference by a clamping mechanism (not shown) at least at the intermediate flange 102. If it is ensured that the intermediate vibration element 110 is supported by a mechanical support mechanism, the intermediate flange 102 may be omitted. As shown in FIG. 1 , the intermediate vibration element 110 has a substantially cylindrical shape with a substantially constant outer diameter in the axial direction behind the intermediate flange 102 (leftward in FIG. 1 ). As shown in FIG. 1 , the intermediate vibration element 110 has a substantially cylindrical shape (a shape in which a substantially truncated conical shape and a substantially cylindrical shape are coaxially connected) with a substantially constant outer diameter after continuously tapering partway toward the tip of the intermediate vibration element 110 (rightward in FIG. 1 ).

[0015] 1, the second vibration element 112 is provided with a frequency adjustment element 1120 having a generally regular octagonal shape with rounded corners that protrudes radially around the entire circumference at a midpoint in the axial direction of the second vibration element 112. The frequency adjustment element 1120 adjusts the resonance frequencies of the longitudinal vibration component and the torsional vibration component of the ultrasonic vibration.

[0016] 1, the second vibration element 112 has a plurality of slits 1124 formed on its outer surface behind the frequency adjustment element 1120. A plurality of slits 1124 may be formed on the outer surface of the second vibration element 112 ahead of the frequency adjustment element 1120. The slits 1124 extend obliquely in the second vibration element 112 when viewed from the side, or extend in the axial direction while being displaced in the circumferential direction in phase with each other. N (N=2, 3, ...) slits 1124 may be arranged to have N-fold rotational symmetry (e.g., N=8, 12, or 16) around the central axis of the second vibration element 112.

[0017] 1, the second vibration element 112 is provided with a tip portion 1126 that is approximately regular octagonal in shape with rounded corners and that protrudes radially around the entire circumference at the tip position in the axial direction. Holes 1128 (or through holes) are formed in the tip portion 1126 at a plurality of locations spaced apart in the circumferential direction. The N (N=2, 3, ...) holes 1128 may be arranged to have N-fold rotational symmetry (e.g., N=4) around the central axis of the second vibration element 112. A female thread is provided on the inner surface of the hole 1128.

[0018] Horn tip 140 has a substantially truncated cone-shaped base portion and a tip portion that abuts against workpiece W1, the uppermost of workpieces W1 and W2. A male thread provided at the base end of horn tip 140 screws into a female thread provided in hole 1128 in tip portion 1126 of second vibration element 112, thereby removably fixing horn tip 140 to second vibration element 112. Horn tips 140 of various shapes are available, so that horn tips 140 can be appropriately replaced depending on the type of metal to be joined, etc.

[0019] The balancer for adjusting the phase difference between the longitudinal vibration and the torsional vibration at the tip 1126 of the second vibration element 112, and therefore at the horn tip 140, may be removably fixed to the tip 1126 of the second vibration element 112 by screwing the male thread of the balancer into the female thread of the hole 1128.

[0020] Anvil 118 is disposed so as to vertically face the tip of horn tip 140. For example, substantially flat workpieces W1 and W2 are placed one on top of the other on the upper surface of anvil 118. Anvil 118 may be configured to passively or actively displace up and down in response to the pressure of horn tip 140 that it receives through workpieces W1 and W2.

[0021] As shown in FIG. 1, the ultrasonic bonding apparatus further includes a control device 120 , a high frequency power supply device 121 , a pressure device 122 , a status sensor 124 , and an interface device 126 .

[0022] The high-frequency power supply device 121 is configured to apply a high-frequency AC voltage to the piezoelectric element 1112 of the first vibration element 111 in response to power supplied from a commercial power source (not shown), thereby exciting the first vibration element 111 in the axial direction. The pressure device 122 includes a pressure block and is configured to apply pressure to the workpieces W1 and W2 from the horn tip 140 by displacing a support mechanism, such as a clamp mechanism, that supports the intermediate vibration element 110 using the pressure block. The status sensor 124 includes a stroke sensor that outputs a signal corresponding to the displacement of the pressure block constituting the pressure device 122, as well as an amplitude sensor that outputs a signal corresponding to the amplitude of the horn tip 140 (corresponding to a specified parameter). The amplitude sensor may be a sensor module configured with an imaging device and a device that calculates the amplitude by analyzing an image acquired through the imaging device.

[0023] As specified parameters that change depending on the progress of joining one workpiece W1 and the other workpiece W2, in addition to the amplitude A of the horn tip 40, the axial displacement, displacement speed and / or displacement acceleration of the vibration element (e.g., the second vibration element 12) may be measured as specified parameters.

[0024] The interface device 126 is, for example, configured as a display, and displays or outputs on the display the displacement amount of the pressure block and / or the time series of the pressure according to the output signal of the status sensor 124. The display may be configured as a touch panel display, and may be configured to accept setting operations for allowing the user to directly or indirectly specify parameters, such as one of a plurality of bonding modes that defines a time series pattern of the target pressure.

[0025] The control device 120 is configured with a microcomputer, an arithmetic processing device (CPU, microprocessor, processor core, etc.), and a storage device (memory such as ROM and RAM). The control device 120 is configured to control the displacement operation of the pressure block by the pressure device 122 based on a time series of the displacement amount of the pressure block represented by an output signal of a stroke sensor constituting the status sensor 124. The control device 120 is configured to control the power supplied to the piezoelectric body 1112 based on the amplitude (corresponding to a specified parameter) of the horn tip 140 represented by an output signal of an amplitude sensor constituting the status sensor 124, and thereby to control the ultrasonic vibration power of the vibration elements (first vibration element 111, intermediate vibration element 110, and second vibration element 112) and the ultrasonic vibration power of the horn tip 140. As the status sensor 124, a pressure sensor is provided that outputs a signal corresponding to the pressure acting on the intermediate vibration element 110 from the pressure block of the pressure device 122 (the pressure applied by the horn tip 140 to the workpieces W1 and W2), and the control device 120 may control the time series of the pressure to be constant or in a specified manner based on the output signal of the pressure sensor.

[0026] (Configuration of the Bonding State Estimation Device) The bonding state estimation device 20 according to one embodiment of the present invention shown in FIG. 2 includes an input interface 21, a bonding state estimation model generation unit 22, a bonding state estimation unit 23, and an output interface 24. The bonding state estimation device 20 is configured by a computer such as a server computer (including a cloud server), a personal computer, a smartphone, or a tablet computer. The respective components of the bonding state estimation device 20 include an arithmetic processing device (configured by a CPU and a processor (core)), a storage device (configured by memory (ROM, RAM, etc.)), an I / O circuit, etc. The components of the bonding state estimation device 20 have a function in which the arithmetic processing device reads necessary programs (software) and data from the storage device and executes designated arithmetic processing in accordance with the program based on the data.

[0027] The input interface 21 includes a learning input data acquisition unit 211, a learning output data acquisition unit 212, and an estimation input data acquisition unit 213. The input interface 21 may include touch panel buttons, a keyboard, and / or a voice input device.

[0028] The learning input data acquisition unit 211 is configured to acquire learning input data. A microphone M collects a sound (vibration signal) generated when the ultrasonic bonding device (see FIG. 1) performs one ultrasonic bonding operation (described later) on the workpieces W1 and W2. The microphone M may be a component of the input interface 21. The "learning input data" may include, for example, a spectrogram of the sound shown in FIG. 7, which is divided into a plurality of frequency domains [f i , f i+1 ] (i=1, 2, . . . N) and multiple time domains [t j , t j+1 ] (j = 1, 2, ... M) ij The data is defined by a plurality of sets of frequency f, time t, and the intensity I(f, t) of the signal component of frequency f at time t. i, f i+1 ] width f i+1 -f i may be the same as each other, or may be different from each other, such as gradually narrowing (or widening) as it goes towards the high frequency region. j , t j+1 ] width t j+1 -t j may be identical to each other, or may be different from each other, such as gradually narrowing (or widening) as the spectrograms approach the initial or final stage of junction. Only a portion of the N×M pieces of N×M divided spectrogram data may be included in the training input data (and the estimation input data described below), rather than all of the N×M pieces of divided spectrogram data. For example, only n×m pieces (n<N, m<M) of divided spectrogram data may be included in the training input data, with priority given to the mid-frequency region, the low-frequency region, or the high-frequency region, and with priority given to the early, middle, or final stages of junction.

[0029] The learning output data acquisition unit 212 is configured to acquire learning output data. The "learning output data" includes at least one of quantitative evaluation data (bonding strength, base material bond area, interface bond area, and their composition ratio, amount of contamination, amount of plastic deformation) regarding the bonding of the workpieces W1 and W2, or data representing at least one pair (or one of) non-quantitative evaluations (such as comments indicating the bonding status) ("bonding index value (label)"), such as full base material fracture, partial base material fracture, interface fracture, tool escape, upper plate fracture, etc. The quantitative evaluation data regarding the bonding of the workpieces W1 and W2 includes, for example, a tensile strength test (JIS L 1096) or a peel strength test (JIS Z 3140) in accordance with JIS standards. However, a quality evaluation method unique to each secondary battery manufacturer may also be used to evaluate the bonding status of irregularly shaped parts.

[0030] The estimation input data acquisition unit 213 is configured to acquire estimation input data. The "estimation input data" includes a spectrogram of a sound generated during one ultrasonic joining operation (described later) of the workpieces W1 and W2, which is divided into a plurality of frequency domains f i ~fi+1 (i=1, 2, . . . N) and multiple time domains [t j , t j+1 ] (j = 1, 2, ... M) ij (See Figure 7).

[0031] The bonding condition estimation model generation unit 22 generates a trained model by machine learning the correlation between the learning input data and the risk of defective mixing based on a plurality of learning data including at least the learning input data, using "supervised learning," "semi-supervised learning," "unsupervised learning," or the like. The learning model is stored and held in a storage device, or is read from a database and then stored in a storage device, and is read from the storage device.

[0032] The bonding state estimation unit 23 is configured to output an estimation result of the bonding state of the workpieces W1 and W2 by inputting the estimation input data acquired by the estimation input data acquisition unit 213 into the trained model generated by the bonding state estimation model generation unit 22.

[0033] The output interface 24 is composed of an image output device, or an image output device and an audio output device, and is configured to output the estimation results of the joining state of the workpieces W1 and W2 output by the joining state estimation unit 23.

[0034] The "bonding state learning device" according to one embodiment of the present invention is configured with a learning input data acquisition unit 211 and a learning output data acquisition unit 212, which are included in the input interface 21, and a bonding state estimation model generation unit 22. The "bonding state estimation device" according to one embodiment of the present invention is configured with an estimation input data acquisition unit 213, which is included in the input interface 21, a bonding state estimation unit 23, and an output interface 24.

[0035] (Function of Ultrasonic Bonding Device) The procedure for ultrasonically bonding the workpieces W1 and W2 using the ultrasonic bonding device will now be described. A reference amplitude A0 is set (FIG. 3 / STEP 100). For example, the reference amplitude A0 may be directly or indirectly designated by the user via a touch panel display constituting the interface device 126, such as by selecting one of a plurality of bonding modes for which the reference amplitude A0 is determined.

[0036] Pressurizing device 122 moves first vibration element 111, intermediate vibration element 110, second vibration element 112, and horn tip 140 downward (FIG. 3 / STEP 102). Pressure P received by workpieces W1 and W2 on the tip of horn tip 140 (free end of rod portion 44) is measured based on the output signal of the pressure sensor constituting state sensor 124 (FIG. 3 / STEP 104). When the tip of horn tip 140 is separated from workpiece W1, P=0, and when it abuts against workpiece W1, P>0.

[0037] It is determined whether the pressure P received by horn tip 140 is equal to or greater than the designated pressure P0 (FIG. 3 / STEP 106). If the determination result is negative (FIG. 3 / STEP 106...NO), pressure device 122 moves first vibration element 111, intermediate vibration element 110, second vibration element 112, and horn tip 140 downward (FIG. 3 / STEP 102). This adjusts the vertical position of horn tip 140, and therefore the static pressure applied from the tip of horn tip 140 to workpieces W1 and W2, to fall within the designated static pressure range (e.g., 200 N to 800 N).

[0038] If the determination result is affirmative (FIG. 3 / STEP 106...YES), ultrasonic vibrations are generated in the vibration element (FIG. 3 / STEP 108). Specifically, in response to power being supplied to the high-frequency power supply device 121 from a commercial power source (not shown), the high-frequency power supply device 121 applies a high-frequency AC voltage to the piezoelectric body 1112 of the first vibration element 111. This causes the first vibration element 111 to vibrate in its axial direction at, for example, approximately 20 kHz, generating ultrasonic vibrations. The ultrasonic vibrations are transmitted from the first vibration element 111 to the intermediate vibration element 110 in its axial direction, and the amplitude of the ultrasonic vibrations is amplified. Furthermore, the ultrasonic vibrations with amplified amplitude are transmitted from the intermediate vibration element 110 to the second vibration element 112 in its axial direction.

[0039] In this way, a portion of the longitudinal vibration component (axial component of the second vibration element 112) of the ultrasonic vibration transmitted to the second vibration element 112 is converted into a torsional vibration component by the multiple slits 1124 formed on the outer surface of the second vibration element 112. Then, a composite vibration generated by combining the longitudinal vibration component and the torsional vibration component is transmitted to the horn tip 140 fixed to the tip 1126 of the second vibration element 112.

[0040] In response, the tip of the horn tip 140 displaces or vibrates horizontally, tracing a circular or elliptical orbit. As a result, as shown schematically in FIG. 4 , the amplitude and ultrasonic vibration power of the horn tip 140 gradually increase from the vibration start time t = t0. During this process, impurities on the contact surfaces of the workpieces W1 and W2 are removed, further promoting plastic deformation of the contact surfaces of the workpieces W1 and W2. Then, as shown in FIG. 4 , the rate of increase in the amplitude and ultrasonic vibration power of the horn tip 140 significantly decreases at time t = t1, after which the amplitude and ultrasonic vibration power of the horn tip 140 gradually increase. This is because the metal oxide coatings and other components constituting the joining surfaces of the workpieces W1 and W2 are removed, revealing clean, activated metal atoms on the joining surfaces. The temperature rise due to frictional heat activates the atomic movement, generating a mutual attraction between the atoms.

[0041] At this time, the amount of pressing of the workpieces W1 and W2 by the horn tip 140 and / or the static pressure applied to the workpieces W1 and W2 are adjusted, and a composite vibration is applied to one of the workpieces W1 and W2, thereby solid-state joining the workpieces W1 and W2.

[0042] An amplitude sensor constituting status sensor 124 optically measures amplitude A at a specified location (e.g., a location where the amplitude is relatively large) of horn tip 140 (FIG. 6 / STEP 10). It is then determined whether the time derivative δA (= current amplitude A(k)−previous amplitude A(k−1)) of amplitude A at horn tip 140 is negative and whether amplitude A is equal to or less than a reference amplitude A0 (FIG. 6 / STEP 12). Instead of this determination process, it may be determined whether amplitude A has decreased by the reference amplitude A0 (or a reference ratio based on the maximum value) using the maximum value at the time when amplitude A began to decrease as a reference.

[0043] If the determination result is negative (FIG. 6 / STEP 12...NO), ultrasonic vibrations are continuously generated in the vibration element (FIG. 3 / STEP 108). On the other hand, if the determination result is positive (FIG. 6 / STEP 12...YES), the generation of ultrasonic vibrations in the vibration element is stopped (FIG. 6 / STEP 14). For example, as shown in FIG. 4, after time t=t2 when the amplitude of horn tip 140 changes from increasing to decreasing and becomes equal to or less than reference amplitude A0, the ultrasonic power of horn tip 140 is controlled to be 0 with a slight response delay.

[0044] (Method for Estimating Bonding State (First Embodiment)) In the first embodiment, a learning model is made to learn the correlation between learning input data and learning output data by "supervised learning."

[0045] The learning input data acquisition unit 211 acquires learning input data. Specifically, joining acoustic data generated when the workpieces W1 and W2 are joined using the ultrasonic complex vibration device 10 is acquired through the microphone M (FIG. 4 / STEP 210). As a result, for example, as shown in FIG. 6, an acoustic signal with time on the horizontal axis and sound intensity on the vertical axis is acquired as joining acoustic data.

[0046] Learning output data is acquired by the learning output data acquisition unit 212. Specifically, the quality of the bonding state evaluated based on the bonding strength of the workpieces W1 and W2 is acquired as a label (FIG. 4 / STEP 211).

[0047] The training input data acquisition unit 211 performs frequency analysis on the combined acoustic data to generate a spectrogram (FIG. 4 / STEP 212). As a result, a spectrogram is generated based on the combined acoustic data, with the horizontal axis representing time and the vertical axis representing frequency, and brightness representing signal strength, as shown in FIG. 7. The group of spectrograms generated as described above is stored in a storage device and / or database as a combined acoustic data set (constituting the training input data) (FIG. 4 / STEP 213).

[0048] Next, the spectrogram is divided in the frequency domain (FIG. 4 / STEP 214). As a result, the spectrogram is divided into each of a plurality of frequency domains, for example, as shown in FIG. 7. Furthermore, the spectrogram is divided in the time domain (FIG. 4 / STEP 215). As a result, the spectrogram is divided into each of a plurality of time domains, for example, as shown in FIG. 7. As a result, divided spectrograms are obtained in which the spectrogram is divided in each of the frequency domain and the time domain.

[0049] Then, the prediction problem of the bonding state of the workpieces W1 and W2 is optimized by the machine learning model (FIG. 4 / STEP 216). Specifically, learning input data constituting the learning data is input to the learning model, and model output data is output from the learning model. The "learning input data" includes data representing each of a plurality of divided spectrograms obtained by dividing a spectrogram in the frequency domain and the time domain. Model parameters defining the learning model are identified based on the comparison results between the model output data and the learning output data constituting the learning data. The "learning output data" includes a label (a bonding index value indicating the quality of the bonding state of the workpieces W1 and W2).

[0050] When "supervised learning" is performed, for example, a neural network model is used as the learning model to generate a trained model. In a neural network model, training input data is input to an input layer, and model output data is output from an output layer as an estimation or inference result. It is determined whether the error between the model output data and training output data (teacher data) included in the training data is equal to or less than a reference value (FIG. 4 / STEP 217).

[0051] If the determination result is negative (FIG. 4 / STEP 217...NO), further optimization of the prediction problem of the bonding state of the workpieces W1 and W2 using a machine learning model is attempted (FIG. 4 / STEP 216).On the other hand, if the determination result is positive (FIG. 4 / STEP 217...YES), the trained model defined by the model parameters identified at that stage is saved in a storage device and / or a database (FIG. 4 / STEP 218).

[0052] The trained model is generated for each combination of the specifications of the workpieces W1 and W2 (shape, size, material, surface roughness, etc.) and the specifications of the ultrasonic joining device (structure of the ultrasonic complex vibration device 10, shape, size, material, presence or absence and type of uneven structure on the tip surface, reference amplitude A0, specified pressure P0, etc.).

[0053] The estimation input data acquisition unit 213 acquires the estimation input data. The estimation input data is acquired according to the same procedure as the training input data. Specifically, bonding acoustic data generated when the workpieces W1 and W2 are bonded using the ultrasonic complex vibration device 10 is acquired through the microphone M (FIG. 5 / STEP 220 (see FIG. 6)). The bonding acoustic data is subjected to frequency analysis to generate a spectrogram (FIG. 5 / STEP 222 (see FIG. 7)). The group of spectrograms generated as described above is stored in a storage device and / or database as a bonding acoustic data set (constituting the estimation input data) (FIG. 5 / STEP 223). Next, the spectrogram is divided into the frequency domain and the time domain (FIG. 5 / STEP 224, STEP 225).

[0054] Next, the input data for estimation is input to the trained model, and a joining index value representing the quality of the joining state of the workpieces W1 and W2 is estimated (FIG. 5 / STEP 226). The "input data for estimation" includes data representing each of a plurality of divided spectrograms obtained by dividing a spectrogram based on the joining acoustic data in each of the frequency domain and the time domain.

[0055] It is determined whether the bonding index value is equal to or greater than the standard value (FIG. 5 / STEP 227). If the determination result is affirmative (FIG. 5 / STEP 227...YES), the bonding condition of the workpieces W1 and W2 is determined to be "good" (FIG. 5 / STEP 228). If the determination result is negative (FIG. 5 / STEP 227...NO), the bonding condition of the workpieces W1 and W2 is determined to be "poor" (FIG. 5 / STEP 229).

[0056] Then, the estimation result of the bonding state of the workpieces W1 and W2 as the model output data is output through the output interface 24.

[0057] Bonding condition estimation method (Second embodiment) The second embodiment differs from the first embodiment in that "unsupervised learning" is adopted as the machine learning method instead of "supervised learning." The following mainly describes the differences from the first embodiment.

[0058] A plurality of pieces of learning input data are acquired by the learning input data acquisition unit 211 as a plurality of pieces of learning data (see FIG. 4 / STEP 210, STEP 212, STEP 213, STEP 214, and STEP 215). That is, the learning data acquisition unit is not made up of both the learning input data acquisition unit 211 and the learning output data acquisition unit 212, but is made up of only the learning input data acquisition unit 211. Label acquisition processing is omitted (see FIG. 4 / STEP 211).

[0059] The bonding condition estimation model generation unit 22 inputs learning data into the learning model, causing the unsupervised learning model to learn the correlation between the learning input data included in the learning data and data indicating that the bonding condition is normal or good, and generates a bonding condition estimation model (trained model) (see Figure 4 / STEP 216, STEP 217, STEP 218).

[0060] For example, an autoencoder (AE) model is employed as the learning model. According to the autoencoder model, learning input data relating to the normal or favorable bonding state of the workpieces W1 and W2 is input to the input layer as learning data. Subsequently, in the intermediate layer, scores are assigned (weighted) according to the importance of the data, and data with low scores are eliminated (encoded). Weighting is then applied again when moving to the output layer, and the sum of the data received from multiple edges is output (decoded) as the final value. Through this procedure, patterns and / or trends in the learning input data are learned so that the model output data matches the learning input data. This series of steps is repeatedly executed, and when specified conditions are met, learning by the learning model ends, and a trained model is generated.

[0061] As the autoencoder (AE), a stacked autoencoder (SAE), a convolutional autoencoder (CAE), a variational autoencoder (VAE) or a conditional autoencoder may be employed.

[0062] Then, the estimation input data acquisition unit 213 acquires data representing a split spectrogram (see Figure 7) derived from the joining sound of the workpieces W1 and W2, whose joining state is unknown, as estimation input data (see Figure 5 / STEP 220, STEP 222, STEP 223, STEP 224, STEP 225).

[0063] Next, the bonding condition estimation unit 23 inputs the estimation input data to the input layer of the bonding condition estimation model. In response to this, an estimation result of the bonding index value representing the quality of the bonding condition of the workpieces W1 and W2 is acquired as model output data output from the output layer of the bonding condition estimation model (see FIG. 5 / STEP 226). A deviation is calculated between the estimation input data input to the bonding condition estimation model or a feature value based thereon and the model output data output from the bonding condition estimation model or a feature value based thereon. Furthermore, if the deviation is less than a threshold value, the bonding condition is determined to be "good," and if the deviation is equal to or greater than the threshold value, the bonding condition is determined to be "poor" (see FIG. 5 / STEP 227, STEP 228, STEP 229).

[0064] A case where feedback control is performed using this processing sequence is shown in Fig. 6. If it is estimated that the value is below the standard value based on the feature amount of the time series change, the joining operation continues until it reaches or exceeds the standard value, and then stops at an appropriate estimated joining index value (see Fig. 6 / STEP 227 and STEP 228).

[0065] Then, the estimation result of the bonding state of the workpieces W1 and W2 as the model output data is output through the output interface 24.

[0066] (Other Embodiments) Note that in the above-described embodiments, a neural network (first embodiment) or an autoencoder (second embodiment) is used as the learning method, but any other machine learning method may be used. For example, tree-type learning models such as decision trees and regression trees, ensemble learning models such as bagging and boosting, neural network-type learning models (including deep learning) such as recurrent neural networks and convolutional neural networks, clustering models such as hierarchical clustering, non-hierarchical clustering, k-nearest neighbors, and k-means, multivariate analysis models such as principal component analysis (PCA), factor analysis, and logistic regression, and support vector machines (SVMs) may be used as learning models. In addition to supervised learning and unsupervised learning, semi-supervised learning may also be used.

[0067] In the above embodiment, the microphone M collects "sound" as a vibration signal generated during one ultrasonic bonding operation of the workpieces W1 and W2 by the ultrasonic bonding device (see FIG. 1). In another embodiment, the vibration sensor may collect vibrations propagating to the vibration element, the anvil 118, or a structure connected to or in contact with the vibration element or the anvil 118 as the vibration signal. From the spectrogram of the vibration signal detected by the vibration sensor, a part of the spectrogram, the i-th partial spectrogram SP i The i-th partial spectrogram data for is acquired.

[0068] The vibration sensor may be a contact type (such as a frequency change type, piezoelectric type, electrodynamic type, or servo type) vibration sensor and / or a non-contact type (such as an eddy current type, capacitance type, or optical type) vibration sensor. The vibration sensor may be a component of the input interface 21. The optical vibration sensor may be a single laser Doppler vibration sensor, a scanning laser Doppler vibration sensor, or a microscope-type laser Doppler vibration sensor.

[0069] 10: Ultrasonic complex vibration device 102: Intermediate flange 110: Intermediate vibration element 111: First vibration element 1112: Piezoelectric body 112: Second vibration element 1120: Frequency adjustment element 1121: Cylindrical portion 1122: Cylindrical portion 1124: Slit 1126: Tip 1128: Hole 118: Anvil 120: Control device 121: High frequency power supply device 122: Pressure device 124: Status sensor 126: Interface device 140: Horn tip 20: Bonding condition estimation device 21: Input interface 211: Learning input data acquisition unit 212: Learning output data acquisition unit 213: Estimation input data acquisition unit 22: Bonding condition estimation model generation unit (model generation unit) 23: Bonding condition estimation unit 24: Output interface W1: One workpiece W2: Other workpiece.

Claims

1. A bonding state learning device comprising: a learning data acquisition unit that acquires, as learning input data, multiple sets of learning data including data representing split spectrograms in which a spectrogram of a vibration signal generated during the bonding process of multiple workpieces using an ultrasonic bonding device is divided into each of multiple frequency domains and each of multiple time domains; and a model generation unit that generates a trained model by having a learning model learn the correlation between the learning input data and the bonding state of the multiple workpieces based on the multiple sets of learning data acquired by the learning data acquisition unit.

2. A bonding state learning device according to claim 1, wherein the learning data acquisition unit acquires the plurality of sets of learning data including data representing the bonding states of the plurality of workpieces as learning output data, and the model generation unit generates the trained model by having the learning model learn the correlation between the learning input data and the learning output data by supervised learning or semi-supervised learning.

3. A bonding state learning device according to claim 1, wherein the model generation unit generates the trained model by having the learning model learn features of the plurality of learning input data contained in each of the plurality of sets of learning data through unsupervised learning.

4. A bonding state learning program for causing a computer to execute a bonding state learning method, the program comprising: a learning data acquisition step of acquiring, as learning input data, multiple sets of learning data, the data representing split spectrograms in which a spectrogram of a vibration signal generated during the bonding process of multiple workpieces using an ultrasonic bonding device is divided into multiple frequency domains and multiple time domains; and a model generation step of generating a trained model by having a learning model learn the correlation between the learning input data and the bonding states of the multiple workpieces, based on the multiple sets of learning data acquired by the learning data acquisition step.

5. A bonding state estimation device comprising: an estimation input data acquisition unit that acquires, as estimation input data, divided spectral data obtained by dividing spectral data of a vibration signal generated during the bonding process of multiple workpieces using an ultrasonic bonding device into each of multiple frequency domains and each of multiple time domains; and a bonding state estimation unit that inputs the input data acquired by the estimation input data acquisition unit into the trained model generated by the bonding state learning device described in any one of claims 1 to 3, thereby acquiring an estimated result of the bonding state of the multiple workpieces as an output from the trained model.

6. A bonding condition estimation program for causing a computer to execute a bonding condition estimation method, the method including: an estimation input data acquisition step for acquiring, as estimation input data, divided spectral data obtained by dividing spectral data of a vibration signal generated during the bonding process of a plurality of workpieces using an ultrasonic bonding device into each of a plurality of frequency domains and each of a plurality of time domains; and a bonding condition estimation step for inputting the input data acquired in the estimation input data acquisition step into the trained model generated by the bonding condition learning device described in any one of claims 1 to 3, thereby acquiring an estimated result of the bonding condition of the plurality of workpieces as an output from the trained model.

Images