A deep learning compensation method and system for mechanical assembly errors

Abstract

The application relates to a mechanical assembly error deep learning compensation method and system, and relates to the technical field of mechanical assembly optimization, which comprises the following steps: synchronously collecting point gluing images and point gluing point cloud data according to a preset monitoring time interval, identifying and outputting a point gluing state distribution sequence; performing key state identification and feature extraction on the point gluing state distribution sequence, performing window fluctuation analysis of assembly errors, and outputting a compensation time window sequence; predicting a predicted point gluing state distribution sequence, taking optimal optical performance as a target, performing pressure control parameter optimization, and outputting an optimal pressure control parameter sequence; and optimizing the control of the fitting assembly process in a preset time zone according to the compensation time window sequence and the optimal pressure control parameter sequence. The application solves the problems that traditional mechanical assembly only relies on a global fixed pressure strategy, resulting in insufficient fitting precision of a 3C camera module, low production yield and uncontrollable optical performance of the camera.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of mechanical assembly optimization, and in particular to a mechanical assembly error deep learning compensation method and system. BACKGROUND

[0002] With the improvement of 3C electronic manufacturing precision requirements, the assembly quality of camera module and glass cover plate becomes a key technical problem affecting product performance.

[0003] At present, the traditional mechanical assembly method adopts a global fixed pressing strategy, which cannot dynamically adjust parameters according to the dispensing state, thereby not only reducing the optical quality consistency of finished products, but also increasing the cost of subsequent product detection and rework. SUMMARY

[0004] The present application provides a mechanical assembly error deep learning compensation method and system, which improves the precision, production yield and controllability of camera optical performance of 3C camera module and glass cover plate assembly.

[0005] The present application discloses the following technical solutions:

[0006] In a first aspect, the present application provides a mechanical assembly error deep learning compensation method, which comprises:

[0007] In the assembly process of 3C camera module and glass cover plate, industrial camera and three-dimensional laser scanner are used to synchronously collect dispensing image and dispensing point cloud data according to a preset monitoring time interval, and dispensing state recognition is performed to output dispensing state distribution sequence;

[0008] Key state recognition and feature extraction are performed on the dispensing state distribution sequence, window fluctuation analysis of assembly error is performed according to the key dispensing state distribution sequence, and a compensation time window sequence in a preset time zone is output;

[0009] A predicted dispensing state distribution sequence of the compensation time window sequence is obtained based on the key dispensing state distribution sequence, and optimal pressing control parameter optimization is performed according to the predicted dispensing state distribution sequence with the optimal camera optical performance as the target, and an optimal pressing parameter sequence is output;

[0010] The assembly process in the preset time zone is optimized and controlled according to the compensation time window sequence and the optimal pressing parameter sequence.

[0011] In a second aspect, the present application provides a mechanical assembly error deep learning compensation system, which comprises:

[0012] The dispensing data acquisition module is configured to, in a process of bonding and assembling the 3C camera module and the glass cover plate, acquire dispensing images and dispensing point cloud data synchronously by using an industrial camera and a three-dimensional laser scanner according to a preset monitoring time interval, perform dispensing state recognition, and output a dispensing state distribution sequence.

[0013] The window fluctuation analysis module is configured to perform key state recognition and feature extraction on the dispensing state distribution sequence, perform window fluctuation analysis of assembly error according to the key dispensing state distribution sequence, and output a compensation time window sequence in a preset time zone.

[0014] The compression parameter optimization module is configured to predict a predicted dispensing state distribution sequence of the compensation time window sequence based on the key dispensing state distribution sequence, perform compression control parameter optimization according to the predicted dispensing state distribution sequence with the goal of optimizing the optical performance of the camera, and output an optimal compression parameter sequence.

[0015] The assembly optimization control module is configured to perform optimization control on the bonding and assembling process in the preset time zone according to the compensation time window sequence and the optimal compression parameter sequence.

[0016] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0017] The present application provides a deep learning compensation method and system for mechanical assembly error. Through the cooperative operation of industrial equipment data acquisition, key state recognition, time window prediction, dispensing state prediction, compression control parameter optimization and assembly process regulation, the present application realizes accurate compensation of the bonding and assembling error of the 3C camera module and the glass cover plate. Firstly, in the bonding and assembling process, the dispensing images and point cloud data are synchronously acquired by using an industrial camera and a three-dimensional laser scanner, and a dispensing state recognition plug-in is trained by combining a convolutional neural network to output a dispensing state distribution sequence. Then, the key state of the dispensing state distribution sequence is extracted, a compensation time window predictor is constructed based on a deep neural network, and a compensation time window sequence in a preset time zone is obtained. Subsequently, the dispensing state in the compensation window is predicted by using a long short-term memory network, and an optimal compression parameter sequence is generated by iteratively optimizing the compression control parameter by combining an optical performance predictor constructed by deep learning and taking the optimal optical performance of the camera as the goal. Finally, the assembly process is optimized and controlled in the whole period according to the time sequence corresponding relationship between the compensation time window and the optimal compression parameter.

[0018] The technical solution of the present application solves the problems of error compensation lag caused by dynamic changes in dispensing state in traditional mechanical assembly, fixed compression control parameters that cannot adapt to different dispensing scenarios, and the difficulty in balancing assembly precision and optical performance, realizes the prospective compensation of assembly error and the stable guarantee of the optical performance of the camera, and improves the efficiency and product yield of the bonding and assembling of the 3C camera module. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0020] Figure 1 A flowchart of a mechanical assembly error deep learning compensation method provided by the embodiment of the present application is shown.

[0021] Figure 2 A structural diagram of a mechanical assembly error deep learning compensation system provided by the embodiment of the present application is shown.

[0022] In the drawings, the components represented by the numbers are described as follows:

[0023] Dispensing data acquisition module 01, window fluctuation analysis module 02, compression parameter optimization module 03, assembly optimization control module 04. DETAILED DESCRIPTION

[0024] The present application provides a mechanical assembly error deep learning compensation method and system, which is used to solve the problems of insufficient assembly precision, low production yield, and uncontrollable final camera optical performance caused by relying only on a global fixed compression strategy during the assembly of a 3C camera module and a glass cover plate.

[0025] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present application.

[0026] In the description of the present application, the terms "first" and "second" are used for description purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0027] In the description of the present application, the term "for example" is used to indicate "as an example, instance, or illustration." Any embodiment described as "for example" in this application is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, for the purpose of explanation, details are set forth. It is apparent to those skilled in the art that the present application can be practiced without the use of these specific details. In other instances, well-known structures and processes are not elaborated in order not to obscure the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0028] Embodiment one, as shown in the accompanying drawings Figure 1 The present application provides a deep learning compensation method for mechanical assembly errors, which comprises the following steps:

[0029] S110: In the bonding assembly process of the 3C camera module and the glass cover plate, the industrial camera and the three-dimensional laser scanner are used to synchronously collect the dispensing image and the dispensing point cloud data according to the preset monitoring time interval, and the dispensing state is identified, and the dispensing state distribution sequence is output.

[0030] In the dispensing state monitoring scene of the bonding assembly of the 3C camera module and the glass cover plate in the present application, in order to accurately obtain the dispensing state to support subsequent assembly optimization, the sample data used for training the dispensing state recognition model is first obtained, and the plug-in capable of accurately recognizing the dispensing state is trained, so as to realize effective analysis and state output of the real-time collected dispensing image and dispensing point cloud data.

[0031] Specifically, first, the industrial camera and the three-dimensional laser scanner are used to synchronously collect the dispensing image and the dispensing point cloud data according to the preset monitoring time interval, and the dispensing image sequence and the dispensing point cloud data sequence are obtained by orderly integrating the images and the point cloud data collected for multiple times, so as to comprehensively and continuously present the dispensing state change at different times.

[0032] Further, based on the historical bonding assembly record, the sample dispensing image set and the sample dispensing point cloud data set are collected, and the dispensing state under different sample dispensing images and sample dispensing point cloud data is labeled to obtain the dispensing state distribution label set.

[0033] Further, the sample dispensing image set and the sample dispensing point cloud data set are used as input, and the dispensing state distribution label set is used as supervision, and the convolutional neural network is trained to converge to generate the dispensing state recognition plug-in.

[0034] Further, in the actual bonding assembly process, the trained dispensing state recognition plug-in is used to recognize the dispensing state according to the obtained dispensing image sequence and dispensing point cloud data sequence, and a dispensing state distribution sequence is output.

[0035] This step provides an accurate data basis for subsequent assembly optimization based on dispensing state by first training an accurate dispensing state recognition model and then applying it to real-time dispensing data analysis, thereby effectively controlling key states such as glue line shape, width, and thickness during the assembly process.

[0036] The step S110 in the method provided by the embodiment of the present application comprises:

[0037] According to a preset monitoring time interval, industrial cameras and three-dimensional laser scanners are used to synchronously collect dispensing images and dispensing point cloud data, to obtain a dispensing image sequence and a dispensing point cloud data sequence;

[0038] Based on historical bonding assembly records, sample dispensing image sets and sample dispensing point cloud data sets are collected, and the dispensing state under different sample dispensing images and sample dispensing point cloud data is labeled to obtain a dispensing state distribution label set, wherein the dispensing state at least includes glue line shape, glue line width, and thickness distribution;

[0039] The sample dispensing image sets and sample dispensing point cloud data sets are used as input, and the dispensing state distribution label set is used as supervision to train a convolutional neural network to convergence, to generate a dispensing state recognition plug-in;

[0040] The dispensing state recognition plug-in is used to recognize the dispensing state according to the dispensing image sequence and the dispensing point cloud data sequence, to output a dispensing state distribution sequence.

[0041] In the embodiment of the present application, in order to accurately recognize the dispensing state in the bonding assembly process of the 3C camera module and the glass cover plate, and to provide accurate basis for subsequent assembly optimization, sample data needs to be collected and a special recognition plug-in needs to be trained to realize efficient analysis and state output of real-time dispensing images and point cloud data.

[0042] Specifically, first, industrial cameras and three-dimensional laser scanners are used to synchronously collect dispensing images and dispensing point cloud data according to a preset monitoring time interval.

[0043] Among them, the industrial camera is responsible for capturing two-dimensional visual information of the dispensing area, which can clearly present the appearance form of the glue line, etc.; the three-dimensional laser scanner is used to obtain three-dimensional point cloud data of the dispensing area, which can accurately reflect the three-dimensional dimensional features such as thickness and width of the glue line.

[0044] Meanwhile, the point gluing images and point cloud data collected multiple times are sequentially integrated to obtain a point gluing image sequence and a point gluing point cloud data sequence, so as to comprehensively and continuously present the point gluing state change at different times and provide a data basis for subsequent analysis of the dynamic characteristics of the point gluing process.

[0045] Further, based on the historical bonding assembly records, a sample point gluing image set and a sample point gluing point cloud data set are collected.

[0046] Among them, the point gluing data under different working conditions and different product batches are covered in the historical bonding assembly records, and representative samples are selected to construct the sample point gluing image set and the sample point gluing point cloud data set.

[0047] Meanwhile, professional personnel are organized to label the point gluing state under different sample point gluing images and sample point gluing point cloud data, and the point gluing state at least includes glue line shape, glue line width, thickness distribution, etc., so as to finally obtain a point gluing state distribution label set, and one plane may include several point gluing positions, so as to ensure that the label set can comprehensively and carefully reflect the diversity of the point gluing state.

[0048] For example, from the bonding assembly records in the past year, samples of various typical conditions such as normal point gluing, glue line deviation, glue breakage, and bubbles can be selected, not less than 200 groups of sample point gluing images and sample point gluing point cloud data are collected for each condition, and then experienced process engineers label the samples one by one to clearly define the specific state parameters such as glue line shape, width, and thickness distribution corresponding to each sample, forming a point gluing state distribution label set to provide supervision information for subsequent model training.

[0049] Further, the sample point gluing image set and the sample point gluing point cloud data set are used as input, and the point gluing state distribution label set is used as supervision to train the convolutional neural network to convergence, and generate a point gluing state recognition plug-in.

[0050] Specifically, ResNet-18 is selected as the basic convolutional neural network architecture, the input layer adopts a double-branch structure, one branch is a 3-channel input layer to receive point gluing image data collected by an industrial camera, and the other branch is a multi-dimensional feature input layer to receive point gluing point cloud data collected by a three-dimensional laser scanner, and after the feature dimensions are unified by 1x1 convolution, the two are spliced and fused to realize the synchronous input of point gluing image and point cloud data.

[0051] In addition, the hidden layer of the network retains the original residual block structure of ResNet-18 to enhance the extraction ability of deep features, and the output layer sets a corresponding number of neurons according to the number of point gluing state categories, and adopts a Softmax activation function to make the output a probability distribution of each point gluing state.

[0052] In the model training stage, the sample dispensing image set, the sample dispensing point cloud data set and the dispensing state distribution label set are divided into a training set, a validation set and a test set in a ratio of 6:2:2. For example, 1200 groups are extracted from 2000 groups of sample data as the training set, 400 groups are extracted as the validation set, and 400 groups are extracted as the test set.

[0053] At the same time, the model is iteratively trained using the training set, and the cross-entropy loss function is used to measure the difference between the predicted results and the labels. The Adam optimizer (learning rate set to 0.0005) is used to update the network parameters.

[0054] Specifically, the model performance is evaluated using the validation set every 30 iterations. After the first iteration, the average recognition accuracy of each dispensing state on the validation set is 65%; when the iteration reaches 200 times, the average accuracy is improved to 83%; when the iteration reaches 400 times and the average accuracy on the validation set is stable above 95%, and the accuracy fluctuation is not more than 1% for 20 consecutive iterations, it is determined that the model converges, and the training of the dispensing state recognition plug-in is completed.

[0055] Further, based on the trained dispensing state recognition plug-in, the dispensing state recognition is performed according to the obtained dispensing image sequence and dispensing point cloud data sequence.

[0056] Specifically, the dispensing state recognition plug-in will quickly process the input dispensing image and point cloud data, extract the features related to the glue line shape, width, thickness distribution, etc., and then determine the current dispensing state based on the trained model parameters. Finally, the dispensing state distribution sequence is output, which can reflect the state change of each dispensing point in the assembly process in real time, and provide accurate dispensing state information for subsequent assembly error analysis and pressing parameter optimization.

[0057] For example, when the dispensing image collected at a certain time shows that the glue line has a slight curvature, and the point cloud data reflects that the glue line thickness has a small amplitude of unevenness in the local area, the dispensing state recognition plug-in will quickly extract these image and point cloud features, determine that the state is "slight curvature of glue line and local uneven thickness" based on the model parameters, and add this result to the dispensing state distribution sequence.

[0058] At the same time, as the assembly process progresses, the dispensing state distribution sequence will record the dispensing state of each point at different times in sequence, such as the state of "glue line straight but slightly narrow width" appearing at a subsequent point. The complete dispensing state distribution sequence can clearly present the state evolution of each point from the start of dispensing to the current time, so that engineers can accurately analyze the dispensing link problems caused by assembly errors based on the dispensing state distribution sequence, and then optimize the pressing parameters.

[0059] S120: Perform key state recognition and feature extraction on the point dispensing state distribution sequence, perform window fluctuation analysis of assembly error according to the key point dispensing state distribution sequence, and output a compensation time window sequence in a preset time zone;

[0060] In the error compensation scenario of the assembly of the 3C camera module and the glass cover plate, in order to timely and accurately determine the time window for assembly error compensation and to ensure assembly precision and product yield, the key state recognition, fluctuation analysis and prediction model construction are required to obtain a compensation time window sequence that can dynamically adapt to the assembly process.

[0061] Specifically, first, the output point dispensing state distribution sequence is subjected to key state recognition and feature extraction to obtain a key point dispensing state distribution sequence.

[0062] The key states include states such as glue line offset, glue breakage and thickness exceeding the normal range that may cause assembly errors, and the distribution data corresponding to these key states is filtered from the point dispensing state distribution sequence in a similar comparison manner with adjacent point dispensing state distributions to form the key point dispensing state distribution sequence.

[0063] Further, the interval duration of adjacent key point dispensing state distributions in the key point dispensing state distribution sequence is set as a fluctuation time window, and all interval durations are arranged in time sequence to obtain a fluctuation time window sequence.

[0064] Further, sample training data is collected based on historical assembly records. The historical records include corresponding fluctuation time window data and finally determined compensation time window data under different point dispensing states, and representative samples are filtered to construct a sample training data set.

[0065] Further, a deep neural network is trained to convergence based on the corresponding relationship between the fluctuation time window sequence in the historical assembly records and the compensation time window data in the sample training data to construct a compensation time window predictor.

[0066] Finally, the compensation time window predictor will accurately predict the compensation time window duration corresponding to each time period in the preset time zone according to the trained deep neural network and the change trend of the fluctuation time window sequence to form a compensation time window sequence, which provides a time dimension basis for subsequent assembly process optimization control.

[0067] The method provided in the embodiments of the present application comprises the following steps:

[0068] Perform key state recognition and feature extraction on the point dispensing state distribution sequence to obtain a key point dispensing state distribution sequence;

[0069] The interval duration of adjacent key dispensing state distributions in the key dispensing state distribution sequence is set as a fluctuation time window to obtain a fluctuation time window sequence.

[0070] Based on historical fitting assembly records, sample training data is collected, a deep neural network is trained to convergence, and a compensation time window predictor is constructed.

[0071] The compensation time window predictor is used to predict a compensation time window sequence in a preset time zone according to the fluctuation time window sequence.

[0072] In the embodiments of the present application, in order to accurately determine the compensation time window of the assembly error in the fitting assembly process of the 3C camera module and the glass cover plate, and provide a clear time basis for subsequent error compensation, the key state construction sequence needs to be extracted from the dispensing state distribution sequence, the fluctuation time window needs to be counted, and then the model is trained based on historical data to predict the compensation time window, so as to dynamically adapt to the assembly error accumulation law and ensure the assembly accuracy.

[0073] Specifically, first, the output dispensing state distribution sequence is subjected to key state recognition and feature extraction to obtain a key dispensing state distribution sequence, so as to accurately locate the key nodes that may cause assembly errors.

[0074] The method provided in the embodiments of the present application includes:

[0075] The first dispensing state distribution of the dispensing state distribution sequence is selected as a first key dispensing state distribution, and the adjacent dispensing state distribution of the first key dispensing state distribution is selected as a second dispensing state distribution.

[0076] The first key dispensing state distribution and the second dispensing state distribution are subjected to similarity comparison, if the similarity is greater than or equal to a preset similarity scalar, the second dispensing state distribution is discarded, and the first key dispensing state distribution is taken as a starting point to continue similarity comparison based on the dispensing state distribution sequence.

[0077] If the similarity is less than the preset similarity scalar, the second dispensing state distribution is set as a second key dispensing state distribution, and the second key dispensing state distribution is taken as a starting point to continue similarity comparison based on the dispensing state distribution sequence until the iteration is completed, and multiple key dispensing state distributions are combined to construct a key dispensing state distribution sequence.

[0078] In the embodiments of the present application, in order to accurately screen out the key glue line state that may cause assembly error in the assembly process of the 3C camera module and the glass cover plate, and provide accurate data basis for subsequent window fluctuation analysis of assembly error, the key state distribution is extracted from the continuous glue state distribution sequence through similar comparison of adjacent glue state distribution, so as to construct a key glue state distribution sequence that can reflect the change of key state.

[0079] Specifically, first, the first glue state distribution is selected from the glue state distribution sequence as the first key glue state distribution, and the adjacent glue state distribution of the first key glue state distribution is selected as the second glue state distribution.

[0080] The glue state distribution sequence is the glue state recorded in time sequence at each time, and the first glue state distribution corresponds to the glue line state at the beginning of assembly, which is set as the first key glue state distribution, and the next glue state distribution at the next time is the second glue state distribution.

[0081] Further, the first key glue state distribution and the second glue state distribution are compared. The similarity comparison can be realized by calculating the similarity index of the two in the feature dimensions such as glue line shape, width and thickness distribution.

[0082] Specifically, the cosine similarity algorithm is used to calculate the cosine similarity of the feature vectors of the first key glue state distribution and the second glue state distribution in the glue line shape, width and thickness distribution.

[0083] If the similarity of the two calculated is greater than or equal to the preset similarity scalar, it means that the second glue state distribution has little difference with the first key glue state distribution in the key features, and no key state change that may cause assembly error occurs.

[0084] Therefore, the second glue state distribution is discarded, and the first key glue state distribution is taken as the starting point to continue the similarity comparison with the next adjacent glue state distribution based on the glue state distribution sequence.

[0085] On the contrary, if the similarity of the two is less than the preset similarity scalar, it means that the second glue state distribution has a relatively obvious change in the key features such as glue line shape, width or thickness distribution compared with the first key glue state distribution, which belongs to the key state that may cause assembly error.

[0086] At this time, the second glue state distribution is set as the second key glue state distribution, and the second key glue state distribution is taken as the starting point to continue the similarity comparison with the next adjacent glue state distribution based on the glue state distribution sequence.

[0087] The above process is repeated until the entire dispensing state distribution sequence is traversed. Finally, all the key dispensing state distributions screened are combined to form a key dispensing state distribution sequence.

[0088] For example, assuming that there are 100 continuous dispensing state distributions in the dispensing state distribution sequence, the first dispensing state distribution is set as the first key dispensing state distribution, and then compared with the second dispensing state distribution. If the similarity is 0.9 (greater than the preset similarity scalar 0.85), the second dispensing state distribution is discarded, and the comparison between the first key dispensing state distribution and the third dispensing state distribution is continued.

[0089] Further, if the similarity with the third dispensing state distribution is 0.7 (less than the preset similarity scalar 0.85), the third dispensing state distribution is set as the second key dispensing state distribution, and then compared with the fourth dispensing state distribution.

[0090] In this way, a plurality of key dispensing state distributions are screened from the 100 dispensing state distributions, and a key dispensing state distribution sequence is formed after combination, which can accurately reflect the change of the key glue line state in the assembly process and provide core data support for subsequent analysis of the window fluctuation of the assembly error.

[0091] Further, the interval duration of adjacent key dispensing state distributions in the key dispensing state distribution sequence is counted, the interval duration is set as a fluctuation time window, and the interval duration of all adjacent key dispensing state distributions is arranged in time sequence to obtain a fluctuation time window sequence.

[0092] For example, in the key dispensing state distribution sequence, the recording time of the first key dispensing state distribution is 0 seconds after the start of assembly, and the recording time of the second key dispensing state distribution is 25 seconds. The interval duration of 25 seconds between the two is set as the first fluctuation time window.

[0093] Further, the interval duration between the second key dispensing state distribution and the third key dispensing state distribution is 20 seconds, which is set as the second fluctuation time window. In this way, all the interval durations are arranged in sequence to form a complete fluctuation time window sequence, which can intuitively reflect the time law of the change of the key glue line state and provide basic data in the time dimension for subsequent prediction of the compensation time window.

[0094] Further, based on the historical fitting assembly records, sample training data is collected, which needs to cover the fluctuation time window sequences under different production batches and different equipment working conditions, and the compensation time window sequences actually applied under the corresponding working conditions.

[0095] Specifically, from the historical fitting assembly records of the past year, production data containing key adhesive line state changes are screened, the fluctuation time window sequence is extracted as the input sample, and the compensation time window sequence verified effective in the corresponding scene is extracted as the output label to construct the sample training data set.

[0096] For example, for different key state change scenarios such as adhesive line deviation and broken adhesive, at least 300 corresponding fluctuation time window sequence and compensation time window sequence samples are collected for each scenario to ensure the diversity and representativeness of the sample data and provide sufficient and comprehensive learning basis for training the deep neural network.

[0097] Further, the deep neural network is trained to converge to construct a compensation time window predictor.

[0098] Specifically, a deep neural network architecture including an input layer, a hidden layer, and an output layer is selected, the number of input layer neurons matches the feature dimension of the fluctuation time window sequence, three layers of hidden layers are set (the number of neurons is 128, 64, and 32 respectively), the ReLU activation function is used to enhance the network's ability to extract nonlinear features, the number of output layer neurons corresponds to the dimension of the compensation time window sequence, the mean square error loss function is used to measure the difference between the predicted value and the label value, and the Adam optimizer (learning rate is set to 0.001) is used to update the network parameters.

[0099] In the model training phase, the sample training data is divided into a training set, a validation set, and a test set in a ratio of 7:2:1. For example, 2100 groups of sample data are extracted from 3000 groups of sample data as the training set for model parameter learning, 600 groups are used as the validation set for adjusting the model hyperparameters, and 300 groups are used as the test set for evaluating the final performance of the model.

[0100] At the same time, the prediction accuracy of the model is evaluated using the validation set every 50 iterations. Initially, the average error of the model's prediction of the compensation time window may be 5 seconds; as the number of iterations increases, the error gradually decreases, and when the iteration reaches 500 times, the average prediction error on the validation set stabilizes within 1.5 seconds, and the error fluctuation does not exceed 0.3 seconds for 30 consecutive iterations. The model is determined to have converged, and the network parameters at this time are saved, completing the construction of the compensation time window predictor.

[0101] Further, based on the trained compensation time window predictor, the fluctuation time window sequence is predicted to obtain the compensation time window sequence within the preset time zone.

[0102] Since the error accumulates continuously as the job time increases, the compensation window should be dynamically shortened according to the degree of error accumulation, and the compensation time window predictor will analyze the trend of the fluctuation time window sequence based on the trained model.

[0103] Specifically, if the fluctuation time window sequence presents a gradually shortened trend, indicating that the critical glue line state change frequency accelerates and the error accumulation speed increases, the compensation time window predictor will output a gradually shortened compensation time window sequence accordingly; if the fluctuation time window sequence is relatively stable, indicating that the error accumulation speed is gentle, the compensation time window predictor will output a relatively stable compensation time window sequence.

[0104] Exemplarily, for a certain preset assembly period of 1 hour, according to the input fluctuation time window sequence, the compensation time window predictor can output a compensation time window sequence of “30 seconds for the first 20 minutes, 25 seconds for the middle 20 minutes, and 20 seconds for the last 20 minutes”, providing a time basis for subsequent precise error compensation of the assembly process, to ensure that parameter adjustment can be made in time at different stages of error accumulation, and to ensure the assembly precision.

[0105] S130: Based on the key point glue state distribution sequence, a predicted key point glue state distribution sequence of the compensation time window sequence is obtained, and according to the predicted key point glue state distribution sequence, a pressing control parameter optimization is performed with the optimal camera optical performance as the target, and an optimal pressing parameter sequence is output.

[0106] In the embodiment of the application, in order to grasp the point glue state in the compensation time window in advance, and then adjust the pressing parameters to ensure that the camera optical performance reaches the optimal, a prediction model needs to be constructed to obtain a predicted point glue state distribution sequence, and based on the sequence, a pressing control parameter optimization is completed to output an optimal pressing parameter sequence that can adapt to different point glue states.

[0107] Specifically, first, in the compensation time window sequence, a first compensation time window is randomly selected by a random sampling method, and a first time interval between the first compensation time window and the current time node is calculated to clearly define the time dimension reference of the prediction.

[0108] Further, based on the historical assembly records, a sample key point glue state distribution sequence set and a sample time interval set are collected, and a historical point glue state distribution after different sample key point glue states in the sample time interval is obtained, which is set as a sample predicted point glue state distribution, to obtain a sample predicted point glue state distribution set.

[0109] Further, the sample key point glue state distribution sequence set and the sample time interval set are used as input, and the sample predicted point glue state distribution set is used as supervision, and the long short-term memory network is trained to convergence to obtain a point glue state distribution predictor.

[0110] Further, based on the constructed dispensing state distribution predictor, a first predicted dispensing state distribution is predicted according to the key dispensing state distribution sequence and the first time interval, and is added to the predicted dispensing state distribution sequence to gradually improve the dispensing state prediction result in the compensation time window.

[0111] Further, a first predicted dispensing state distribution is randomly selected from the predicted dispensing state distribution sequence and is used as a dispensing state benchmark for parameter optimization.

[0112] Subsequently, a pressing control parameter threshold of the bonding device is obtained. The pressing control parameter threshold is determined by the hardware performance of the device, and a first pressing control parameter is randomly selected within the safe operation parameter range and the process requirement range of the bonding device to ensure that the selected parameter meets the operation capability of the device.

[0113] Further, an optical performance predictor of the camera is constructed based on deep learning. That is, by using historical bonding assembly records, the attribute information of the 3C camera module and the glass cover plate is used as a constraint to collect a sample dispensing state distribution set, a sample pressing control parameter set, and to obtain a historical optical performance coefficient after bonding of different sample combinations, thereby forming a sample optical performance coefficient set.

[0114] Meanwhile, the deep learning model is trained using the above samples, and when the prediction accuracy of the optical performance coefficient of the model reaches 95% or more, the construction of the optical performance predictor is completed.

[0115] Further, the constructed optical performance predictor is used to perform optical performance prediction according to the first predicted dispensing state distribution and the first pressing control parameter, and a first optical performance coefficient is output to quantify the optical performance of the camera under the current pressing control parameter.

[0116] Finally, the iteration optimization is continued, the pressing control parameter is adjusted each time, and the optical performance prediction step is repeated until a preset convergence number is reached, and the pressing control parameter corresponding to the maximum optical performance coefficient is output as the first optimal pressing parameter and is added to the optimal pressing parameter sequence.

[0117] This step realizes the precision and foresight of parameter adjustment in the 3C camera module bonding assembly process through the progressive scheme of predicting the dispensing state of the compensation window and the targeted optimization of the pressing control parameter, which not only avoids the limitation that the traditional fixed pressing strategy cannot adapt to the dynamic dispensing state, but also outputs the pressing control parameter that adapts to different dispensing scenarios with the core target of optimizing the optical performance of the camera.

[0118] The step S130 in the method provided in the embodiments of the present application includes:

[0119] randomly selecting a first compensation time window from the sequence of compensation time windows, and calculating a first time interval between the first compensation time window and a current time node;

[0120] Based on the historical fitting assembly record, a sample key point glue state distribution sequence set and a sample time interval set are collected, and a historical point glue state distribution of different sample key point glue state distributions after a sample time interval is obtained as a sample predicted point glue state distribution, to obtain a sample predicted point glue state distribution set.

[0121] The sample key point glue state distribution sequence set and the sample time interval set are used as input, and the sample predicted point glue state distribution set is used as supervision to train the long short-term memory network to convergence, to obtain a point glue state distribution predictor.

[0122] The point glue state distribution predictor is used to predict a first predicted point glue state distribution from the key point glue state distribution sequence and the first time interval, and the first predicted point glue state distribution is added to the predicted point glue state distribution sequence.

[0123] A first predicted point glue state distribution is randomly selected from the predicted point glue state distribution sequence.

[0124] A compression control parameter threshold of the fitting device is obtained, and a first compression control parameter is randomly selected, wherein the compression control parameter at least includes a compression speed, a pressure distribution and a curing time.

[0125] An optical performance predictor of the camera is constructed based on deep learning.

[0126] The optical performance predictor is used to perform optical performance prediction according to the first predicted point glue state distribution and the first compression control parameter, and output a first optical performance coefficient.

[0127] The iteration optimization is continued until a preset convergence number is reached, and a compression control parameter corresponding to a maximum optical performance coefficient is output as a first optimal compression parameter, which is added to the optimal compression parameter sequence.

[0128] In the embodiments of the application, in order to predict the point glue state in the compensation time window and optimize the compression parameter in advance in the process of fitting and assembling the 3C camera module and the glass cover plate, ensure that the optical performance of the camera reaches the optimum, the predicted point glue state distribution sequence is obtained by constructing a point glue state prediction model, and the optimal compression parameter sequence suitable for different point glue scenes is output by combining optical performance prediction and parameter iteration optimization, to realize precise control and performance guarantee in the assembly process.

[0129] Specifically, first, a first compensation time window is randomly selected from the sequence of compensation time windows, and a first time interval between the first compensation time window and a current time node is calculated.

[0130] For example, if the compensation time window sequence includes three time periods of "0-5min, 8-12min, 15-18min", "8-12min" is randomly selected as the first compensation time window, and the current time is 3min after assembly starts, then the first time interval is 5min. In this way, the time dimension reference of the dispensing state prediction is determined, and a time reference for predicting the dispensing state in the subsequent compensation window is provided.

[0131] Further, based on the historical fitting assembly records, a sample key dispensing state distribution sequence set and a sample time interval set are collected, and a historical dispensing state distribution after different sample key dispensing state distributions in the sample time interval is obtained, which is set as a sample predicted dispensing state distribution, to obtain a sample predicted dispensing state distribution set.

[0132] The historical fitting assembly records cover assembly data in different product batches and different equipment conditions in the past 12 months. 1000 groups of data containing typical key states such as glue line deviation, uneven thickness, and broken glue are selected from the data, each group of data contains a key dispensing state distribution sequence at a certain time, a corresponding time interval, and a dispensing state distribution actually detected after the time interval, to form a sample key dispensing state distribution sequence set and a sample predicted dispensing state distribution set, so as to ensure that the sample data can cover various key state change scenarios and provide comprehensive learning basis for model training.

[0133] Further, the obtained sample key dispensing state distribution sequence set and sample time interval set are used as input, and the sample predicted dispensing state distribution set is used as supervision to train a long short-term memory network (LSTM) to convergence, to obtain a dispensing state distribution predictor.

[0134] Specifically, during training, the sample data is first divided into a training set (700 groups), a validation set (200 groups), and a test set (100 groups) in a ratio of 7:2:1. The mean square error (MSE) is used as the loss function, and the Adam optimizer (learning rate is set to 0.001) is used for iterative training.

[0135] At the same time, the prediction accuracy of the model is evaluated using the validation set every 50 iterations. The average prediction error of the validation set after the initial iteration is 8%. When the iteration reaches 300 times, the error decreases to 4%. When the iteration reaches 500 times and the error of the validation set is continuously stable within 3% for 25 times, it is determined that the model converges, the network parameters at this time are saved, and the construction of the dispensing state distribution predictor is completed, to ensure that the model can accurately predict the dispensing state after different time intervals.

[0136] Further, using the constructed dispensing state distribution predictor, the first predicted dispensing state distribution is predicted according to the key dispensing state distribution sequence and the first time interval, and is added to the predicted dispensing state distribution sequence.

[0137] Exemplarily, the current key point dispensing state distribution sequence (dispensing line offset 1mm, width 0.8mm, thickness 30μm) and the first time interval 5min are input into the constructed dispensing state distribution predictor, the long short-term memory network of the predictor extracts and analyzes the time sequence characteristics, combines the dispensing state change rule of the similar key point dispensing state sequence and the time interval in the historical sample, and outputs the first predicted dispensing state distribution "dispensing line offset 1.2mm, width 0.78mm, thickness 29.5μm", and adds the predicted dispensing state distribution sequence in time sequence.

[0138] Similarly, the above process is repeated to predict other time windows in the compensation time window sequence, gradually improve the predicted dispensing state distribution sequence, and form the dispensing state prediction result covering the entire preset assembly period, thereby providing accurate state benchmark for subsequent compression parameter optimization.

[0139] Further, after completing the construction of the predicted dispensing state distribution sequence, a first predicted dispensing state distribution is randomly selected in the sequence, such as "dispensing line offset 1.5mm, width 0.75mm, thickness 31μm", and used as the initial state basis for compression control parameter optimization, to ensure that the parameter optimization can be targeted to adapt to the actual possible dispensing state.

[0140] Subsequently, the compression control parameter threshold of the bonding device is obtained. The compression control parameter threshold is determined by the hardware performance of the device, for example, the safe operation range of the compression speed is 5-20mm / s, the effective adjustment range of the pressure distribution is 0.1-0.5MPa, and the process requirement range of the curing time is 10-30s.

[0141] Further, a first compression control parameter is randomly selected in the range of the compression control parameter threshold, such as "compression speed 12mm / s, pressure 0.3MPa, curing time 20s", to ensure that the selected compression control parameter meets the device operation capability and provides a reasonable initial value for subsequent performance prediction and parameter adjustment.

[0142] Further, an optical performance predictor of the camera is constructed based on deep learning to accurately predict the optical performance of the camera under different combinations of dispensing states and compression parameters.

[0143] In the method provided by the embodiments of the present application, the "optical performance predictor of the camera constructed based on deep learning" includes:

[0144] Based on historical fitting assembly records, the attribute information of the 3C camera module and the glass cover plate is taken as a constraint to collect a sample dispensing state distribution set and a sample compression control parameter set, and to obtain a historical optical performance coefficient after fitting of different sample dispensing state distributions and sample compression control parameters, which is set as a sample optical performance coefficient, to obtain a sample optical performance coefficient set;

[0145] The sample dispensing state distribution set, the sample compression control parameter set, and the sample optical performance coefficient set are taken as training data to train a deep learning model to convergence to obtain an optical performance predictor.

[0146] In the embodiments of the present application, in order to accurately predict the optical performance of the camera under different dispensing states and compression parameter combinations and provide reliable performance evaluation basis for subsequent compression control parameter optimization, a sample library needs to be constructed by collecting historical assembly data, and a deep learning model needs to be trained to form an optical performance predictor that can quickly output quantitative results of optical performance.

[0147] Specifically, first, based on historical fitting assembly records, the attribute information of the 3C camera module and the glass cover plate is taken as a constraint to collect a sample dispensing state distribution set and a sample compression control parameter set.

[0148] The attribute information of the 3C camera module includes module pixel specification, lens focal length, sensor size, etc., and the attribute information of the glass cover plate includes light transmittance, refractive index, thickness, etc. These attributes will directly affect the optical performance after fitting and need to be taken as constraint conditions to ensure the pertinence and effectiveness of the sample data.

[0149] Further, the sample dispensing state distribution set is collected, that is, samples covering multiple key states are selected from historical records, including glue line shape, glue line width, thickness distribution, etc., and no less than 50 samples are collected for each key state combination, to finally form a sample dispensing state distribution set containing 1000 groups of data. Each group of data is stored in the form of a parameter matrix, recording the specific state parameters of each dispensing point.

[0150] In addition, the collection of the sample compression control parameter set covers key controllable parameters of the fitting equipment, including compression speed, pressure distribution, and curing time, and the orthogonal test method is used to combine parameters to ensure that the compression control parameters cover the threshold range of the equipment compression control parameters and are uniformly distributed. A total of 800 samples of different parameter combinations are collected, and each group of parameters is labeled with the corresponding equipment operating condition to reduce the influence of irrelevant variables on the quality of the samples.

[0151] Further, the historical optical performance coefficient after fitting of different sample dispensing state distributions and sample compression control parameters is obtained, and is set as a sample optical performance coefficient set.

[0152] Meanwhile, the historical assembled products are detected by professional optical detection equipment to obtain quantitative indicators of optical performance, including transmittance (unit: %), resolution (unit: PPI), color restoration degree (unit: %), distortion rate (unit: %), etc.

[0153] Among them, the transmittance needs to reach more than 90%, the resolution needs to meet the standard corresponding to the module pixel, the color restoration degree needs to reach more than 85%, and the distortion rate needs to be controlled within 3%, which are quantified as optical performance coefficients and associated with the corresponding sample dispensing state distribution and sample pressing control parameters to form a sample optical performance coefficient set containing 1000 complete data.

[0154] For example, the sample dispensing state distribution is "glue line straight line, width 0.8 mm, thickness 30 μm", the sample pressing control parameter is "pressing speed 12 mm / s, pressure 0.3 MPa, curing time 20 s", and the adaptive 3C camera module is 4800 million pixels (corresponding to resolution standard 7200 PPI).

[0155] After detection, the transmittance of the assembled product is 94% (meeting the requirement of more than 90%, quantified as 0.94 according to "detection value / 100"), the resolution reaches 7200 PPI (meeting the module pixel standard, quantified as 1.0), the color restoration degree is 88% (meeting the requirement of more than 85%, quantified as 0.88), and the distortion rate is 2.1% (meeting the requirement of within 3%, quantified as 0.963 according to "(3-detection value) / 3"). This set of optical performance coefficients is bound to the corresponding dispensing state distribution and pressing control parameter as 1 group of data and included in the sample optical performance coefficient set.

[0156] Further, the sample dispensing state distribution set, the sample pressing control parameter set and the sample optical performance coefficient set are used as training data to train a deep learning model to convergence to obtain an optical performance predictor.

[0157] Specifically, a multi-layer perception (MLP) is selected as the deep learning model architecture, which can effectively process the mapping relationship between multi-dimensional input and multi-dimensional output, and adapt to the complex nonlinear relationship between dispensing state, pressing parameter and optical performance coefficient.

[0158] The model architecture is specifically set as follows: the number of input layer neurons matches the input feature dimension, the sample dispensing state distribution set contains 3 features (glue line shape, width, thickness), the sample pressing control parameter set contains 3 features (pressing speed, pressure, curing time), and there are 6 input features, so 6 neurons are set in the input layer; 3 hidden layers are set, with 128 neurons in the first layer, 64 neurons in the second layer and 32 neurons in the third layer, all using ReLU activation function to enhance the model's ability to extract nonlinear features and avoid gradient vanishing problem.

[0159] In addition, the number of output layer neurons corresponds to the dimension of the optical performance coefficient, and contains 4 output neurons corresponding to the optical performance coefficients of transmittance, resolution, color restoration degree and distortion rate. The output value is mapped to the 0-1 interval by using the Sigmoid activation function, which is consistent with the quantization range of the optical performance coefficient.

[0160] In the model training stage, 1000 groups of training data are divided into training set (700 groups), validation set (200 groups) and test set (100 groups) according to the ratio of 7:2:1. The root mean square error (RMSE) is used as the loss function to measure the difference between the predicted value of the model and the true value of the sample optical performance coefficient.

[0161] At the same time, the Adam optimizer is used for parameter updating, the learning rate is set to 0.0008, the batch size is set to 32, and the iteration number is set to 1000.

[0162] During the training process, the model performance is evaluated every 50 iterations using the validation set, and the RMSE change of the validation set is recorded. At the initial iteration (50th), the RMSE of the validation set is 0.08, indicating that the prediction error of the model is large; as the number of iterations increases, the error gradually decreases, and at the 300th iteration, the RMSE decreases to 0.05, and at the 500th iteration, the RMSE decreases to 0.03; when the iteration reaches 800 times, the RMSE of the validation set is stable within 0.02 for 30 consecutive times, and the RMSE of the test set is 0.025, which meets the prediction accuracy requirement (error less than 5%), and the model training converges, and the construction of the optical performance predictor is completed.

[0163] Finally, the constructed optical performance predictor can realize fast prediction, that is, when a certain group of dispensing state distribution and pressing control parameters are input, the model can output the corresponding optical performance coefficient in a short time, providing real-time and accurate performance evaluation support for subsequent pressing control parameter optimization, ensuring that the best pressing parameter combination can be selected as the target of optimizing the optical performance of the camera.

[0164] Further, the optical performance predictor is used to predict the optical performance according to the first predicted dispensing state distribution and the first pressing control parameter to output a first optical performance coefficient.

[0165] Specifically, the first predicted dispensing state distribution and the first pressing control parameter are input into the optical performance predictor in the form of a feature vector, the model receives 6-dimensional feature data through the input layer, performs nonlinear feature extraction and conversion through the ReLU activation function of 3 hidden layers, and finally outputs 4-dimensional optical performance coefficients through the Sigmoid activation function of the output layer. The optical performance of the camera under the current pressing control parameter combination is quantified, providing a clear basis for the evaluation of the pressing parameters.

[0166] On this basis, iteration optimization is continuously performed until a preset convergence number is reached, and the pressing control parameter corresponding to the maximum optical performance coefficient is set as the first optimal pressing parameter and is added to the optimal pressing parameter sequence.

[0167] In the iteration optimization process, based on the optical performance coefficient result of the current pressing control parameter each time, the pressing parameter is adjusted by a preset step, such as increasing or decreasing the pressing speed by 1 mm / s in the range of 5-20 mm / s, increasing or decreasing the pressure by 0.02 MPa in the range of 0.1-0.5 MPa, and increasing or decreasing the curing time by 1 s in the range of 10-30 s, to form a new pressing control parameter combination.

[0168] Further, the new pressing control parameter and the first predicted dispensing state distribution are input into the optical performance predictor again to obtain a new optical performance coefficient, which is compared with the optical performance coefficient of the last round, and the pressing control parameter combination with better optical performance is retained.

[0169] For example, if the preset convergence number is 100 times, the first optical performance coefficient comprehensive score is 0.91 (calculated according to the light transmittance, resolution, color restoration degree, and distortion rate coefficient, each accounting for 0.25 weight) at the initial iteration; when the iteration is 30 times, the comprehensive score of the adjusted pressing control parameter (pressing speed 13 mm / s, pressure 0.32 MPa, and curing time 21 s) is improved to 0.94.

[0170] When the iteration is 80 times, the comprehensive score of the pressing control parameter (pressing speed 14 mm / s, pressure 0.33 MPa, and curing time 22 s) reaches 0.96, and in the subsequent 20 iterations, the comprehensive score of the adjusted pressing control parameter fluctuates by less than 0.01, at which time the convergence condition is determined.

[0171] Finally, the pressing control parameter corresponding to the maximum comprehensive score (0.96) is output as the first optimal pressing parameter, which is associated with the first predicted dispensing state distribution in time sequence and is added to the optimal pressing parameter sequence to provide accurate parameter guidance for the pressing control under the same dispensing state in the subsequent fitting assembly process.

[0172] S140: Optimize the control of the fitting assembly process in the preset time zone according to the compensation time window sequence and the optimal pressing parameter sequence.

[0173] In the embodiments of the present application, in order to ensure that the pressing operation in each key period of the assembly process can adapt to the corresponding dispensing state, and thus ensure the overall assembly precision and the stability of the camera optical performance, the compensation time window sequence and the optimal pressing parameter sequence are applied to the assembly process through time sequence matching and real-time regulation to realize the fitting assembly optimization of the whole period and high precision.

[0174] Specifically, the timing correspondence between the compensation time window sequence and the optimal compression parameter sequence is first established.

[0175] The compensation time window sequence divides multiple key compensation periods within a preset time zone in chronological order, with each period corresponding to a specific dispensing state in the predicted dispensing state distribution sequence; while the optimal pressing parameter sequence generates suitable pressing control parameters for each predicted dispensing state.

[0176] Furthermore, by matching timestamps, each compensation time window is bound to the corresponding optimal pressing control parameter to form a "time-pressing control parameter" linkage control table, so as to clarify the pressing operation standards to be performed during different assembly periods.

[0177] Furthermore, during the bonding and assembly process, the current assembly time is monitored in real time. When the time enters a preset period in the compensation time window sequence, the optimal pressing parameters corresponding to that period are automatically called to control the bonding equipment to perform the pressing operation.

[0178] For example, when the assembly time changes from 7 minutes to 8 minutes (i.e., the start time of the "8-12 min" compensation time window), the assembly control platform will immediately read the optimal pressing parameters corresponding to this time window: "pressing speed 13 mm / s, pressure 0.32 MPa, curing time 21 s", and send pressing control parameter instructions to the drive module of the bonding equipment to adjust the pressing speed motor, pressure sensor and curing time controller of the equipment to ensure that the equipment operates strictly in accordance with the optimal pressing parameters during this period.

[0179] Meanwhile, during the pressing control process in each compensation time window, real-time dispensing status data and pressing process data are collected simultaneously to dynamically verify the adaptability of the parameters.

[0180] Specifically, using industrial cameras and 3D laser scanners, images and point cloud data of the current dispensing area are collected at preset monitoring time intervals. The trained dispensing state recognition plugin is used to identify the current dispensing state in real time and compare it with the predicted dispensing state corresponding to that time period.

[0181] If the deviation between the real-time dispensing state and the predicted state is less than a preset threshold, the current optimal pressing parameters are determined to be suitable, and the current control continues.

[0182] Conversely, if the deviation between the real-time dispensing state and the predicted state exceeds a preset threshold, such as a real-time glue line offset of 1.5mm (predicted offset of 1.2mm), the dispensing state distribution predictor is temporarily invoked to re-predict the dispensing state based on the real-time key dispensing state sequence and the remaining time interval, and to quickly match the backup pressing control parameters under similar historical scenarios to ensure that the pressing process always adapts to the actual dispensing state.

[0183] In addition, after the assembly process of the preset time zone is completed, the optimization control effect of the whole time period is reviewed and evaluated. The optical performance of the final assembly product is detected by an optical detection device, and the product yield and optical performance compliance rate are counted.

[0184] At the same time, compared with the assembly data under the control mode of the traditional fixed pressing parameter, if the product yield and the average compliance rate of the optical performance are improved after the optimization control of this step, the effectiveness of the optimization control scheme is verified, and the corresponding relationship between the "compensation time window-optimal pressing parameter" can be supplemented to the historical fitting assembly record to provide more sample data for subsequent model training.

[0185] This step converts the prediction results and the pressing parameter optimization results of the previous period into precise control actions in the actual assembly process, which not only solves the problem of fixed parameters and slow response in traditional assembly, but also ensures the adaptability of the pressing control parameters through real-time monitoring and dynamic adjustment, and finally realizes high-precision and high-performance production of 3C camera module and glass cover plate fitting assembly.

[0186] Through the specific embodiments described above, the embodiments of the present application achieve the following technical effects:

[0187] The present application provides a deep learning compensation method for mechanical assembly errors. First, point glue images and point cloud data are synchronously collected by an industrial camera and a three-dimensional laser scanner according to a preset monitoring time interval, and a point glue state recognition plug-in trained by a convolutional neural network is combined to output a point glue state distribution sequence. Then, key state recognition and feature extraction are performed on the sequence, the key point glue state distribution sequence is selected through similarity comparison, the interval duration of adjacent key states is counted to form a fluctuation time window sequence, a deep neural network is trained based on historical data to construct a compensation time window predictor, and a compensation time window sequence in a preset time zone is output. Then, a window is randomly selected in the compensation time window sequence and the interval with the current time is calculated, a long short-term memory network is trained based on historical samples to obtain a point glue state distribution predictor, the point glue state in the compensation window is predicted to form a predicted point glue state distribution sequence. Then, a reference state is selected in the predicted sequence, a pressing control parameter threshold of a fitting device is obtained, and a first pressing control parameter is randomly selected. An optical performance predictor is constructed based on deep learning, and an optimal pressing parameter sequence is output through iterative optimization. Finally, the assembly process is optimized and controlled in the whole time period according to the time sequence correspondence between the compensation time window and the optimal pressing parameter, the real-time state is monitored synchronously, and the parameters are dynamically adjusted. The optimization control effect is evaluated after assembly.

[0188] The method provided in this application solves the problems in traditional mechanical assembly caused by the dynamic changes in the dispensing state leading to delayed error compensation, the inability of fixed pressing parameters to adapt to different scenarios, and the difficulty in balancing assembly accuracy and optical performance. It achieves forward-looking compensation for assembly errors and stable assurance of camera optical performance, while reducing invalid data collection and redundant parameter debugging, significantly improving the efficiency of 3C camera module bonding assembly, product yield, and optical performance compliance rate.

[0189] Example 2, as shown in the appendix Figure 2 As shown, based on the inventive concept of a deep learning compensation method for mechanical assembly errors provided in Embodiment 1, this application also provides a deep learning compensation system for mechanical assembly errors, specifically including:

[0190] The dispensing data acquisition module 01 is used to synchronously acquire dispensing images and dispensing point cloud data using an industrial camera and a 3D laser scanner according to a preset monitoring time interval during the bonding and assembly process of the 3C camera module and the glass cover plate, and to perform dispensing status recognition and output the dispensing status distribution sequence.

[0191] The window fluctuation analysis module 02 is used to identify key states and extract features from the dispensing state distribution sequence, perform window fluctuation analysis of assembly errors based on the key dispensing state distribution sequence, and output a compensation time window sequence within a preset time zone.

[0192] The pressing parameter optimization module 03 is used to obtain the predicted dispensing state distribution sequence of the compensation time window sequence based on the predicted dispensing state distribution sequence, and optimize the pressing control parameters according to the predicted dispensing state distribution sequence with the goal of optimizing the optical performance of the camera, and output the optimal pressing parameter sequence.

[0193] Assembly optimization control module 04 is used to optimize and control the bonding assembly process within the preset time zone according to the compensation time window sequence and the optimal pressing parameter sequence.

[0194] In one embodiment, the dispensing data acquisition module 01 is further configured to:

[0195] According to a preset monitoring time interval, point gluing image and point gluing point cloud data are synchronously collected by using an industrial camera and a three-dimensional laser scanner to obtain a point gluing image sequence and a point gluing point cloud data sequence; based on historical fitting assembly records, a sample point gluing image set and a sample point gluing point cloud data set are collected, and the point gluing states under different sample point gluing images and sample point gluing point cloud data are labeled to obtain a point gluing state distribution label set, wherein the point gluing state at least includes a glue line shape, a glue line width and a thickness distribution; the sample point gluing image set and the sample point gluing point cloud data set are used as input, and the point gluing state distribution label set is used as supervision to train a convolutional neural network to convergence to generate a point gluing state recognition plug-in; the point gluing state recognition plug-in is used to perform point gluing state recognition according to the point gluing image sequence and the point gluing point cloud data sequence, and a point gluing state distribution sequence is output.

[0196] In one embodiment, the window fluctuation analysis module 02 is further configured to:

[0197] The point gluing state distribution sequence is subjected to key state recognition and feature extraction to obtain a key point gluing state distribution sequence; the interval time length of adjacent key point gluing states in the key point gluing state distribution sequence is set as a fluctuation time window to obtain a fluctuation time window sequence; sample training data are collected based on historical fitting assembly records, and a deep neural network is trained to convergence to construct a compensation time window predictor; the compensation time window predictor is used to predict a compensation time window sequence in a preset time zone based on the fluctuation time window sequence.

[0198] Further, the window fluctuation analysis module 02 further comprises:

[0199] The first point gluing state distribution of the point gluing state distribution sequence is selected as a first key point gluing state distribution, and the adjacent point gluing state distribution of the first key point gluing state distribution is selected as a second point gluing state distribution; the first key point gluing state distribution and the second point gluing state distribution are subjected to similarity comparison, if the similarity of the two is greater than or equal to a preset similarity scalar, the second point gluing state distribution is discarded, and the first key point gluing state distribution is taken as a starting point to continue similarity comparison based on the point gluing state distribution sequence; if the similarity of the two is less than the preset similarity scalar, the second point gluing state distribution is taken as a second key point gluing state distribution, and the second key point gluing state distribution is taken as a starting point to continue similarity comparison based on the point gluing state distribution sequence until the traversal is completed, and a plurality of key point gluing state distributions are merged to construct a key point gluing state distribution sequence.

[0200] In one embodiment, the compression parameter optimization module 03 is further configured to:

[0201] randomly selecting a first compensation time window from the sequence of compensation time windows, calculating a first time interval between the first compensation time window and a current time node; based on historical fitting assembly records, collecting a sample key point glue state distribution sequence set and a sample time interval set, and obtaining historical point glue state distributions of different sample key point glue state distributions after sample time intervals as sample predicted point glue state distributions, to obtain a sample predicted point glue state distribution set; using the sample key point glue state distribution sequence set and the sample time interval set as input, using the sample predicted point glue state distribution set as supervision, training a long short-term memory network to convergence, to obtain a point glue state distribution predictor; using the point glue state distribution predictor, predicting a first predicted point glue state distribution according to the key point glue state distribution sequence and the first time interval, and adding the first predicted point glue state distribution to the predicted point glue state distribution sequence. Randomly selecting a first predicted point glue state distribution from the predicted point glue state distribution sequence; obtaining a compression control parameter threshold of a fitting device, and randomly selecting a first compression control parameter, wherein the compression control parameter at least includes a compression speed, a pressure distribution and a curing time; constructing an optical performance predictor of a camera based on deep learning; using the optical performance predictor, performing optical performance prediction according to the first predicted point glue state distribution and the first compression control parameter, and outputting a first optical performance coefficient; continuing iteration optimization until a preset convergence number is reached, and outputting a compression control parameter corresponding to a maximum optical performance coefficient as a first optimal compression parameter, and adding the first optimal compression parameter to the optimal compression parameter sequence.

[0202] Further, the compression parameter optimization module 03 further comprises:

[0203] Based on the historical fitting assembly records, the attribute information of the 3C camera module and the glass cover plate is used as a constraint to collect a sample point glue state distribution set and a sample compression control parameter set, and to obtain historical optical performance coefficients of different sample point glue state distributions and sample compression control parameters after fitting as sample optical performance coefficients, to obtain a sample optical performance coefficient set; using the sample point glue state distribution set, the sample compression control parameter set and the sample optical performance coefficient set as training data, training a deep learning model to convergence to obtain an optical performance predictor.

[0204] It should be noted that the above-mentioned sequence of embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. And the above describes a specific embodiment of the present application. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0205] The above description is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0206] The specification and drawings are only exemplary and illustrative of the present application and are considered to cover any and all modifications, variations, combinations or equivalents that are within the scope of the present application. Obviously, various modifications and changes can be made to the present application by those skilled in the art without departing from the scope of the present application. Thus, it is intended that the present application cover the modifications and changes as they come within the scope of the application, and that the scope of the application be defined not by the description of the application, but by the claims.

Claims

1. A deep learning compensation method for mechanical assembly errors, characterized in that, The method comprises: In the process of assembling the 3C camera module and the glass cover plate, point gluing images and point gluing point cloud data are synchronously collected by using an industrial camera and a three-dimensional laser scanner at preset monitoring time intervals, and point gluing state recognition is performed to output a point gluing state distribution sequence; Key state recognition and feature extraction are performed on the point gluing state distribution sequence, window fluctuation analysis of assembly errors is performed according to the key point gluing state distribution sequence, and a compensation time window sequence in a preset time zone is output; A predicted point gluing state distribution sequence of the compensation time window sequence is obtained based on the key point gluing state distribution sequence, and optimal compression control parameter optimization is performed according to the predicted point gluing state distribution sequence to obtain an optimal compression parameter sequence, including: A first predicted point gluing state distribution is randomly selected from the predicted point gluing state distribution sequence; A compression control parameter threshold of the assembly equipment is obtained, and a first compression control parameter is randomly selected, wherein the compression control parameter at least includes a compression speed, a pressure distribution and a curing time; An optical performance predictor of the camera is constructed based on deep learning, including: Based on historical assembly records, a sample point gluing state distribution set and a sample compression control parameter set are collected by taking the attribute information of the 3C camera module and the glass cover plate as a constraint, and a historical optical performance coefficient after assembly of different sample point gluing state distributions and sample compression control parameters is obtained as a sample optical performance coefficient, to obtain a sample optical performance coefficient set; The sample point gluing state distribution set, the sample compression control parameter set and the sample optical performance coefficient set are used as training data to train a deep learning model to convergence to obtain the optical performance predictor; The optical performance predictor is used to perform optical performance prediction according to the first predicted point gluing state distribution and the first compression control parameter, and a first optical performance coefficient is output; Iterative optimization is continued until a preset convergence number is reached, and a compression control parameter corresponding to a maximum optical performance coefficient is set as a first optimal compression parameter and is added to the optimal compression parameter sequence; The assembly process in the preset time zone is controlled according to the compensation time window sequence and the optimal compression parameter sequence. 2.The method of claim 1, wherein, Point gluing images and point gluing point cloud data are synchronously collected by using an industrial camera and a three-dimensional laser scanner, and point gluing state recognition is performed to output a point gluing state distribution sequence, including: Point gluing images and point gluing point cloud data are synchronously collected by using an industrial camera and a three-dimensional laser scanner at preset monitoring time intervals to obtain a point gluing image sequence and a point gluing point cloud data sequence; Based on historical assembly records, a sample point gluing image set and a sample point gluing point cloud data set are collected, and point gluing states under different sample point gluing images and sample point gluing point cloud data are labeled to obtain a point gluing state distribution label set, wherein the point gluing state at least includes a glue line shape, a glue line width and a thickness distribution; The sample point gluing image set and the sample point gluing point cloud data set are used as input, and the point gluing state distribution label set is used as supervision to train a convolutional neural network to convergence to generate a point gluing state recognition plug-in; The point gluing state recognition plug-in is used to recognize the point gluing state according to the point gluing image sequence and the point gluing point cloud data sequence, and output a point gluing state distribution sequence. 3.The method of claim 1, wherein, The key state recognition and feature extraction are performed on the point gluing state distribution sequence, the window fluctuation analysis of the assembly error is performed according to the key point gluing state distribution sequence, and a compensation time window sequence in a preset time zone is output, including: The key state recognition and feature extraction are performed on the point gluing state distribution sequence, and a key point gluing state distribution sequence is obtained. The interval time length of adjacent key point gluing state distributions in the key point gluing state distribution sequence is set as a fluctuation time window, and a fluctuation time window sequence is obtained. Based on historical fitting assembly records, sample training data is collected, a deep neural network is trained to convergence, and a compensation time window predictor is constructed. The compensation time window predictor is used to predict the compensation time window sequence in the preset time zone according to the fluctuation time window sequence.

4. The method of claim 3, wherein, The key state recognition and feature extraction are performed on the point gluing state distribution sequence, and a key point gluing state distribution sequence is obtained, including: The first point gluing state distribution of the point gluing state distribution sequence is selected as a first key point gluing state distribution, and the adjacent point gluing state distribution of the first key point gluing state distribution is selected as a second point gluing state distribution. The first key point gluing state distribution and the second point gluing state distribution are compared, if the similarity is greater than or equal to a preset similarity scalar, the second point gluing state distribution is discarded, and the first key point gluing state distribution is taken as a starting point to continue the similarity comparison based on the point gluing state distribution sequence. If the similarity is less than the preset similarity scalar, the second point gluing state distribution is set as a second key point gluing state distribution, and the second key point gluing state distribution is taken as a starting point to continue the similarity comparison based on the point gluing state distribution sequence until the iteration is completed, and a plurality of key point gluing state distributions are combined to construct a key point gluing state distribution sequence.

5. The method of claim 1, wherein, Based on the key point gluing state distribution sequence, a prediction point gluing state distribution sequence of the compensation time window sequence is obtained, including: A first compensation time window is randomly selected in the compensation time window sequence, and a first time interval between the first compensation time window and a current time node is calculated. Based on historical fitting assembly records, a sample key point gluing state distribution sequence set and a sample time interval set are collected, and a historical point gluing state distribution after a sample time interval of different sample key point gluing state distributions is obtained as a sample prediction point gluing state distribution, and a sample prediction point gluing state distribution set is obtained. The sample key point gluing state distribution sequence set and the sample time interval set are used as input, the sample prediction point gluing state distribution set is used as supervision, a long short-term memory network is trained to convergence, and a point gluing state distribution predictor is obtained. The point gluing state distribution predictor is used to predict a first prediction point gluing state distribution according to the key point gluing state distribution sequence and the first time interval, and the first prediction point gluing state distribution is added to the prediction point gluing state distribution sequence.

6. A deep learning compensation system for mechanical assembly errors, characterized by, The system is used to execute the deep learning compensation method of the mechanical assembly error according to any one of claims 1-5, and the system includes: The dispensing data acquisition module is configured to, during the assembly of the 3C camera module and the glass cover plate, acquire dispensing images and dispensing point cloud data by using an industrial camera and a three-dimensional laser scanner at a preset monitoring time interval, identify dispensing states, and output a dispensing state distribution sequence. The window fluctuation analysis module is configured to identify key states and extract features from the dispensing state distribution sequence, perform window fluctuation analysis of assembly errors based on the key dispensing state distribution sequence, and output a compensation time window sequence in a preset time zone. The compression parameter optimization module is configured to predict a predicted dispensing state distribution sequence of the compensation time window sequence based on the key dispensing state distribution sequence, perform compression control parameter optimization based on the predicted dispensing state distribution sequence to optimize the optical performance of the camera, and output an optimal compression parameter sequence, including: randomly selecting a first predicted dispensing state distribution from the predicted dispensing state distribution sequence; obtaining compression control parameter thresholds of the assembly device and randomly selecting a first compression control parameter, wherein the compression control parameter at least includes a compression speed, a pressure distribution, and a curing time; constructing an optical performance predictor of the camera based on deep learning, including: based on historical assembly records, collecting a sample dispensing state distribution set and a sample compression control parameter set with the attribute information of the 3C camera module and the glass cover plate as constraints, and obtaining historical optical performance coefficients after the assembly of different sample dispensing state distributions and sample compression control parameters, which are set as sample optical performance coefficients, to obtain a sample optical performance coefficient set; using the sample dispensing state distribution set, the sample compression control parameter set, and the sample optical performance coefficient set as training data to train a deep learning model to convergence to obtain the optical performance predictor; using the optical performance predictor to predict the optical performance based on the first predicted dispensing state distribution and the first compression control parameter, and outputting a first optical performance coefficient; continuing the iterative optimization until a preset convergence number is reached, and outputting a compression control parameter corresponding to the maximum optical performance coefficient as the first optimal compression parameter, which is added to the optimal compression parameter sequence; The assembly optimization control module is configured to optimize the assembly process in the preset time zone based on the compensation time window sequence and the optimal compression parameter sequence.

Images