Intelligent testing method and equipment for mobile phone accessories production

Abstract

The present invention discloses an intelligent testing method and equipment for the production of mobile phone accessories, which relates to the field of image processing technology. The method comprises: dynamically detecting mobile phone display screen accessories through an intelligent optical detection platform to obtain optical images; activating a stage recognition classifier to analyze image features and identify the target production stage; using an intelligent test model to test the image features of the target stage to obtain test results; when the test results do not meet the production standards, an early warning signal is issued, and the production process is controlled in real time based on the signal. The method solves the technical problem that the existing technology cannot dynamically perform accurate detection and real-time early warning for different production stages, thereby causing defects in the production process of mobile phone accessories to be unable to be discovered and processed in a timely manner, and achieves the technical effect of improving the quality control and efficiency of the production process.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an intelligent testing method and device for the production of mobile phone accessories. Background Art

[0002] In the production of mobile phone accessories, especially in the manufacturing process of display accessories, traditional manual inspection methods have problems such as low efficiency and poor accuracy, and are difficult to adapt to the needs of modern large-scale production. With the increase in the types and complexity of product defects, traditional inspection methods are usually unable to effectively identify all potential defects, resulting in omissions or misjudgments. In order to improve detection accuracy and production efficiency, intelligent inspection technology has emerged. By using an intelligent optical inspection platform to dynamically monitor display accessories, combined with image processing and integrated learning methods in artificial intelligence, comprehensive monitoring and real-time early warning of the production process can be achieved, ensuring quality control at each production stage, reducing human intervention, and improving product qualification rate. Through automated detection and real-time early warning mechanisms, this solution effectively improves the automation level of the production line and optimizes the production process.

[0003] At present, relevant technologies have the technical problem of being unable to carry out accurate detection and real-time warning dynamically at different production stages, resulting in the inability to timely discover and deal with defects in the production process of mobile phone accessories. Summary of the invention

[0004] This application solves the technical problem in the prior art that defects in the production process of mobile phone accessories cannot be discovered and handled in a timely manner, by providing an intelligent testing method and equipment for the production of mobile phone accessories.

[0005] This application provides an intelligent testing method for mobile phone accessories production, including:

[0006] The production process of mobile phone display screen accessories is dynamically detected through a predetermined intelligent optical detection platform to obtain an optical image of the display screen; an activation stage recognition classifier is used to identify, judge and analyze a first image feature information group of the optical image of the display screen to obtain a target production stage corresponding to the optical image of the display screen; a second image feature information group of the optical image of the display screen is tested and analyzed through a target stage test channel of the target production stage matched in an intelligent integrated fusion test model to obtain a target test result; when the target test result does not meet the target production constraint corresponding to the target production stage, a real-time warning signal is issued; and real-time quality control intervention is performed on the mobile phone display screen accessories based on the real-time warning signal.

[0007] The present application also provides an electronic device, comprising: a memory for storing executable instructions; and a processor for implementing an intelligent testing method for mobile phone accessory production when executing the executable instructions stored in the memory.

[0008] The intelligent testing method and equipment for mobile phone accessories production proposed in this application firstly conducts dynamic detection of mobile phone display accessories through an intelligent optical detection platform to obtain optical images; activates the stage recognition classifier to analyze image features and identifies the target production stage; uses the intelligent testing model to test the image features of the target stage to obtain test results; when the test results do not meet the production standards, an early warning signal is issued, and based on this signal, real-time quality control of the production process is performed, thereby achieving the technical effect of improving the quality control and efficiency of the production process by realizing accurate detection and real-time early warning for different production stages in the production process of mobile phone accessories. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the accompanying drawings of the embodiment of the present invention will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0010] Figure 1 A schematic diagram of a process of an intelligent testing method and device for producing mobile phone accessories provided in an embodiment of the present application;

[0011] Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0012] Description of the reference numerals: input device 401 , processor 402 , memory 403 , output device 404 . DETAILED DESCRIPTION

[0013] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0014] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0015] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0016] The present application embodiment provides an intelligent testing method for mobile phone accessories production, such as Figure 1 As shown, the method includes:

[0017] Step S100, dynamically detect the production process of mobile phone display screen accessories through a predetermined intelligent optical detection platform to obtain an optical image of the display screen. Specifically, when building a predetermined intelligent optical detection platform, carefully select and install hardware equipment, adjust the parameters of high-resolution optical camera lenses, equip with a multi-angle stable high-brightness lighting system and a high-precision low-noise image sensor, strictly debug the hardware to ensure that the connection, transmission and acquisition are correct, and configure a software system with acquisition control, preprocessing, and storage management functions. Design shooting parameters and optimize algorithms according to production requirements. During production, when the accessories arrive at the detection position, the sensor or beat signal triggers the detection, and the camera shoots according to the preset parameters. Large-size accessories are photographed in full or in different areas and then spliced. The image is transmitted to the processing unit via a high-speed line. After verification and error correction, the noise is first reduced, the contrast and sharpness are enhanced, and the features are highlighted. The preliminarily processed image is stored in the designated area to lay the foundation for subsequent work.

[0018] Step S200, the stage recognition classifier is activated to identify, judge and analyze the first image feature information group of the display screen optical image, and obtain the target production stage corresponding to the display screen optical image. Specifically, the stage recognition classifier is activated immediately after the display screen optical image is obtained. Previously, the classifier has been trained by a large number of display screen image samples with annotated production stages. For the input image, texture, color and shape analysis techniques are first used to extract various features and integrate and quantify them into the first image feature information group, such as obtaining the display screen texture features by the texture analysis algorithm, determining the color distribution by the color feature extraction method, and finding the edge contour by the shape feature recognition technology to calculate the geometric shape features. The classifier adopts advanced algorithms such as convolutional neural networks, whose convolution layers extract high-level abstract features, learn deep patterns through multi-layer stacking, and then match the extracted features with the standard feature patterns of each stage obtained by training, and determine which stage standard features are closest to the similarity measurement method, and then determine the target production stage corresponding to the display screen optical image, providing a key basis for subsequent test analysis.

[0019] In a possible implementation, the activation stage identification classifier identifies and judges the first image feature information group of the display screen optical image to obtain the target production stage corresponding to the display screen optical image, and step S200 further includes step S210, forming a display screen image set. Specifically, it is necessary to determine the image acquisition source and range for forming the display screen image set. Images can be collected from different production links and different batches of products on the mobile phone display screen accessory production line. For example, in the substrate manufacturing link, the substrate image being produced is collected at regular time intervals; in the coating link, the display screen coated with different materials and different thickness coatings is imaged; in the assembly link, the overall image of the display screen at different assembly stages and after assembly is taken. At the same time, some display screen images in historical production data can also be collected. These images cover various situations under normal production conditions and images that have had quality problems, so as to make the image set more comprehensive and representative. In addition, in order to ensure the diversity of the image set, image acquisition can also be performed under different lighting conditions and shooting angles to simulate various situations that may occur in the actual production environment, thereby constructing a rich display screen image set to provide sufficient data basis for subsequent training and analysis.

[0020] Step S220, the stage of the predetermined production stage is marked for any display screen image in the display screen image set to obtain the marking information of any stage. Specifically, for any display screen image in the display screen image set, it is necessary to accurately mark the stage according to the predetermined production stage to obtain the marking information of any stage. Detailed marking standards are formulated to clarify the characteristics and boundaries of each production stage. For example, in the substrate manufacturing stage, the image marked as this stage should show the original material form of the substrate, the shape and structure after preliminary processing, etc.; the image of the coating stage should reflect the characteristics of the coating starting to cover and the change of coating uniformity; the image of the assembly stage should be able to reflect the process of the combination and connection of each component and the overall effect of the final assembly. The marking method can adopt a combination of manual marking and automatic marking. For some images with obvious features and easy to judge, automatic marking can be performed by writing a specific algorithm to improve the marking efficiency. For some complex images that are difficult to accurately judge by algorithms, they are manually marked by experienced technicians to ensure the accuracy of the marking. In the marking process, the marking information of each image should be recorded in detail, including the image number, acquisition time, production batch and corresponding production stage, etc., for subsequent data management and analysis.

[0021] Step S230, forming the first training data based on the first arbitrary image feature information group of the arbitrary display screen image and the arbitrary stage annotation information. Specifically, forming the first training data based on the first arbitrary image feature information group of the arbitrary display screen image and the arbitrary stage annotation information. When extracting the first arbitrary image feature information group, a variety of image analysis techniques should be used in combination. From the aspect of texture features, the texture direction and complexity of the display screen surface are determined by calculating the rate of change of the pixel grayscale value in different directions in the image. For example, the substrate in the substrate manufacturing stage may have a rougher texture, while the coating in the coating stage may make the texture smooth. In terms of color features, the image is converted to the LAB color space, and the statistical features of the three channels of L (brightness), A (color component from green to red), and B (color component from blue to yellow) are calculated respectively, such as mean, variance, etc., because different production stages may have different color performances, such as more color components may be combined together in the assembly stage. For shape features, edge detection algorithms, such as Canny edge detection, are used to extract the edge contours of the display screen and internal components, and then the geometric features of the contours, such as perimeter, area, circularity, etc., are calculated. The shape of the display screen at different stages will change due to processing and assembly. These texture, color, shape and other feature information are combined together to form the first arbitrary image feature information group, which is then paired with the corresponding arbitrary stage annotation information to construct the first training data, providing data input for subsequent machine learning.

[0022] Step S240, preprocessing the first training data to obtain first preprocessed training data, and performing supervised machine learning on the first preprocessed training data to obtain the stage identification classifier. Specifically, preprocessing the first training data to obtain first preprocessed training data. First, check the integrity of the data to see if there is any missing image data, incorrect annotation information, etc., and repair or remove the problematic data. Then, perform data normalization processing to uniformly adjust the numerical ranges of different features. For example, the numerical range of the color feature is normalized from 0-255 to 0-1 so that it can be better calculated and compared in the machine learning algorithm. Next, standardize the data so that the data conforms to a specific statistical distribution, such as a normal distribution, which can improve the convergence speed and accuracy of the machine learning algorithm. In addition, data enhancement technology can also be used to rotate, flip, scale, etc. the image data to increase the number and diversity of training data and avoid overfitting. Supervised machine learning is performed on the first preprocessed training data to obtain a stage identification classifier. Select a suitable machine learning algorithm, such as a support vector machine (SVM) or a convolutional neural network (CNN) in deep learning. If SVM is used, it is necessary to determine the type of kernel function, such as linear kernel, polynomial kernel or Gaussian kernel, and select and optimize it according to the characteristics of the training data. By inputting the preprocessed training data into SVM, the parameters of the model, such as penalty coefficient, kernel function parameters, etc., are continuously adjusted so that the model can accurately distinguish different production stages according to the image feature information group. If CNN is used, it is necessary to build a network structure, including the design of convolution layer, pooling layer, fully connected layer, etc. The convolution layer is used to extract the local features of the image, the pooling layer is used to reduce the data dimension, and the fully connected layer is used for classification judgment. During the training process, the back propagation algorithm is used to continuously adjust the weights and biases of the network to minimize the error between the network's prediction results and the actual annotation information. After training and optimization of a large amount of data, a stage recognition classifier that can accurately identify the corresponding production stage of the display screen image is finally obtained.

[0023] In a possible implementation, based on the first arbitrary image feature information group of the arbitrary display screen image and the arbitrary stage annotation information, the first training data is formed, and step S230 further includes step S231, performing semantic segmentation and slicing on the arbitrary display screen image to obtain an arbitrary slicing result, and the arbitrary slicing result includes an arbitrary display screen slice and an arbitrary background slice. Specifically, for semantic segmentation and slicing of arbitrary display screen images, it is necessary to select a suitable semantic segmentation algorithm, such as the U-Net architecture based on deep learning. U-Net has a unique encoder-decoder structure. The encoder part can extract features from the image, gradually reduce the image resolution and obtain high-level semantic information; the decoder part gradually samples the low-resolution feature map to restore it to the original image size, and at the same time fuses the feature information of the corresponding level in the encoder, thereby realizing accurate segmentation of different semantic areas in the image. Before applying the algorithm, it is necessary to train it, using a large number of image samples with the display screen and background areas labeled, so that the algorithm learns how to accurately distinguish the display screen from the background. After the training is completed, the arbitrary display screen image is input into the trained semantic segmentation algorithm model. After being processed by the semantic segmentation algorithm, an arbitrary slicing result is obtained, including an arbitrary display screen slice and an arbitrary background slice. For example, in an image of a mobile phone display screen accessory, the algorithm can accurately segment the display screen area from the entire image background and generate a separate display screen slice, which only contains pixel information of the display screen; at the same time, it also generates a corresponding background slice, which contains pixel information of the surrounding environment except the display screen. The slice results provide basic data for subsequent feature analysis of the display screen and background.

[0024] Step S232, collect and analyze the color features of the arbitrary display screen slice to obtain an arbitrary color feature value. Specifically, for any display screen slice, when collecting and analyzing the color features, first convert the image from the common RGB color space to other spaces that are more suitable for color feature description, such as the HSV (hue, saturation, brightness) color space. In the HSV space, hue (H) can intuitively reflect the category of the color, saturation (S) represents the purity of the color, and brightness (V) reflects the brightness of the color. Perform color feature statistics on the converted HSV image, calculate the proportion of the number of pixels of each hue value, for example, calculate the proportion of blue hue pixels in the image to the total pixels; and calculate the mean, variance and other statistics of saturation and brightness. In addition, a color histogram can be constructed to intuitively display the distribution of different colors in the image. Through these statistics and analyses, arbitrary color feature values ​​can be obtained, which can comprehensively describe the color features of the display screen slice, such as the overall hue tendency of the display screen, color uniformity and other information.

[0025] Step S233, collect and analyze the texture features of the arbitrary background slice to obtain an arbitrary texture feature value. Specifically, collect and analyze the texture features of the arbitrary background slice, and use a variety of texture feature extraction methods. For example, use the gray level co-occurrence matrix (GLCM) to describe the texture features. First, calculate the gray level co-occurrence matrix of the background slice, which reflects the frequency relationship of the occurrence of different gray value pixel pairs in the image in a specific direction and distance. From the gray level co-occurrence matrix, multiple texture feature parameters can be extracted, such as contrast, which indicates the clarity of the texture in the image. The higher the contrast, the more obvious the texture; energy, which reflects the uniformity and regularity of the texture. The higher the energy value, the more uniform the texture; entropy, which reflects the complexity of the image texture. The larger the entropy, the more complex and diverse the texture. In addition, methods such as wavelet transform can be used to further analyze the texture features of the background, decompose the image into sub-bands of different frequencies, and analyze the energy distribution of each sub-band and other information. Through these texture feature extraction methods, any texture feature value is obtained, and any texture feature value can characterize the texture characteristics of the background slice, such as the roughness of the background, the regularity of the texture, and other information.

[0026] Step S234, the arbitrary color feature value and the arbitrary texture feature value constitute the first arbitrary image feature information group. Specifically, the obtained arbitrary color feature value and arbitrary texture feature value are integrated to form the first arbitrary image feature information group. During the integration process, these feature values ​​are quantized so that they can be easily applied to subsequent machine learning model training or other analysis tasks. For example, the color feature value and the texture feature value can be combined into a feature vector, in which each element corresponds to a specific color or texture feature parameter, such as [blue hue pixel ratio, saturation mean, contrast, energy, etc.]. The first arbitrary image feature information group can comprehensively reflect the visual feature information of the display screen image, provide a rich data basis for tasks such as production stage recognition based on image features, and help improve the accuracy and reliability of related models or algorithms.

[0027] In a possible implementation, any display screen image in the display screen image set is labeled with a predetermined production stage to obtain any stage labeling information, and step S220 further includes step S221, which sequentially counts the substrate manufacturing labeling frequency of the substrate manufacturing stage, the coating labeling frequency of the coating stage, and the assembly labeling frequency of the assembly stage in the first training data. Specifically, the first training data is comprehensively traversed. During the traversal process, the production stage labeling information corresponding to each piece of data is checked and counted. When encountering training data labeled as the substrate manufacturing stage, the substrate manufacturing labeling frequency is added by one; similarly, for the labeling data of the coating stage and the assembly stage, the coating labeling frequency and the assembly labeling frequency are counted accordingly. For example, in a set containing thousands of training data, it may be found that 800 data are labeled as the substrate manufacturing stage, and the substrate manufacturing labeling frequency is recorded as 800. Through such detailed traversal and statistics, the number of labels of each production stage in the first training data can be accurately obtained, providing basic data basis for subsequent screening of stages that do not meet the predetermined frequency threshold.

[0028] Step S222, screening the corresponding stages that do not meet the predetermined frequency threshold in the substrate manufacturing annotation frequency, the coating annotation frequency and the assembly annotation frequency, and recording them as the target preprocessing stage. Specifically, the predetermined frequency threshold is set in advance according to actual needs and data distribution characteristics. This threshold can be determined by analyzing a large amount of historical data and estimating the current production situation. For example, after analysis, it is found that in a balanced training data set, the ideal annotation frequency ratio of the substrate manufacturing stage, the coating stage and the assembly stage should be approximately 3:3:4. If the total amount of training data is 10,000, then the predetermined frequency threshold of the substrate manufacturing stage may be set to about 2,500, the coating stage to about 2,500, and the assembly stage to about 3,300. Compare the statistically obtained substrate manufacturing annotation frequency, coating annotation frequency and assembly annotation frequency with their respective predetermined frequency thresholds. If the annotation frequency of a certain stage is lower than its threshold, then the stage is determined as the corresponding stage that does not meet the predetermined frequency threshold, recorded as the target preprocessing stage. For example, if the substrate manufacturing annotation frequency is only 1,800, which is lower than the set threshold, then the substrate manufacturing stage becomes the target preprocessing stage.

[0029] Step S223, the training data of the target preprocessing stage is enhanced and constitutes the first preprocessing training data. Specifically, for the training data determined as the target preprocessing stage, it is necessary to perform enhancement processing to improve the balance and representativeness of the data. A variety of data enhancement methods can be used, such as image rotation, flipping, scaling and other operations. Taking image rotation as an example, if it is the target preprocessing data of the substrate manufacturing stage, the display screen image is rotated at a certain angle (such as 90 degrees, 180 degrees, 270 degrees) to generate new image data, and its annotation information is updated accordingly to still be the substrate manufacturing stage. For image flipping, horizontal flipping or vertical flipping operations can be performed, and new training data is also generated. In terms of scaling, the image is scaled at a certain ratio (such as 0.8 times, 1.2 times, etc.) to obtain image data of different sizes for training. Through these data enhancement operations, the number and diversity of the training data of the target preprocessing stage can be increased. The training data of the target preprocessing stage after enhancement processing is recombined with other training data that originally meet the frequency threshold requirements to form the first preprocessing training data. The first preprocessed training data obtained in this way is more balanced in the labeling frequency of each production stage, which can provide a better data basis for subsequent machine learning model training, improve the accuracy and generalization ability of the model, and enable it to more accurately identify the corresponding production stage when faced with various display screen images in actual production.

[0030] Step S300, the target test result is obtained by testing and analyzing the second image feature information group of the optical image of the display screen through the target stage test channel of the target production stage matched in the intelligent integrated fusion test model. Specifically, the intelligent integrated fusion test model contains special test channels for different production stages. After determining the target production stage of the optical image of the display screen, the corresponding target stage test channel is located and matched in the model. If it is determined to be the coating stage, the corresponding channel is found. Then the second image feature information group of the optical image of the display screen is extracted. This group focuses on specific quality features of the target production stage. For example, the coating stage focuses on the uniformity of coating thickness, surface flatness and fit, etc. Through a variety of image analysis techniques, such as inferring thickness uniformity through grayscale changes, checking flatness through edge and shape analysis, and judging fit through image segmentation and similarity calculation, these feature information are integrated and quantified to form this group of data. It is then input into the target stage test channel, and the corresponding quality inspection algorithms and models are run in the channel. For example, the coating stage channel has thickness uniformity assessment, flatness inspection, fit judgment and other algorithms. The results of various feature inspections are combined to ultimately obtain a quantitative quality score or a qualitative evaluation of the target test result to reflect the quality status of the display accessories in the current production stage and provide a basis for quality control decisions.

[0031] In a possible implementation, the target stage test channel of the target production stage matched in the intelligent integrated fusion test model is used to test and analyze the second image feature information group of the display screen optical image to obtain the target test result. Step S300 further includes step S310, in which the substrate manufacturing image set of the substrate manufacturing stage, the coating image set of the coating stage and the assembly image set of the assembly stage are sequentially extracted from the display screen image set. Specifically, data mining is performed on the display screen image set. By accurately identifying the image annotation information, the display screen images belonging to the substrate manufacturing stage are extracted one by one, and the substrate manufacturing image set is aggregated. In this process, the production stage annotation of each image should be carefully checked to ensure its accuracy and avoid the mixing of erroneous data. For example, if the image annotation information clearly records that the image is collected from a specific process link of the substrate manufacturing production line, and the image features also meet the typical features of the substrate manufacturing stage, such as showing the raw material form of the substrate, the shape after preliminary processing, etc., then it is included in the substrate manufacturing image set. Similarly, similar operations are performed on the images of the coating stage and the assembly stage, and the coating image set and the assembly image set are extracted respectively, laying the foundation for the subsequent construction of targeted training data.

[0032] Step S320, analyze the substrate manufacturing image set to form a second training data, and conduct supervised learning on the second training data to obtain the substrate manufacturing test model of the substrate manufacturing stage. Specifically, a detailed analysis is carried out on the substrate manufacturing image set to form the second training data. From the perspective of image features, a variety of key feature information is extracted. For example, for the flatness feature of the substrate, it can be indirectly judged by analyzing the grayscale change of the pixels on the surface of the substrate in the image. If the grayscale change is relatively uniform, the flatness is good; for the material uniformity feature of the substrate, the color feature and texture feature of the image can be used for comprehensive analysis. If the color distribution is uniform and the texture is consistent, the material uniformity is high. The extracted feature information is combined with the corresponding image label (such as qualified or unqualified substrate image) to form the second training data. The second training data is trained using a supervised learning algorithm. Select a suitable algorithm, such as a support vector machine (SVM). The training data is divided into a training set and a validation set. During the training process, the SVM algorithm distinguishes different categories of data (qualified and unqualified substrate images) by constructing a hyperplane, and continuously adjusts the parameters of the hyperplane to achieve the best classification effect. In the verification phase, the trained model is evaluated using the verification set data, and the model parameters are further adjusted based on the evaluation results, such as the selection of kernel functions and parameter optimization. After multiple rounds of training and verification, the substrate manufacturing test model of the substrate manufacturing phase is finally obtained, which can accurately judge the quality status of the substrate based on the input substrate image features.

[0033] Step S330, analyze the coating image set to form a third training data, and perform supervised learning on the third training data to obtain the coating test model of the coating stage. Specifically, the coating image set is deeply analyzed to form the third training data. Focus on the relevant features of the coating, such as the coating thickness feature. The thickness can be estimated by analyzing the optical reflection characteristics of the coating area in the image. The reflection intensity and angle of light for coatings of different thicknesses will be different; the uniformity feature of the coating is judged by checking the consistency of the coating color and texture in the image. If there are areas with different shades of color or disordered texture, it may indicate uneven coating. Combine these feature information closely related to the coating quality with the image label (qualified or unqualified coating image) to construct the third training data. The third training data is trained using a supervised learning algorithm, such as a multi-layer perceptron (MLP) neural network. During the training process, the image feature data is input into the input layer of the MLP, and after the neurons of the hidden layer are calculated and converted, the predicted result (qualified or unqualified) is obtained at the output layer. The weights and biases of the hidden layer are continuously adjusted through the back propagation algorithm to minimize the error between the predicted result and the actual label. The validation set data is used to monitor and adjust the training process. After repeated training and optimization, a coating test model for the coating stage is obtained, which can effectively detect the coating quality.

[0034] Step S340, analyze the assembly image set to form the fourth training data, and perform supervised learning on the fourth training data to obtain the assembly test model of the assembly stage. Specifically, analyze the assembly image set to form the fourth training data. Focus on extracting features related to assembly accuracy, such as gap features between components, and determine the gap size by measuring the distance between the edges of the components through images; component alignment features, use the shape and position information of the image to determine whether the components are aligned, if the edges of the components are not parallel or offset, it means that the alignment is poor. Combine these assembly feature information with image labels (qualified or unqualified assembly images) into the fourth training data. Use a supervised learning algorithm to train the fourth training data, such as a convolutional neural network (CNN). The convolution layer of CNN can automatically extract local features in the image, such as the edges and corners of the components, the pooling layer reduces the data dimension, and the fully connected layer performs the final classification judgment. During the training process, the network parameters of CNN are continuously adjusted through a large amount of assembly image data so that it can accurately identify the assembly quality. After training and optimization, an assembly test model of the assembly stage is obtained, which can accurately evaluate the quality of the assembled display screen.

[0035] Step S350, the substrate manufacturing test model is used as the substrate manufacturing test channel, the coating test model is used as the coating test channel, and the assembly test model is used as the assembly test channel to build the intelligent integrated fusion test model. Specifically, the trained substrate manufacturing test model is used as the substrate manufacturing test channel, the coating test model is used as the coating test channel, and the assembly test model is used as the assembly test channel for organic integration. In the intelligent integrated fusion test model, when an optical image of a display screen is input, its target production stage is first determined by the stage recognition classifier, and then the image feature information is input into the corresponding target stage test channel. For example, if the image is determined to be an image of the coating stage, its feature information is input into the coating test channel (coating test model) for quality inspection. The various test channels work together to form a complete intelligent integrated fusion test model, which can comprehensively and efficiently detect and evaluate the quality of mobile phone display screen accessories at different production stages.

[0036] In a possible implementation, the substrate manufacturing image set is analyzed to form a second training data, and the second training data is supervised to learn to obtain the substrate manufacturing test model of the substrate manufacturing stage, and step S320 further includes step S321, obtaining a first display screen slice of the first image in the substrate manufacturing image set. Specifically, the first image is selected from the substrate manufacturing image set. After the image is obtained, in order to improve the accuracy and efficiency of subsequent analysis, it needs to be preprocessed. This includes image denoising, using a filtering algorithm such as Gaussian filtering to remove noise points generated during image acquisition to make the image clearer. Then, an image enhancement operation is performed, such as adjusting the contrast and brightness of the image to highlight the detail information in the image. After completing the preprocessing, the first display screen slice is extracted using image segmentation technology. A threshold-based segmentation method can be used to set a suitable threshold according to the difference in pixel grayscale values ​​between the display screen and the background, separate the display screen area from the entire image, and obtain a first display screen slice, which only contains pixel information related to the display screen, providing an accurate data basis for subsequent significance analysis.

[0037] Step S322, performing a saliency analysis on the first display screen slice to obtain a first salient area, and obtaining a first defect type of the first salient area. Specifically, when performing a saliency analysis on the first display screen slice, it is first divided into regions. The display screen slice is divided into a plurality of smaller regions, for example, by using a uniform grid division or an adaptive division method based on image content. Then, for each region, features of predetermined feature dimensions are collected, and these predetermined feature dimensions include brightness dimension, color dimension, and direction dimension. In the brightness dimension, features such as average brightness and brightness variance of each region are calculated; in the color dimension, the distribution of different colors in the region is statistically analyzed, such as color histogram information; in the direction dimension, the texture direction feature in the region is determined by calculating the gradient direction of the pixel gray value. The features of these different dimensions are fused to obtain a first slice feature fusion map. The first slice feature fusion map is subjected to saliency analysis using the central-surrounding difference principle. This principle is based on the characteristic that the human visual system is more sensitive to regions in the image that are more different from the surrounding environment. The degree of difference between each region and the surrounding region in the feature fusion map is calculated, and the greater the difference, the more significant the region. By setting a suitable saliency threshold, a salient region, i.e., a first salient region, is screened out. For these first significant areas, their defect types are further analyzed. For example, if there are spots with abnormal colors in the significant areas, and the texture characteristics of the spots are significantly different from those of the normal display area, it may be a defect caused by material impurities; if there are areas with sudden changes in brightness and irregular shapes, it may be scratches or damage on the surface of the display. Through comprehensive judgment of these features, the first defect type of the first significant area is determined, providing key label information for the subsequent construction of training data.

[0038] Step S323, collect multi-dimensional image features for the first significant area to obtain a first significant feature information group, and form the second training data with the first defect type. Specifically, collect multi-dimensional image features for the first significant area. In addition to the brightness, color and direction dimensional features already involved in the saliency analysis, other relevant features can be further extracted. For example, in terms of shape features, the geometric parameters such as the perimeter, area, and circularity of the significant area are calculated, and these parameters can reflect the shape characteristics of the defect; in terms of texture features, the contrast, energy, entropy and other features of the texture in the significant area are analyzed by using methods such as gray level co-occurrence matrix to deeply understand the texture structure of the defect area. The feature information extracted from different dimensions is integrated to obtain the first significant feature information group. The first significant feature information group is combined with the first defect type to form the second training data. For example, if the first significant feature information group contains feature information such as the brightness mean, color distribution variance, shape perimeter, texture contrast and the like of the significant area, and the first defect type is material impurities, then these information are combined into a training data record. By collecting and organizing a large amount of such training data, we can provide rich and representative data resources for the subsequent training test model in the substrate manufacturing stage, so that the model can accurately learn the relationship between different defect types and image features in the substrate manufacturing process, thereby improving the accuracy and reliability of the model in substrate quality detection.

[0039] In a possible implementation, a saliency analysis is performed on the first display screen slice to obtain a first salient area, and a first defect type of the first salient area is obtained. Step S322 further includes step S3221, performing area division on the first display screen slice to obtain a first area division result, wherein the first area division result includes multiple areas. Specifically, area division is performed on the first display screen slice. A variety of division methods can be used, such as a uniform grid division method, to divide the display screen slice into multiple rectangular areas of equal size according to fixed row and column spacing. For example, for a display screen slice with a resolution of 1920×1080, it can be divided into 10×10 rectangular areas of 192×108 pixels. This division method is simple and direct, and can quickly divide the slice into multiple sub-areas, which is convenient for subsequent feature analysis and processing. In addition, an adaptive division method based on image content can also be used to automatically determine the division boundary according to information such as changes in pixel grayscale values ​​and texture features in the display screen slice. For example, when there are obvious texture areas or brightness mutation areas in the display screen slice, the adaptive segmentation method can be divided along the boundaries of these feature changes, making the divided areas more semantic and the image content in each area more consistent, which is conducive to extracting more accurate regional features. Through segmentation, the first regional segmentation result is obtained, which contains multiple areas and provides a basic framework for subsequent feature collection and significance analysis.

[0040] Step S3222, collect features of the predetermined feature dimension for the first display screen slice to obtain a first slice feature fusion map. Specifically, collect features of the predetermined feature dimension for the first display screen slice, where the predetermined feature dimension includes brightness dimension, color dimension and direction dimension. In the brightness dimension, for each divided area, calculate its average brightness value, that is, the sum of the brightness values ​​of all pixels in the area divided by the number of pixels. At the same time, calculate the brightness variance to measure the dispersion of the brightness in the area. For example, if the brightness variance in a region is large, it means that there is uneven brightness in the region, which may indicate that there is a problem of abnormal local brightness on the display screen. In the color dimension, convert the image from the RGB color space to the HSV color space, and for each region, count the histogram information of the three channels of H (hue), S (saturation), and V (brightness) in the HSV color space. By analyzing the color histogram, the distribution of colors in the region can be understood, such as the proportion of a certain hue, the richness of the color, etc. In the direction dimension, the gradient information of the pixel gray value is used to determine the direction feature. Calculate the gradient vector of each pixel, whose direction indicates the direction of the fastest grayscale change. By performing statistical analysis on the gradient directions of all pixels in the region, for example, using the directional histogram method, the main directional information of the texture in the region is obtained. Fuse the feature information of the brightness, color, and directional dimensions to obtain the first slice feature fusion map. For example, the brightness average score, brightness variance score, color histogram feature vector, and directional histogram feature vector of each region can be combined into a new feature vector in a certain order. The feature vectors of all regions are combined together to form the first slice feature fusion map. The fusion map integrates image feature information of multiple dimensions and can more comprehensively reflect the visual characteristics of the display screen slice.

[0041] Step S3223, using the central-surrounding difference principle to perform a saliency analysis on the first slice feature fusion map, and combining the multiple regions to determine the first salient region. Specifically, using the central-surrounding difference principle to perform a saliency analysis on the first slice feature fusion map. For each region in the first region division result, calculate the degree of difference between it and the surrounding regions on the first slice feature fusion map. With a certain region as the center, select the adjacent regions around it as the comparison object. When calculating the difference, calculate each feature dimension in the fusion map separately. For example, in the brightness dimension, calculate the sum of the absolute values ​​of the average brightness difference between the central region and the surrounding region; in the color dimension, calculate a certain distance metric between the color histograms, such as the Bhattacharyya distance, to measure the difference in color distribution; in the direction dimension, calculate the difference metric of the direction histogram, such as the chi-square distance. Perform a weighted sum of the difference values ​​of different dimensions to obtain the central-surrounding difference score of the region. By setting a suitable significance threshold, the region with a difference score greater than the threshold is determined as a salient region, that is, the first salient region. For example, if an area is significantly different from the surrounding areas in terms of brightness, color and direction, and its comprehensive difference score exceeds the set threshold, then the area is judged as a first significant area that may have defects or special visual features. Combining the analysis results of multiple areas, all first significant areas are finally determined. These areas will serve as the focus of further analysis of defect types and collection of significant feature information groups, which will help to accurately locate potential problem areas in display slices and deeply analyze their characteristics.

[0042] In a possible implementation, the features of the first display screen slice in a predetermined feature dimension are collected to obtain a first slice feature fusion map, and step S3222 further includes step S32221, and the predetermined feature dimension includes a brightness dimension, a color dimension, and a direction dimension. Specifically, in the analysis of the predetermined feature dimension, for the brightness dimension, the pixel brightness value is calculated according to a specific formula of the RGB color model, and the average brightness value of the display screen slice is calculated to reflect the overall brightness. At the same time, the brightness variance is calculated to measure its distribution uniformity, so as to determine whether there are over-bright, over-dark or local brightness abnormal areas. In the color dimension, the display screen slice is converted to the HSV color space, and the three channel histograms of H, S, and V are constructed. The hue histogram counts the proportion of pixels of different hues, the saturation histogram reflects the color purity distribution, and the lightness histogram supplements the brightness information. By comparing the histograms, color presentation and rendering problems can be found. In the directional dimension, the Sobel operator is used to calculate the pixel gradient vector to determine the texture direction, and the pixel gradient directions in each area are counted to construct a directional histogram. This is used to judge the texture situation in the area and the difference with other areas. The comprehensive feature extraction results of the three dimensions can fully describe the display screen slice and provide a rich information basis for subsequent analysis and processing.

[0043] Step S400, when the target test result does not meet the target production constraint corresponding to the target production stage, a real-time warning signal is issued. Specifically, for each target production stage, the target production constraints are pre-defined according to industry standards, enterprise specifications and practical experience. For example, the substrate manufacturing stage has limitations on flatness and material density, the coating stage has requirements for coating thickness, uniformity and roughness, and the assembly stage has standards for assembly clearance and connection firmness. After obtaining the target test results, they are compared in detail with the corresponding constraints. If the indicators are out of range, such as the coating thickness or uniformity deviation does not meet the requirements during coating, it is determined that there is a quality deviation, and the real-time warning signal sending mechanism is triggered immediately. The signal is in various forms. The workshop has a unique audio alarm sound and a flashing red warning light. The management system monitoring interface has a pop-up window showing the details of the problem, and it is sent to the management personnel terminal via network communication to help them deal with it in time, prevent the generation of defective products, reduce losses and ensure quality.

[0044] Step S500, based on the real-time warning signal, the real-time quality control intervention of the mobile phone display screen accessories is carried out. Specifically, the signal monitoring system in the production environment monitors the real-time warning signal of the intelligent detection platform in real time, and immediately starts the linkage mechanism after receiving it, communicates with the automation equipment control system, and sends a notification to the management terminal and the staff mobile terminal, and the management terminal displays the details of the problem. Links with a high degree of automation, such as automatic adjustment of spraying parameters when the coating thickness is uneven in the coating stage, triggering equipment leveling and grinding when there is a problem with the flatness in the substrate manufacturing stage, and automatic adjustment of robot movements when the assembly accuracy does not meet the requirements in the assembly stage. Where it cannot be solved automatically or needs to be confirmed, the staff will manually check it according to the warning, such as checking and repairing minor defects with tools, and checking the production environment and equipment and cleaning and maintenance when foreign objects are attached. The intervention operations, problems and results are recorded in detail during the process and stored in the database for quality traceability and analysis, and the production process is improved according to the data rules to improve the production quality and stability of mobile phone display screen accessories.

[0045] The embodiment of the present application uses an intelligent optical inspection platform to dynamically inspect mobile phone display screen accessories to obtain optical images; activates a stage recognition classifier to analyze image features and identifies the target production stage; uses an intelligent test model to test the image features of the target stage to obtain test results; when the test results do not meet the production standards, an early warning signal is issued, and based on this signal, real-time quality control of the production process is performed, thereby achieving the technical effect of improving the quality control and efficiency of the production process by realizing accurate detection and real-time early warning for different production stages in the production process of mobile phone accessories.

[0046] Figure 2 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, showing a block diagram of an exemplary electronic device suitable for implementing an implementation of the present invention. Figure 2The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention. The electronic device is in the form of a general computing device, and its components may include but are not limited to an input device 401, a processor 402, a memory 403, and an output device 404. Among them, the processor 402 may be one or more; the memory 403 may include a computer-readable medium and at least one program product, and the program product has a set (at least one) of program modules, which are configured to perform the functions of each embodiment of the present application.

[0047] The memory 403 shown in the embodiment of the present invention may adopt any combination of one or more computer-readable media; the computer-readable storage medium may be, but is not limited to, an infrared, semiconductor system, device or component, or any combination of the above, for storing software programs, computer executable programs and modules, such as the program instructions / modules corresponding to the intelligent testing method for mobile phone accessories production in the embodiment of the present invention. The processor 402 executes various functional applications and data processing of the computer device by running the software programs, instructions and modules stored in the memory 403, thereby realizing the above-mentioned intelligent testing method for mobile phone accessories production.

[0048] The intelligent testing device for mobile phone accessories production provided by the embodiment of the present invention can execute the intelligent testing method for mobile phone accessories production provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0049] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules may be used and run on the user terminal and / or server, and the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0050] The above specific implementation manner does not constitute a limitation to the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application. In some cases, the actions or steps recorded in the present application can be performed in an order different from that in the embodiment and can still achieve the desired results. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. An intelligent testing method for mobile phone accessories production, characterized in that: include: Dynamically inspect the production process of mobile phone display screen accessories through a predetermined intelligent optical inspection platform to obtain an optical image of the display screen; Activate the stage recognition classifier to perform recognition, judgment and analysis on the first image feature information group of the display screen optical image to obtain the target production stage corresponding to the display screen optical image; By testing and analyzing the second image feature information group of the optical image of the display screen in the target stage test channel of the target production stage matched in the intelligent integrated fusion test model, a target test result is obtained; When the target test result does not meet the target production constraint corresponding to the target production stage, a real-time warning signal is issued; Performing real-time quality control intervention on the mobile phone display screen accessories based on the real-time warning signal; Build a display screen image set; Marking any display screen image in the display screen image set at a predetermined production stage to obtain marking information of any stage; Building first training data based on a first arbitrary image feature information group of the arbitrary display screen image and the arbitrary stage annotation information; Preprocessing the first training data to obtain first preprocessed training data, and performing supervised machine learning on the first preprocessed training data to obtain the stage identification classifier; The first training data is formed based on the first arbitrary image feature information group of the arbitrary display screen image and the arbitrary stage annotation information, including: Performing semantic segmentation and slicing on the arbitrary display screen image to obtain an arbitrary slicing result, wherein the arbitrary slicing result includes an arbitrary display screen slice and an arbitrary background slice; Collecting and analyzing color features of the arbitrary display screen slice to obtain arbitrary color feature values; Collecting and analyzing texture features of the arbitrary background slices to obtain arbitrary texture feature values; The arbitrary color feature value and the arbitrary texture feature value constitute the first arbitrary image feature information group; The predetermined production stages include a substrate manufacturing stage, a coating stage, and an assembly stage. Preprocessing the first training data to obtain first preprocessed training data includes: Sequentially counting the substrate manufacturing annotation frequency of the substrate manufacturing stage, the coating annotation frequency of the coating stage, and the assembly annotation frequency of the assembly stage in the first training data; Screening the corresponding stages of the substrate manufacturing marking frequency, the coating marking frequency and the assembly marking frequency that do not meet the predetermined frequency threshold, and recording them as target preprocessing stages; The training data in the target preprocessing stage is enhanced and constitutes the first preprocessed training data.

2. The intelligent testing method for mobile phone accessories production according to claim 1, characterized in that: The target test result is obtained by testing and analyzing the second image feature information group of the optical image of the display screen through the target stage test channel of the target production stage matched in the intelligent integrated fusion test model, including: sequentially extracting a substrate manufacturing image set of the substrate manufacturing stage, a coating image set of the coating stage, and an assembly image set of the assembly stage from the display screen image set; Analyzing the substrate manufacturing image set to form second training data, and performing supervised learning on the second training data to obtain a substrate manufacturing test model of the substrate manufacturing stage; Analyze the coating image set to form third training data, and perform supervised learning on the third training data to obtain a coating test model for the coating stage; Analyze the assembly image set to form fourth training data, and perform supervised learning on the fourth training data to obtain an assembly test model of the assembly stage; The substrate manufacturing test model is used as a substrate manufacturing test channel, the coating test model is used as a coating test channel, and the assembly test model is used as an assembly test channel to construct the intelligent integrated fusion test model.

3. The intelligent testing method for mobile phone accessories production according to claim 2, characterized in that: include: Acquire a first display screen slice of a first image in the substrate manufacturing image set; Performing a saliency analysis on the first display screen slice to obtain a first salient area, and acquiring a first defect type of the first salient area; Multi-dimensional image features are collected for the first salient area to obtain a first salient feature information group, which is combined with the first defect type to form the second training data.

4. The intelligent testing method for mobile phone accessories production according to claim 3, characterized in that: Performing saliency analysis on the first display screen slice to obtain a first salient area includes: Performing region division on the first display screen slice to obtain a first region division result, wherein the first region division result includes a plurality of regions; Collecting features of a predetermined feature dimension on the first display screen slice to obtain a first slice feature fusion map; The first slice feature fusion map is subjected to saliency analysis using the center-surrounding difference principle, and the first salient region is determined in combination with the multiple regions.

5. The intelligent testing method for mobile phone accessories production according to claim 4, characterized in that: The predetermined characteristic dimensions include brightness dimension, color dimension and direction dimension.

6. An electronic device, characterized in that: The electronic device comprises: a processor; a memory for storing instructions executable by the processor; wherein the processor is used to execute the intelligent testing method for mobile phone accessory production as described in any one of claims 1 to 5.

Images