Mobile phone appearance detection system and method based on visual identification

Abstract

The invention discloses a mobile phone appearance detection system and method based on visual identification, and belongs to the technical field of visual identification. The method comprises the steps of performing image acquisition and preprocessing on a mobile phone to obtain a mobile phone appearance image; extracting visual features based on the mobile phone appearance image, wherein the visual features comprise edge features, texture features and screen abnormal features; comparing and analyzing the extracted visual features with a pre-established defect-free mobile phone appearance feature database, and training a model by using a machine learning algorithm to enable the model to learn differences between normal mobile phone appearance features and various defect features; obtaining fault information stored in the mobile phone; historical fault data are collected, and an association database of appearance damage and internal faults is established; carrying out data integration and preliminary matching, and carrying out multi-factor logical reasoning and verification by considering the relationship between different damaged parts and various fault information; and carrying out probability evaluation to generate a comprehensive analysis conclusion.

Description

Technical Field

[0001] The present invention relates to the technical field of visual recognition, and in particular to a mobile phone appearance detection system and method based on visual recognition. Background Art

[0002] With the rapid development of the smartphone market, the production and sales of mobile phones have reached a huge scale. In the production process of mobile phones, appearance quality inspection is a crucial link, which directly affects the quality image and market competitiveness of the product. At the same time, in the field of mobile phone after-sales repair, quickly and accurately diagnosing the relationship between mobile phone appearance damage and internal faults is also extremely critical to improving repair efficiency, reducing repair costs and improving user satisfaction.

[0003] The speed of manual visual inspection is limited by factors such as personnel proficiency and concentration, and it is difficult to meet the high efficiency requirements of large-scale production or large-scale maintenance tasks. Human visual judgment is easily affected by subjective factors. For some minor appearance defects, such as slight scratches, tiny stains or imperceptible shell deformation, manual inspection may miss or misjudge. The data records in the manual inspection process are often not standardized and comprehensive, and it is difficult to form a systematic database for reference in quality analysis and fault diagnosis. Summary of the invention

[0004] The purpose of the present invention is to provide a mobile phone appearance detection system and method based on visual recognition to solve the problems raised in the prior art.

[0005] To achieve the above object, the present invention provides the following technical solutions: A method for detecting the appearance of a mobile phone based on visual recognition, the method comprising: Capture and preprocess the image of the mobile phone to obtain the appearance image of the mobile phone; Extracting visual features based on the mobile phone appearance image, wherein the visual features include edge features, texture features, and screen abnormality features; The method for extracting visual features includes: using an edge detection algorithm to extract edge features in a mobile phone appearance image, and identifying edge lines of various parts of the mobile phone; using a texture analysis algorithm to extract texture features in a mobile phone appearance image, and detecting whether there are scratches, wear or stains on the mobile phone appearance; for a mobile phone screen, extracting pixel features of its display area, checking whether there is a display abnormality, judging whether there is a local abnormal pixel point aggregation, and obtaining screen abnormality features; The extracted visual features are compared and analyzed with a pre-established database of appearance features of defect-free mobile phones, and a model is trained using a machine learning algorithm to enable the model to learn the differences between the appearance features of normal mobile phones and the features of various defects; By connecting the mobile phone to the testing equipment, the fault information stored inside the mobile phone is obtained; historical fault data, including visual features and corresponding fault information, is collected to establish a database of association between appearance damage and internal faults; data integration and preliminary matching are performed, the relationship between different damaged parts and various fault information is considered, and multi-factor logical reasoning and verification are performed; Conduct a probabilistic assessment and generate a comprehensive analysis conclusion on the relationship between the appearance damage and internal failures of the mobile phone.

[0006] A Gaussian filter is used to smooth the mobile phone appearance image. The Gaussian filter performs weighted average calculation on each pixel point and its neighboring pixels in the image, and smoothes the mobile phone appearance image and retains the edge information of the mobile phone appearance image according to the set Gaussian kernel size and standard deviation; Calculate the gradient amplitude and direction of the image, and use the partial derivatives of the smoothed image in the horizontal and vertical directions to determine the gradient amplitude and direction of each pixel in the image. The gradient amplitude indicates the severity of the image change at the pixel, and the direction indicates the direction of the edge, highlighting the junction between the border and the panel in the mobile phone appearance image. Apply non-maximum suppression technology, along a certain edge direction, if the gradient amplitude of a pixel is not the maximum value in that direction, then it will be marked as a non-edge point.

[0007] Through dual threshold detection and edge connection, a high threshold and a low threshold are set. Pixels above the high threshold are edge points, and pixels below the low threshold are not edge points. Pixels between the high threshold and the low threshold are judged as edge points based on their connectivity with the determined edge points. If a pixel is connected to a determined edge point and its gradient magnitude is greater than the low threshold, then the pixel is also considered an edge point; Connect the discontinuous edges to form edge lines of various parts of the phone.

[0008] Using gray-level co-occurrence matrix to perform texture analysis; determining parameters for calculating gray-level co-occurrence matrix, including displacement direction and displacement distance; wherein, for detecting the texture of the surface of the mobile phone shell, the displacement direction selects horizontal and vertical directions, and the displacement distance is set according to the fineness of the texture of the mobile phone shell; Calculate the grayscale co-occurrence matrix; for each pixel in the image, count the frequency of grayscale combinations with the pixel in the specified displacement direction and distance to form a grayscale co-occurrence matrix; in a mobile phone appearance image, if the back cover has a texture pattern, then in the grayscale co-occurrence matrix, the element value corresponding to a specific grayscale value combination will reflect the frequency of occurrence of the grayscale combination in the image; Texture feature parameters are extracted from the gray level co-occurrence matrix, and the texture feature parameters include contrast, correlation, energy and entropy; wherein the contrast reflects the clarity of the texture in the image and the depth of the texture grooves, and is used to detect scratches or worn areas; the correlation measures the linear dependence of the gray values ​​in the image; the energy represents the uniformity of the image gray distribution and the coarseness of the texture; the entropy reflects the randomness of the image texture, and scratches, wear or stains will change the randomness of the image texture, thereby causing a change in the entropy value.

[0009] For a mobile phone screen, determine the range of the screen display area; based on the prior knowledge of the mobile phone model and screen size, determine the mobile phone screen display area in the mobile phone appearance image; Count the brightness and color value distribution of pixels in the display area; divide the screen display area into several rectangular grids, calculate the statistics of the pixels in each rectangular grid, including the average brightness, color mean and variance, and determine whether the statistics are normal, so as to check whether there is any display abnormality; Detect abnormal pixel clustering; set a threshold to determine whether a single pixel is an abnormal pixel. If the difference between the brightness value of a pixel and the average brightness value of the surrounding pixels exceeds the set brightness threshold, or the color value deviates from the normal color range, the pixel is marked as an abnormal pixel. Whether the color value deviates from the normal color range is determined based on the distance metric in the color space. Perform cluster analysis on the marked abnormal pixels. If it is found that several abnormal pixels are clustered together to form an area, it is determined that the screen display is abnormal in the local area. Screen abnormal features are extracted based on the brightness and color value distribution of pixels in the display area and the aggregation of abnormal pixels.

[0010] Collect samples of brand new mobile phones with no appearance defects of different brands and models; for each mobile phone, obtain a set of standard appearance images under different lighting conditions and different placement angles according to the same image acquisition and preprocessing process as the actual inspection; Extracting visual features from the standard appearance images in the standard appearance image set, sorting, classifying and storing the extracted visual feature data, and building a defect-free mobile phone appearance feature database; According to the brand and model information of the mobile phone, the corresponding non-defective mobile phone feature data subset in the non-defective mobile phone appearance feature database is located; for edge features, the differences in shape, length, curvature and continuity of edge lines in the same parts of the mobile phone to be tested and the non-defective mobile phone are compared; for texture features, the differences in texture feature parameters of the shell of the mobile phone to be tested and the non-defective mobile phone are compared to determine whether there are appearance defects. If there are appearance defects, the type and severity of the appearance defects are determined; for screen pixel features, the pixel brightness distribution, color mean and variance of the screen display area of ​​the mobile phone to be tested are compared with the deviation of the non-defective mobile phone. If there are abnormal brightness and color areas caused by local abnormal pixel aggregation, the location and possible causes of the screen display abnormality are determined in combination with the normal screen pixel features in the database.

[0011] Select the SVM algorithm as the machine learning algorithm, use the appearance feature data of defect-free mobile phones as positive samples, and use the collected feature data of mobile phones with various appearance defects as negative samples for model training; The sorted positive and negative sample feature vectors are randomly divided into training sets and validation sets in proportion, and input into the SVM model for training in batches. In each batch of training, the model attempts to construct a classification hyperplane based on the sample labels (positive samples are one category, negative samples are another category) and the current kernel function and parameter settings, calculates the distance from the sample to the hyperplane, and adjusts the position of the hyperplane to separate the positive and negative samples as accurately as possible to minimize the classification error. For example, when encountering a feature vector with a suspected scratch on the appearance, the model continuously corrects the hyperplane during the training process to allow it to more accurately judge whether the vector corresponds to a defect or a normal appearance, and gradually learns the association between different feature patterns and classification results.

[0012] After a certain number of batches are completed in each round of training, the cross-validation mechanism is enabled. The reserved validation set is input into the model of the current training stage to evaluate the model performance indicators, such as accuracy, recall, F1 value, etc. If it is found that the accuracy of the model on the validation set is not as expected or overfitting occurs, the kernel function parameters are adjusted accordingly. For example, increasing the gamma value allows the model to pay more attention to local sample details and improve the ability to capture complex defect features; or appropriately increasing the C value to enhance the model's tolerance for training errors, so that the decision boundary can be more flexible to adapt to a variety of samples, and then re-enter the training, and repeat this cycle to continuously optimize the model until satisfactory performance is achieved on the validation set.

[0013] After the model has undergone multiple rounds of training and parameter adjustment, and its performance on the validation set has stabilized and reached the preset standard, an independent test set (newly collected mobile phone appearance samples that have not participated in training and validation) is used to conduct a final performance evaluation of the model. The accuracy, recall rate, F1 value of the model on the test set, as well as the recognition sensitivity and specificity of various defect features are comprehensively considered to ensure that the model not only has a high overall accuracy rate, but can also accurately identify various rare or complex appearance defects, truly reflecting its effectiveness in actual mobile phone appearance inspection scenarios.

[0014] The visual feature data is represented by feature vectors. By adjusting the kernel function and related parameters, the SVM model can find an optimal classification hyperplane to separate the appearance features of non-defective mobile phones from the appearance features of defective mobile phones, thereby learning the difference patterns between the appearance features of normal mobile phones and the features of various defects. During the model training process, the cross-validation method is used to evaluate and optimize the model performance.

[0015] The testing equipment is equipped with several interface adapter modules, which are compatible with the connection interfaces of mobile phones of different brands and models. When the mobile phone is placed on the testing platform, the testing equipment automatically identifies the mobile phone interface type and obtains the fault information stored inside the mobile phone. Collect historical fault data from mobile phone repair centers, after-sales departments, and production testing. Each piece of fault data includes fault information and visual features. Establish a relational database structure and use MySQL to organize and store data. Each record in the database represents a mobile phone fault case, including fields such as mobile phone model, visual features, fault information, and fault occurrence time. According to the mobile phone model, a historical fault data subset of the same model of mobile phone is screened out in the associated database; the visual features of the current mobile phone are preliminarily matched with the visual features in the historical fault data subset to form a candidate set; Construct a hardware structure mapping diagram of the mobile phone's hardware structure and functions. For each case in the candidate set, use a model trained by a machine learning algorithm to determine the damaged parts of the mobile phone's appearance. Determine the affected internal hardware components and their associated functional modules on the hardware structure mapping diagram to obtain the initial fault point; obtain the fault propagation path by analyzing cases in historical fault data. For cases in the candidate set, combine the initial fault point and analyze whether the fault information conforms to the propagation logic along the fault propagation path; use data mining technology to mine potential association rules between different damaged parts, visual features and fault information from an associated database. For each case in the candidate set, use the association rules to verify and check whether the visual features of the current mobile phone and the fault information meet the association rules.

[0016] According to the historical failure data subset of the same model of mobile phone in the association database, the prior probability is calculated for different combinations of damage parts and fault information, and the prior probability matrix is ​​constructed; after checking whether the visual features of the current mobile phone and the fault information meet the association rules, the Bayesian theorem reasoning method is used to update the probability estimate of the relationship between the appearance damage and the internal fault of the current mobile phone according to the prior probability matrix; Based on the probability estimation results, determine the most likely cause of the failure; generate a comprehensive analysis conclusion on the relationship between the appearance damage and internal failure of the mobile phone, explain the association logic between the appearance damage and internal failure, explain the application of the mined association rules in the current case, and how the visual features and failure information support or deviate from the association rules; A mobile phone appearance detection system based on visual recognition, comprising: Image acquisition and preprocessing module: including: image acquisition unit and image preprocessing unit; wherein, the image acquisition unit is equipped with an industrial camera, which can automatically adjust the focal length, aperture and exposure time according to the preset program, and includes a lighting system to provide a lighting environment for image acquisition; the image preprocessing unit preprocesses the acquired image, including grayscale, noise reduction and image enhancement; Visual feature extraction module: including: edge feature extraction unit, texture feature extraction unit and screen pixel feature extraction unit; wherein the edge feature extraction unit uses edge detection algorithm to identify the edge lines of various parts of the mobile phone; the texture feature extraction unit uses texture analysis algorithm to detect whether there are scratches, wear or stains on the appearance of the mobile phone; the screen pixel feature extraction unit extracts the pixel features of the display area of ​​the mobile phone screen, checks whether there is any display abnormality, and obtains the screen abnormality features; Model training and comparative analysis module: including: defect-free feature database construction unit, comparative analysis unit and model training unit; the defect-free feature database construction unit collects samples of new mobile phones of different brands and models without appearance defects, performs image acquisition, preprocessing and visual feature extraction, and constructs a defect-free mobile phone appearance feature database; the comparative analysis unit compares and analyzes the extracted visual features with the pre-established defect-free mobile phone appearance feature database; the model training unit uses machine learning algorithms to train the model so that the model learns the differences between normal mobile phone appearance features and various defect features; Internal fault information processing module: including: fault information acquisition unit, associated database construction unit and data integration and matching unit; wherein the fault information acquisition unit acquires the fault information stored inside the mobile phone; the associated database construction unit collects historical fault data and establishes an associated database of appearance damage and internal faults; the data integration and matching unit performs data integration and preliminary matching, considers the relationship between different damaged parts and various fault information, and performs multi-factor logical reasoning and verification; Comprehensive analysis and conclusion generation module: including: probability evaluation and conclusion generation unit; wherein the probability evaluation and conclusion generation unit performs probability evaluation and generates a comprehensive analysis conclusion on the relationship between the appearance damage and internal faults of the mobile phone.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention is based on visual recognition technology, uses industrial cameras to automatically capture images of the appearance of mobile phones, and performs automatic detection through a series of image processing and analysis algorithms. The entire process does not require excessive human intervention and can quickly and accurately complete the task of detecting the appearance of mobile phones.

[0018] 2. The present invention utilizes edge detection algorithms, texture analysis algorithms, and screen pixel feature extraction algorithms to accurately identify various defects on the appearance of a mobile phone, including extremely fine scratches, wear, stains, and screen display anomalies.

[0019] 3. During the detection process, the present invention can automatically collect and organize a large amount of mobile phone appearance visual feature data and internal fault information data, and establish a systematic associated database. These data can be used for in-depth data analysis and mining, and enterprises can use these data to conduct comprehensive statistical analysis on mobile phone product quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the steps of a method for detecting the appearance of a mobile phone based on visual recognition according to the present invention; Figure 2 It is a system structure diagram of a mobile phone appearance detection system based on visual recognition of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Example: Figure 1-Figure 2 As shown, the present invention provides a technical solution. According to one embodiment of the present invention, Figure 1 A method for detecting the appearance of a mobile phone based on visual recognition is shown in a schematic diagram of steps. The method for detecting the appearance of a mobile phone based on visual recognition comprises: Capture and preprocess the image of the mobile phone to obtain the appearance image of the mobile phone; Extracting visual features based on the mobile phone appearance image, wherein the visual features include edge features, texture features, and screen abnormality features; The method for extracting visual features includes: using an edge detection algorithm to extract edge features in a mobile phone appearance image, and identifying edge lines of various parts of the mobile phone; using a texture analysis algorithm to extract texture features in a mobile phone appearance image, and detecting whether there are scratches, wear or stains on the mobile phone appearance; for a mobile phone screen, extracting pixel features of its display area, checking whether there is a display abnormality, judging whether there is a local abnormal pixel point aggregation, and obtaining screen abnormality features; The extracted visual features are compared and analyzed with a pre-established database of appearance features of defect-free mobile phones, and a model is trained using a machine learning algorithm to enable the model to learn the differences between the appearance features of normal mobile phones and the features of various defects; By connecting the mobile phone to the testing equipment, the fault information stored inside the mobile phone is obtained; historical fault data, including visual features and corresponding fault information, is collected to establish a database of association between appearance damage and internal faults; data integration and preliminary matching are performed, the relationship between different damaged parts and various fault information is considered, and multi-factor logical reasoning and verification are performed; Conduct a probabilistic assessment and generate a comprehensive analysis conclusion on the relationship between the appearance damage and internal failures of the mobile phone.

[0023] A Gaussian filter is used to smooth the mobile phone appearance image. The Gaussian filter performs weighted average calculation on each pixel point and its neighboring pixels in the image, and smoothes the mobile phone appearance image and retains the edge information of the mobile phone appearance image according to the set Gaussian kernel size and standard deviation; Calculate the gradient amplitude and direction of the image, and use the partial derivatives of the smoothed image in the horizontal and vertical directions to determine the gradient amplitude and direction of each pixel in the image. The gradient amplitude indicates the severity of the image change at the pixel, and the direction indicates the direction of the edge, highlighting the junction between the border and the panel in the mobile phone appearance image. Apply non-maximum suppression technology, along a certain edge direction, if the gradient amplitude of a pixel is not the maximum value in that direction, then it will be marked as a non-edge point.

[0024] Through dual threshold detection and edge connection, a high threshold and a low threshold are set. Pixels above the high threshold are edge points, and pixels below the low threshold are not edge points. Pixels between the high threshold and the low threshold are judged as edge points based on their connectivity with the determined edge points. If a pixel is connected to a determined edge point and its gradient magnitude is greater than the low threshold, then the pixel is also considered an edge point; Connect the discontinuous edges to form edge lines of various parts of the phone.

[0025] Using gray-level co-occurrence matrix to perform texture analysis; determining parameters for calculating gray-level co-occurrence matrix, including displacement direction and displacement distance; wherein, for detecting the texture of the surface of the mobile phone shell, the displacement direction selects horizontal and vertical directions, and the displacement distance is set according to the fineness of the texture of the mobile phone shell; Calculate the grayscale co-occurrence matrix; for each pixel in the image, count the frequency of grayscale combinations with the pixel in the specified displacement direction and distance to form a grayscale co-occurrence matrix; in a mobile phone appearance image, if the back cover has a texture pattern, then in the grayscale co-occurrence matrix, the element value corresponding to a specific grayscale value combination will reflect the frequency of occurrence of the grayscale combination in the image; Texture feature parameters are extracted from the gray level co-occurrence matrix, and the texture feature parameters include contrast, correlation, energy and entropy; wherein the contrast reflects the clarity of the texture in the image and the depth of the texture grooves, and is used to detect scratches or worn areas; the correlation measures the linear dependence of the gray values ​​in the image; the energy represents the uniformity of the image gray distribution and the coarseness of the texture; the entropy reflects the randomness of the image texture, and scratches, wear or stains will change the randomness of the image texture, thereby causing a change in the entropy value.

[0026] For a mobile phone screen, determine the range of the screen display area; based on the prior knowledge of the mobile phone model and screen size, determine the mobile phone screen display area in the mobile phone appearance image; Count the brightness and color value distribution of pixels in the display area; divide the screen display area into several rectangular grids, calculate the statistics of the pixels in each rectangular grid, including the average brightness, color mean and variance, and determine whether the statistics are normal, so as to check whether there is any display abnormality; Detect abnormal pixel clustering; set a threshold to determine whether a single pixel is an abnormal pixel. If the difference between the brightness value of a pixel and the average brightness value of the surrounding pixels exceeds the set brightness threshold, or the color value deviates from the normal color range, the pixel is marked as an abnormal pixel. Whether the color value deviates from the normal color range is determined based on the distance metric in the color space. Perform cluster analysis on the marked abnormal pixels. If it is found that several abnormal pixels are clustered together to form an area, it is determined that the screen display is abnormal in the local area. Screen abnormal features are extracted based on the brightness and color value distribution of pixels in the display area and the aggregation of abnormal pixels.

[0027] Collect samples of brand new mobile phones with no appearance defects of different brands and models; for each mobile phone, obtain a set of standard appearance images under different lighting conditions and different placement angles according to the same image acquisition and preprocessing process as the actual inspection; Extracting visual features from the standard appearance images in the standard appearance image set, sorting, classifying and storing the extracted visual feature data, and building a defect-free mobile phone appearance feature database; According to the brand and model information of the mobile phone, the corresponding non-defective mobile phone feature data subset in the non-defective mobile phone appearance feature database is located; for edge features, the differences in shape, length, curvature and continuity of edge lines in the same parts of the mobile phone to be tested and the non-defective mobile phone are compared; for texture features, the differences in texture feature parameters of the shell of the mobile phone to be tested and the non-defective mobile phone are compared to determine whether there are appearance defects. If there are appearance defects, the type and severity of the appearance defects are determined; for screen pixel features, the pixel brightness distribution, color mean and variance of the screen display area of ​​the mobile phone to be tested are compared with the deviation of the non-defective mobile phone. If there are abnormal brightness and color areas caused by local abnormal pixel aggregation, the location and possible causes of the screen display abnormality are determined in combination with the normal screen pixel features in the database.

[0028] In this embodiment, three common brands of mobile phones (Brand A, Brand B, Brand C) are selected, and two different models of mobile phones are selected from each brand, for a total of 6 new mobile phones without appearance defects as samples, namely, models M1 and M2 of brand A, models N1 and N2 of brand B, and models P1 and P2 of brand C.

[0029] The Canny edge detection algorithm is used to extract edge features in the image. Taking the edge of the front screen frame of the model M1 mobile phone of brand A as an example, after processing, the edge line lengths of the four sides of the screen frame are measured to be approximately (unit: pixel): upper frame 500, lower frame 500, left frame 300, right frame 300; the average curvature of the four edge lines is between 0.01 and 0.02 (the smaller the curvature value, the closer it is to a straight line), and the edge continuity is good, without interruption or abnormal curvature. The edge features of each part of each mobile phone in different images are extracted and the corresponding parameters are recorded as the basis for subsequent comparison.

[0030] Gray-level co-occurrence matrix (GLCM) was used for texture analysis, with horizontal and vertical displacement directions, and the displacement distance was set to 1 pixel. The characteristic parameters of the mobile phone shell texture, such as contrast, correlation, energy, and entropy, were calculated. For example, the texture of the back cover of the model N1 mobile phone of brand B was measured to have a contrast of about 0.2, a correlation of about 0.8, an energy of about 0.6, and an entropy of about 0.7. The fluctuation range of these parameters in the same part of each image was extremely small, indicating that the texture was uniform and normal. Similarly, the texture features of each part of all mobile phone samples were extracted and the relevant data were sorted.

[0031] First, determine the screen display area, divide the screen into 10×10 small grid areas, and count the average brightness, color mean (expressed by the mean of the three RGB channels), and variance of the pixels in each small area. Taking the P1 mobile phone screen of brand C as an example, the average brightness of the pixels in a small grid in the center area of ​​the screen is about 120 (value range 0-255), the color mean is (128, 128, 128) (RGB channel), the brightness variance is about 5, and the color variance is about 3 in each channel. These parameters are evenly distributed in all areas of the entire screen, and there is no abnormal pixel point aggregation phenomenon. Extract the pixel features of each mobile phone screen in different images and record detailed data.

[0032] The various visual feature data extracted above are classified and sorted according to mobile phone brands, models and corresponding parts (such as screen, frame, back cover, etc.), stored in the MySQL database, and a defect-free mobile phone appearance feature database is constructed.

[0033] A mobile phone to be tested is selected, whose brand is brand A and model is unknown (simulating the situation of unknown model mobile phones in actual testing). According to the same image acquisition and preprocessing process as mentioned above, its appearance image is acquired and processed under different lighting conditions and placement angles.

[0034] After edge feature extraction, it was found that the length of the edge line of the upper frame of the screen frame of the mobile phone was about 480 pixels (shorter than the defect-free brand A model M1 mobile phone), the average curvature of the upper frame edge line increased to 0.05 (there was a certain degree of curvature), and the edge was slightly discontinuous in the upper right corner. From this, it was preliminarily judged that the screen frame may have appearance defects such as deformation or damage. Comparing the shell texture feature parameters, it was found that the contrast of the texture of the back cover of the mobile phone increased to 0.4 (originally about 0.2), the correlation decreased to 0.6 (originally about 0.8), the energy decreased to 0.4 (originally about 0.6), and the entropy increased to 0.9 (originally about 0.7). Combined with the data in the defect-free mobile phone appearance feature database, it can be judged that the texture of the back cover is abnormal, and it is speculated that there may be appearance defects such as scratches and wear. By observing the pixel features of the screen display area, we found that in a small 3×3 grid area in the upper left corner of the screen, the average pixel brightness abnormally increased to about 200 (far beyond the normal range), the color mean became (255, 255, 255) (showing white bright spot features), and the brightness variance and color variance also increased significantly. Compared with the pixel features of normal mobile phone screens of the same brand in the database, it was determined that there was screen display abnormality in this area, which might be a problem of dead pixels or bright spots on the screen.

[0035] Select the SVM algorithm as the machine learning algorithm, use the appearance feature data of defect-free mobile phones as positive samples, and use the collected feature data of mobile phones with various appearance defects as negative samples for model training; The visual feature data is represented by feature vectors. By adjusting the kernel function and related parameters, the SVM model can find an optimal classification hyperplane to separate the appearance features of non-defective mobile phones from the appearance features of defective mobile phones, thereby learning the difference patterns between the appearance features of normal mobile phones and the features of various defects. During the model training process, the cross-validation method is used to evaluate and optimize the model performance.

[0036] The testing equipment is equipped with several interface adapter modules, which are compatible with the connection interfaces of mobile phones of different brands and models. When the mobile phone is placed on the testing platform, the testing equipment automatically identifies the mobile phone interface type and obtains the fault information stored inside the mobile phone. Collect historical fault data from mobile phone repair centers, after-sales departments, and production testing. Each piece of fault data includes fault information and visual features. Establish a relational database structure and use MySQL to organize and store data. Each record in the database represents a mobile phone fault case, including fields such as mobile phone model, visual features, fault information, and fault occurrence time. According to the mobile phone model, a historical fault data subset of the same model of mobile phone is screened out in the associated database; the visual features of the current mobile phone are preliminarily matched with the visual features in the historical fault data subset to form a candidate set; Construct a hardware structure mapping diagram of the mobile phone's hardware structure and functions. For each case in the candidate set, use a model trained by a machine learning algorithm to determine the damaged parts of the mobile phone's appearance. Determine the affected internal hardware components and their associated functional modules on the hardware structure mapping diagram to obtain the initial fault point; obtain the fault propagation path by analyzing cases in historical fault data. For cases in the candidate set, combine the initial fault point and analyze whether the fault information conforms to the propagation logic along the fault propagation path; use data mining technology to mine potential association rules between different damaged parts, visual features and fault information from an associated database. For each case in the candidate set, use the association rules to verify and check whether the visual features of the current mobile phone and the fault information meet the association rules.

[0037] According to the historical failure data subset of the same model of mobile phone in the association database, the prior probability is calculated for different combinations of damage parts and fault information, and the prior probability matrix is ​​constructed; after checking whether the visual features of the current mobile phone and the fault information meet the association rules, the Bayesian theorem reasoning method is used to update the probability estimate of the relationship between the appearance damage and the internal fault of the current mobile phone according to the prior probability matrix; Based on the probability estimation results, determine the most likely cause of the failure; generate a comprehensive analysis conclusion on the relationship between the appearance damage and internal failure of the mobile phone, explain the association logic between the appearance damage and internal failure, explain the application of the mined association rules in the current case, and how the visual features and failure information support or deviate from the association rules; A mobile phone appearance detection system based on visual recognition, comprising: Image acquisition and preprocessing module: including: image acquisition unit and image preprocessing unit; wherein, the image acquisition unit is equipped with an industrial camera, which can automatically adjust the focal length, aperture and exposure time according to the preset program, and includes a lighting system to provide a lighting environment for image acquisition; the image preprocessing unit preprocesses the acquired image, including grayscale, noise reduction and image enhancement; Visual feature extraction module: including: edge feature extraction unit, texture feature extraction unit and screen pixel feature extraction unit; wherein the edge feature extraction unit uses edge detection algorithm to identify the edge lines of various parts of the mobile phone; the texture feature extraction unit uses texture analysis algorithm to detect whether there are scratches, wear or stains on the appearance of the mobile phone; the screen pixel feature extraction unit extracts the pixel features of the display area of ​​the mobile phone screen, checks whether there is any display abnormality, and obtains the screen abnormality features; Model training and comparative analysis module: including: defect-free feature database construction unit, comparative analysis unit and model training unit; the defect-free feature database construction unit collects samples of new mobile phones of different brands and models without appearance defects, performs image acquisition, preprocessing and visual feature extraction, and constructs a defect-free mobile phone appearance feature database; the comparative analysis unit compares and analyzes the extracted visual features with the pre-established defect-free mobile phone appearance feature database; the model training unit uses machine learning algorithms to train the model so that the model learns the differences between normal mobile phone appearance features and various defect features; Internal fault information processing module: including: fault information acquisition unit, associated database construction unit and data integration and matching unit; wherein the fault information acquisition unit acquires the fault information stored inside the mobile phone; the associated database construction unit collects historical fault data and establishes an associated database of appearance damage and internal faults; the data integration and matching unit performs data integration and preliminary matching, considers the relationship between different damaged parts and various fault information, and performs multi-factor logical reasoning and verification; Comprehensive analysis and conclusion generation module: including: probability evaluation and conclusion generation unit; wherein the probability evaluation and conclusion generation unit performs probability evaluation and generates a comprehensive analysis conclusion on the relationship between the appearance damage and internal faults of the mobile phone.

[0038] In this embodiment, based on the association database, statistics show that for mobile phones of the same brand A and model M, when the appearance feature of camera lens damage occurs, the number of cases with internal camera failures (such as failure to focus, blurred imaging, application crashes, etc.) is 300, and the total number of cases of camera-related appearance damage in this model of mobile phone is 500. Therefore, the prior probability that camera lens damage leads to internal camera failure is 300 / 500=0.6.

[0039] Similarly, for internal faults related to dead pixels on the screen and display abnormalities, the statistical prior probability is 0.7 (based on database statistics); the prior probability of back cover wear and battery-related faults is 0.2 (because there are relatively few cases where back cover wear affects battery heat dissipation or contact stability).

[0040] Further analysis revealed that the mobile phone to be tested had a recent drop record (judged from the abnormal fluctuations in the data of the built-in acceleration sensor of the mobile phone). In the associated database, 150 cases of mobile phones with brand A and model M that had dropped and had damaged camera lenses were selected, of which 120 had internal camera failures. Therefore, under the drop condition, the conditional probability that the camera lens damage leads to internal camera failure is 120 / 150=0.8.

[0041] Similarly, considering that the yellowish screen display may be related to recent software updates (query the mobile phone system update log), the probability of display-related internal failures in mobile phones with the same software update status and dead pixels on the screen is 0.85, and the conditional probabilities of dead pixels on the screen and display abnormalities are updated.

[0042] Based on the probability assessment results, the following conclusions are drawn: The mobile phone camera cannot work properly. The main reason is that the camera lens is cracked and the surrounding area is deformed. Combined with the recent fall experience, the internal optical components or circuits of the camera are damaged, causing the camera application to crash and other faults. The probability of failure caused by this reason is 0.8, which is highly reliable.

[0043] The screen display is abnormal. Due to the bad pixel in the upper left corner of the screen and the software update, there is a high probability (probability 0.85) that the screen color calibration and display driver module have been affected, causing the overall display to be yellowish and the bad pixel area to malfunction.

[0044] There is a certain correlation between the battery power problem and the wear on the lower left corner of the back cover (probability 0.2). The wear may affect the heat dissipation of the battery and accelerate battery aging. In addition, there may be other unknown factors in the mobile phone that affect the accuracy of battery power estimation. It is recommended to further check the battery health and the circuits related to battery management on the motherboard.

[0045] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A mobile phone appearance detection method based on visual recognition, characterized in that: The method comprises: Capture and preprocess the image of the mobile phone to obtain the appearance image of the mobile phone; Extracting visual features based on the mobile phone appearance image, wherein the visual features include edge features, texture features, and screen abnormality features; The method for extracting visual features includes: using an edge detection algorithm to identify edge lines of various parts of a mobile phone; using a texture analysis algorithm to detect whether there are scratches, wear or stains on the appearance of the mobile phone; for a mobile phone screen, extracting pixel features of its display area, checking whether there is a display abnormality, and obtaining screen abnormality features; The extracted visual features are compared and analyzed with a pre-established database of appearance features of defect-free mobile phones, and a model is trained using a machine learning algorithm to enable the model to learn the differences between the appearance features of normal mobile phones and the features of various defects; Obtain the fault information stored in the mobile phone; collect historical fault data and establish a database of the association between external damage and internal faults; perform data integration and preliminary matching, consider the relationship between different damaged parts and various fault information, and perform multi-factor logical reasoning and verification; Conduct a probabilistic assessment and generate a comprehensive analysis conclusion on the relationship between the appearance damage and internal failures of the mobile phone.

2. The method for detecting the appearance of a mobile phone based on visual recognition according to claim 1, characterized in that: A Gaussian filter is used to smooth the mobile phone appearance image. The Gaussian filter performs weighted average calculation on each pixel point and its neighboring pixels in the image, and smoothes the mobile phone appearance image and retains the edge information of the mobile phone appearance image according to the set Gaussian kernel size and standard deviation; Calculate the gradient amplitude and direction of the image, and use the partial derivatives of the smoothed image in the horizontal and vertical directions to determine the gradient amplitude and direction of each pixel in the image. The gradient amplitude indicates the severity of the image change at the pixel, and the direction indicates the direction of the edge, highlighting the junction between the border and the panel in the mobile phone appearance image. Apply non-maximum suppression technology, along a certain edge direction, if the gradient amplitude of a pixel is not the maximum value in that direction, then it will be marked as a non-edge point.

3. The method for detecting the appearance of a mobile phone based on visual recognition according to claim 2, characterized in that: Through dual threshold detection and edge connection, a high threshold and a low threshold are set. Pixels above the high threshold are edge points, and pixels below the low threshold are not edge points. Pixels between the high threshold and the low threshold are judged as edge points based on their connectivity with the determined edge points. If a pixel is connected to a determined edge point and its gradient magnitude is greater than the low threshold, then the pixel is also considered an edge point; Connect the discontinuous edges to form edge lines of various parts of the phone.

4. The method for detecting the appearance of a mobile phone based on visual recognition according to claim 1, characterized in that: Using gray-level co-occurrence matrix to perform texture analysis; determining parameters for calculating gray-level co-occurrence matrix, including displacement direction and displacement distance; wherein, for detecting the texture of the surface of the mobile phone shell, the displacement direction selects horizontal and vertical directions, and the displacement distance is set according to the fineness of the texture of the mobile phone shell; Calculate the gray level co-occurrence matrix; for each pixel in the image, count the frequency of occurrence of the gray level combination with the pixel in the specified displacement direction and distance to form a gray level co-occurrence matrix; in a mobile phone appearance image, if the back cover has a texture pattern, then in the gray level co-occurrence matrix, the element value corresponding to a specific gray level value combination will reflect the frequency of occurrence of the gray level combination in the image; Texture feature parameters are extracted from the gray level co-occurrence matrix, and the texture feature parameters include contrast, correlation, energy and entropy; wherein the contrast reflects the clarity of the texture in the image and the depth of the texture grooves, and is used to detect scratches or worn areas; the correlation measures the linear dependence of the gray values ​​in the image; the energy represents the uniformity of the image gray distribution and the coarseness of the texture; the entropy reflects the randomness of the image texture, and scratches, wear or stains will change the randomness of the image texture, thereby causing a change in the entropy value.

5. The method for detecting the appearance of a mobile phone based on visual recognition according to claim 1, characterized in that: For a mobile phone screen, determine the range of the screen display area; based on the prior knowledge of the mobile phone model and screen size, determine the mobile phone screen display area in the mobile phone appearance image; Count the brightness and color value distribution of pixels in the display area; divide the screen display area into several rectangular grids, calculate the statistics of the pixels in each rectangular grid, including the average brightness, color mean and variance, and determine whether the statistics are normal, so as to check whether there is any display abnormality; Detect abnormal pixel clustering; set a threshold to determine whether a single pixel is an abnormal pixel. If the difference between the brightness value of a pixel and the average brightness value of the surrounding pixels exceeds the set brightness threshold, or the color value deviates from the normal color range, the pixel is marked as an abnormal pixel. Whether the color value deviates from the normal color range is determined based on the distance metric in the color space. Perform cluster analysis on the marked abnormal pixels. If it is found that several abnormal pixels are clustered together to form a local area, it is determined that the local area has screen display abnormality. Screen abnormal features are extracted based on the brightness and color value distribution of pixels in the display area and the aggregation of abnormal pixels.

6. The method for detecting the appearance of a mobile phone based on visual recognition according to claim 1, characterized in that: The method comprises: collecting mobile phone samples of different brands and models that are brand new and have no appearance defects; for each mobile phone, obtaining a set of standard appearance images of the mobile phone under different lighting conditions and different placement angles according to the same image acquisition and preprocessing process as that of actual detection; Extracting visual features from the standard appearance images in the standard appearance image set, sorting, classifying and storing the extracted visual feature data, and building a defect-free mobile phone appearance feature database; According to the brand and model information of the mobile phone, the corresponding non-defective mobile phone feature data subset in the non-defective mobile phone appearance feature database is located; for edge features, the differences in shape, length, curvature and continuity of edge lines in the same parts of the mobile phone to be tested and the non-defective mobile phone are compared; for texture features, the differences in texture feature parameters of the shell of the mobile phone to be tested and the non-defective mobile phone are compared to determine whether there are appearance defects. If there are appearance defects, the type and severity of the appearance defects are determined; for screen pixel features, the pixel brightness distribution, color mean and variance of the screen display area of ​​the mobile phone to be tested are compared with the deviation of the non-defective mobile phone. If there are abnormal brightness and color areas caused by local abnormal pixel aggregation, the location and possible causes of the screen display abnormality are determined in combination with the normal screen pixel features in the database.

7. The method for detecting the appearance of a mobile phone based on visual recognition according to claim 6, characterized in that: The method comprises: selecting an SVM algorithm as a machine learning algorithm, taking appearance feature data of non-defective mobile phones as positive samples, and taking collected feature data of mobile phones known to have various appearance defects as negative samples, and performing model training; The visual feature data is represented by feature vectors. By adjusting the kernel function and related parameters, the SVM model finds an optimal classification hyperplane to separate the appearance features of defective mobile phones from those of defective mobile phones, thereby learning the difference patterns between the appearance features of normal mobile phones and the features of various defects. During the model training process, the cross-validation method is used to evaluate and optimize the model performance.

8. The method for detecting the appearance of a mobile phone based on visual recognition according to claim 1, characterized in that: The method includes: the detection device is equipped with several interface adapter modules, which can be compatible with the connection interfaces of mobile phones of different brands and models; when the mobile phone is placed on the detection platform, the detection device automatically identifies the type of the mobile phone interface and obtains the fault information stored inside the mobile phone; Collect historical fault data from mobile phone repair centers, after-sales departments, and production testing. Each piece of fault data includes fault information and visual features. Establish a relational database structure and use MySQL to organize and store data. Each record in the database represents a mobile phone fault case, including fields such as mobile phone model, visual features, fault information, and fault occurrence time. According to the mobile phone model, a historical fault data subset of the same model of mobile phone is screened out in the associated database; the visual features of the current mobile phone are preliminarily matched with the visual features in the historical fault data subset to form a candidate set; Construct a hardware structure mapping diagram of the mobile phone's hardware structure and functions. For each case in the candidate set, use a model trained by a machine learning algorithm to determine the damaged parts of the mobile phone's appearance. Determine the affected internal hardware components and their associated functional modules on the hardware structure mapping diagram to obtain the initial fault point; obtain the fault propagation path by analyzing cases in historical fault data. For cases in the candidate set, combine the initial fault point and analyze whether the fault information conforms to the propagation logic along the fault propagation path; use data mining technology to mine potential association rules between different damaged parts, visual features and fault information from an associated database. For each case in the candidate set, use the association rules to verify and check whether the visual features of the current mobile phone and the fault information meet the association rules.

9. The method for detecting the appearance of a mobile phone based on visual recognition according to claim 8, characterized in that: The method comprises: calculating the prior probability for different combinations of damaged parts and fault information according to a subset of historical fault data of the same model of mobile phones in a correlation database, and constructing a prior probability matrix; after checking whether the visual features of the current mobile phone and the fault information satisfy the association rules, using the Bayesian theorem reasoning method, updating the probability estimation of the relationship between the appearance damage and the internal fault of the current mobile phone according to the prior probability matrix; Based on the probability estimation results, the most likely cause of the failure is determined; a comprehensive analysis conclusion on the relationship between the appearance damage and internal failure of the mobile phone is generated, the association logic between the appearance damage and internal failure is explained, the application of the mined association rules in the current case is explained, and how the visual features and failure information support or deviate from the association rules.

10. A mobile phone appearance detection system based on visual recognition, using a mobile phone appearance detection method based on visual recognition according to any one of claims 1 to 9, characterized in that: include: Image acquisition and preprocessing module: including: image acquisition unit and image preprocessing unit; wherein, the image acquisition unit is equipped with an industrial camera, automatically adjusts the focal length, aperture and exposure time according to the preset program, and includes a lighting system to provide a lighting environment for image acquisition; the image preprocessing unit preprocesses the acquired image, including grayscale, noise reduction and image enhancement; Visual feature extraction module: including: edge feature extraction unit, texture feature extraction unit and screen pixel feature extraction unit; wherein the edge feature extraction unit uses edge detection algorithm to identify the edge lines of various parts of the mobile phone; the texture feature extraction unit uses texture analysis algorithm to detect whether there are scratches, wear or stains on the appearance of the mobile phone; the screen pixel feature extraction unit extracts the pixel features of the display area of ​​the mobile phone screen, checks whether there is any display abnormality, and obtains the screen abnormality features; Model training and comparative analysis module: including: defect-free feature database construction unit, comparative analysis unit and model training unit; the defect-free feature database construction unit collects samples of new mobile phones of different brands and models without appearance defects, performs image acquisition, preprocessing and visual feature extraction, and constructs a defect-free mobile phone appearance feature database; the comparative analysis unit compares and analyzes the extracted visual features with the pre-established defect-free mobile phone appearance feature database; the model training unit uses machine learning algorithms to train the model so that the model learns the differences between normal mobile phone appearance features and various defect features; Internal fault information processing module: including: fault information acquisition unit, associated database construction unit and data integration and matching unit; wherein the fault information acquisition unit acquires the fault information stored inside the mobile phone; the associated database construction unit collects historical fault data and establishes an associated database of appearance damage and internal faults; the data integration and matching unit performs data integration and preliminary matching, considers the relationship between different damaged parts and various fault information, and performs multi-factor logical reasoning and verification; Comprehensive analysis and conclusion generation module: including: probability evaluation and conclusion generation unit; wherein the probability evaluation and conclusion generation unit performs probability evaluation and generates a comprehensive analysis conclusion on the relationship between the appearance damage and internal faults of the mobile phone.

Images