Notebook computer appearance defect identification method and device based on convolutional neural network

Abstract

The invention provides a notebook appearance defect identification method and device based on a convolutional neural network, and the method comprises the steps: obtaining the appearance feature data of a notebook based on a defect identification signal, and the appearance feature data at least comprise the large surface data, end surface data and bottom surface data of the notebook; appearance data features are obtained and input into a convolutional neural network detection model to obtain detection data, wherein the convolutional neural network detection model is obtained based on appearance defect sample data training; and determining a detection result of the notebook based on the detection data. According to the invention, the method achieves the detection of the appearance defects of various types of notebook computers through the trained convolutional neural network detection model, is wide in notebook computer appearance model coverage, does not need to independently design specific rules and algorithms for each type in the appearance defect detection process of various types of notebook computers, and improves the detection efficiency. The system development and maintenance difficulty is reduced, and the defect detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a method and device for identifying appearance defects of notebooks based on convolutional neural networks. Background Art

[0002] With the continuous expansion of the electronic equipment market, the notebook computer industry has developed rapidly in the past decade, and has gradually become a daily necessity for people to study and work. With the improvement of consumers' requirements for product quality, consumers not only pay attention to the performance of notebooks, but also pay more attention to the appearance of notebooks. Therefore, in the production field of notebook computer shells, the shells that have been machined and surface treated are not allowed to have defects such as black spots, spots, streaks, scratches, etc. that can be found by the naked eye. Therefore, in the process of computer product manufacturing, defect detection for notebook shells to ensure that only qualified good products can flow into the market has become a key link in the market competitiveness of the product. Therefore, manufacturers will have a special process for quality inspection during production.

[0003] Traditional defect detection methods mainly rely on manual visual inspection, which has many limitations, such as low efficiency, high cost, inconsistent inspection standards, and errors caused by human factors. In addition, as laptops become thinner and lighter, the requirements for assembly process and manufacturing accuracy are getting higher and higher, which further increases the challenges of traditional inspection methods.

[0004] In order to overcome these limitations, the industry has begun to explore automated inspection technologies. Combined with machine vision technology, it has shown great potential in the detection of missing parts in notebook cases, and more and more factories have begun to use visual inspection equipment to detect product defects.

[0005] However, the disadvantage is that when faced with defects of various different types, despite the availability of corresponding machine vision technology, it may be necessary to consider the prominence of various types of defects in different scenarios, and it is necessary to use various different rule algorithms to perform more complex image processing. And this is only for the shell of a certain model material, and for the shells of various other models and materials, corresponding analysis and processing are required. In other words, it usually requires the participation of a large number of professionals and a lot of time to complete the extraction and description of various types of defect features in multi-model product images and the writing of detection rule algorithms. Summary of the invention

[0006] The present invention provides a method and device for identifying notebook appearance defects based on a convolutional neural network, which is used to solve the problem that when facing defects of various different feature types, despite the corresponding machine vision technology, it may be necessary to consider the prominence of various types of defects in different scenes, and it is necessary to use a variety of different rule algorithms to perform more complex processing on the image, and the defects of a single model can be processed.

[0007] According to the first aspect of the present invention, a notebook appearance defect recognition algorithm based on a convolutional neural network is provided, comprising: Based on the defect recognition signal, acquiring appearance feature data of the notebook, the appearance feature data at least including large surface data, end surface data and bottom surface data of the notebook; The appearance data features are obtained and input into a convolutional neural network detection model to obtain detection data, wherein the convolutional neural network detection model is trained based on appearance defect sample data; A detection result of the notebook is determined based on the detection data.

[0008] According to an embodiment of the present invention, obtaining appearance feature data of the notebook specifically includes: Acquire an appearance inspection area of ​​the notebook, and extract regional morphological features of the appearance inspection area, wherein the appearance inspection area at least includes a large surface area, an end surface area, and a bottom surface area of ​​the notebook, and the regional morphological features at least include any one or a combination of particles, paint accumulation, scratches, dents, and over-abrasion in the large surface area, the end surface area, and the bottom surface area; Based on the appearance detection area and the regional morphological features, large surface data of the large surface area, end surface data of the end surface area, and bottom surface data of the bottom surface area are respectively extracted.

[0009] Specifically, this embodiment provides an implementation method for obtaining appearance feature data of a notebook.

[0010] According to an embodiment of the present invention, after acquiring the appearance data features and inputting them into a convolutional neural network detection model to obtain detection data, the method specifically includes: Based on the detection data, determining a plurality of defect target locations of the notebook, wherein the defect target locations at least include defects on a large surface area, an end surface area, and a bottom surface area of ​​the notebook; Based on the plurality of defective target locations, overlapping areas of the plurality of defective target locations are determined, and redundant overlapping areas are removed from all the defective target locations.

[0011] Specifically, this embodiment provides an implementation method for obtaining the appearance data features and inputting them into a convolutional neural network detection model to obtain detection data.

[0012] According to an embodiment of the present invention, after removing the redundant overlapping areas in all the defect target positions, the method specifically includes: Adding a maximum external detection frame to each defect target position after removing the overlapping area; The detection result is generated based on all the defective target positions with the maximum external detection frame added.

[0013] Specifically, this embodiment provides an implementation method after removing the redundant overlapping areas in all the defect target positions.

[0014] According to an embodiment of the present invention, before acquiring the appearance feature data of the notebook based on the recognition signal, the method specifically includes: Based on an input signal, acquiring model feature data of the notebook, wherein the input operation corresponding to the input signal at least includes inputting part of the model feature data of the notebook; Obtaining a classification result of the model feature data input into a convolutional neural network strategy model, wherein the convolutional neural network strategy model is trained based on the model feature sample data of the notebook; A defect detection strategy for the notebook is determined based on the classification result, and the defect detection strategy is transmitted to a convolutional neural network detection model.

[0015] Specifically, this embodiment provides an implementation method before acquiring appearance feature data of a notebook.

[0016] According to an embodiment of the present invention, the step of obtaining the model feature data of the notebook based on the input signal further includes: Acquire the set detection defects of the notebook identified by the input operation, wherein the set detection defects include any one or a combination of length defects, area defects, width defects, particle defects, paint accumulation defects, scratch defects, hole defects and over-wear defects of the notebook; Defects are detected based on the settings, and the model feature data is updated.

[0017] Specifically, this embodiment provides an implementation method for acquiring the model feature data of the notebook based on an input signal.

[0018] According to an embodiment of the present invention, the convolutional neural network detection model is obtained by training based on appearance defect sample data, and specifically includes: Based on the model training signal, obtaining appearance defect sample data of the notebook sample, wherein the appearance defect sample data at least includes a large surface defect sample, an end surface defect sample, and a bottom surface defect sample of the notebook sample; Preprocessing the appearance defect sample data, wherein the preprocessing includes performing any one or a combination of size standardization, grayscale conversion, denoising, data enhancement, and image scaling on the appearance defect sample data; When the appearance defect position data of the notebook sample identified by the appearance defect sample data is less than a set minimum value, the number of current defects at the appearance defect position is expanded based on the AIDG model; The convolutional neural network detection model is obtained by training based on all the appearance defect sample data.

[0019] Specifically, this embodiment provides an implementation method for training a convolutional neural network detection model.

[0020] According to an embodiment of the present invention, the training of the convolutional neural network detection model based on all the appearance defect sample data specifically includes: Segmenting the appearance defect sample data based on a model segmentation algorithm; Based on the segmented appearance defect sample data, performing migration training on the convolutional neural network detection model; Based on the back propagation algorithm, the network parameters of the convolutional neural network detection model are adjusted until the convolutional neural network detection model converges.

[0021] Specifically, this embodiment provides an implementation method for obtaining the convolutional neural network detection model based on training of all the appearance defect sample data.

[0022] According to an embodiment of the present invention, the convolutional neural network detection model is obtained by training based on all the appearance defect sample data, and specifically includes: Based on the segmented appearance defect sample data, a convolutional neural network is trained to detect a first sub-model, a second sub-model, a third sub-model and a fourth sub-model, wherein the training sample of the convolutional neural network for detecting the first sub-model is the cropped large surface line scan image data of the notebook sample, the training sample of the convolutional neural network for detecting the second sub-model is the large surface array image data of the notebook sample, the training sample of the convolutional neural network for detecting the third sub-model is the end surface line scan image data after cropping and splicing, and the training sample of the convolutional neural network for detecting the fourth sub-model is the bottom surface line scan image data of the notebook sample; Model migration training is performed on the first sub-model of the convolutional neural network detection, the second sub-model of the convolutional neural network detection, the third sub-model of the convolutional neural network detection and the fourth sub-model of the convolutional neural network detection to obtain the convolutional neural network detection model.

[0023] Specifically, this embodiment provides another implementation method of obtaining the convolutional neural network detection model based on training of all the appearance defect sample data.

[0024] According to a second aspect of the present invention, a notebook appearance defect recognition device based on a convolutional neural network is provided, comprising: A feature acquisition module, used to acquire appearance feature data of the notebook based on the defect recognition signal, wherein the appearance feature data at least includes the large surface data, the end surface data and the bottom surface data of the notebook; A data detection module, used to obtain the appearance data features and input them into a convolutional neural network detection model to obtain detection data, wherein the convolutional neural network detection model is trained based on appearance defect sample data; A result determination module is used to determine the detection result of the notebook based on the detection data.

[0025] The above one or more technical solutions in the present invention have at least one of the following technical effects: the present invention provides a method and device for identifying notebook appearance defects based on a convolutional neural network, which adopts a convolutional neural network algorithm based on deep learning and machine vision technology as the program algorithm core of the visual detection device, and uses a trained convolutional neural network detection model to realize the detection of appearance defects of various models of notebooks. It has a wide coverage of notebook appearance models. In the process of appearance defect detection of various models of notebooks, there is no need to design specific rules and algorithms for each model separately, which reduces the difficulty of system development and maintenance and improves the efficiency of defect detection.

[0026] Furthermore, compared with traditional machine vision technology, the technical solution of the present invention shows significant advantages in defect detection.

[0027] In traditional machine vision technology, defects are mainly detected by a series of preset rules and algorithms. The main disadvantages of this method include: the complexity of targeted rules. For different models of laptop shells, the characteristics and defects of each model may be significantly different. Traditional technology requires the design of specific rules and algorithms for each model. This not only increases the complexity of system development and maintenance, but also leads to a large number of rule algorithms, which are difficult to manage uniformly. Reliance on manual experience. Traditional rules and algorithms usually rely on the experience and knowledge of engineers. This method may need to redevelop and adjust the algorithm when facing new product models or unknown defect types, thereby increasing response time and development costs. Poor adaptability. When the design or production process of the laptop shell changes, the traditional machine vision system may not be able to adapt quickly, resulting in a decrease in detection capabilities or inability to accurately identify new defects. The adjustment and optimization of this system usually requires redevelopment of algorithms and parameter adjustments.

[0028] In addition, compared with traditional machine vision technology, deep learning models are more versatile and adaptive. Deep learning models can automatically learn and identify defects in the shells of various models of laptops through large-scale training data. During the training process, the model can extract common features from the defect characteristics of different models, without the need to design specific algorithms for each model, thus achieving unified detection capabilities for multiple models. It is also highly efficient. The deep learning model can handle various types of defects under a unified framework, without the need to develop and maintain rules independently for each model. This not only simplifies the development and maintenance of the system, but also significantly improves detection efficiency and accuracy. The same basic model can cover defect detection of defective parts of all models and also lay a good foundation for rapid model change. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 It is a flow chart of a method for identifying appearance defects of a notebook based on a convolutional neural network provided by the present invention; Figure 2 It is a structural schematic diagram of a notebook appearance defect recognition device based on a convolutional neural network provided by the present invention; Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention.

[0031] Reference numerals: 10. Feature acquisition module; 20. Data detection module; 30. Result determination module; 810, processor; 820, communication interface; 830, memory; 840, communication bus. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] The present invention is described in detail below in conjunction with the accompanying drawings of the specification. The specific operating method in the method embodiment can also be applied to the device embodiment or the system embodiment. In the description of the present invention, unless otherwise specified, "at least one" includes one or more. "Multiple" refers to two or more. For example, at least one of A, B and C includes: A exists alone, B exists alone, A and B exist at the same time, A and C exist at the same time, B and C exist at the same time, and A, B and C exist at the same time. In the present invention, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a kind of association relationship that describes the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0034] The present invention will be described in detail below in conjunction with specific implementation modes.

[0035] In some specific embodiments of the present invention, Figure 1 As shown, this scheme provides a notebook appearance defect recognition algorithm based on convolutional neural network, including: Based on the defect recognition signal, acquiring appearance feature data of the notebook, the appearance feature data at least including the notebook's large surface data, end surface data, and bottom surface data; After obtaining the appearance data features and inputting them into the convolutional neural network detection model, the detection data is obtained. The convolutional neural network detection model is trained based on the appearance defect sample data; A detection result of the notebook is determined based on the detection data.

[0036] It should be noted that in order to solve the limitations of the existing technology for notebook appearance inspection, the present invention adopts a convolutional neural network algorithm based on deep learning and machine vision technology as the program algorithm core of the visual inspection method, and cooperates with a visual mechanical device to realize the automatic inspection of the notebook shell, and a set of system models can be used to complete the different requirements for defect detection of different models.

[0037] Furthermore, the present invention has a missed detection rate of less than 5% and an over-detection rate of less than 5% for notebook shell defects, and covers a wide range of notebook shell detection models, including but not limited to different materials such as aluminum-magnesium alloy, carbon fiber, engineering plastics, and different colors such as silver, gray, black, blue, and various sizes. It covers all parts of the notebook shell, including comprehensive detection of 1 large surface, 4 end surfaces, and 1 PL surface (bottom surface), and supports detection of difficult parts such as front arc surfaces, arc top angles, and side edges. It covers many types of defects, including but not limited to detection of more than ten kinds of defects such as particles, scratches, holes, and over-wear. It supports rapid model change, and the model change time is less than 10 minutes, which does not affect the normal production of the factory.

[0038] In some possible implementations of the present invention, obtaining appearance feature data of a notebook specifically includes: Acquire an appearance inspection area of ​​the notebook and extract regional morphological features of the appearance inspection area, wherein the appearance inspection area at least includes a large surface area, an end surface area and a bottom surface area of ​​the notebook, and the regional morphological features at least include any one or a combination of particles, paint accumulation, scratches, dents and over-abrasion in the large surface area, the end surface area and the bottom surface area; Based on the appearance detection area and regional morphological features, large surface data of the large surface area, end surface data of the end surface area, and bottom surface data of the bottom surface area are extracted respectively.

[0039] Specifically, this embodiment provides an implementation method for obtaining the appearance feature data of a notebook. By acquiring the appearance detection area of ​​the notebook, the regional morphological features of the appearance of the notebook can be extracted, and then when the notebook appearance has defects such as particles, paint accumulation, scratches, holes and over-wear, it can be identified in time.

[0040] Furthermore, in order to more accurately and completely reflect the defects of the notebook appearance, the appearance of the notebook is divided into a large surface area, an end surface area and a bottom surface area. By partitioning the notebook appearance, it is convenient to collect data in the corresponding areas and classify and sort the data, so as to determine the defects in the corresponding areas. It can be understood that the large surface area, the end surface area and the bottom surface area are the appearance detection areas of the notebook appearance. Based on the appearance detection areas and the corresponding regional morphological features, the large surface data of the large surface area, the end surface data of the end surface area and the bottom surface data of the bottom surface area can be determined.

[0041] In a possible embodiment, the large surface area includes the front and back of the notebook. The detection requirement for the front side of a certain model is that 4 particles smaller than 0.2mm2 are allowed to appear, 2 particles with 0.2mm2<area<=0.3mm2 are allowed to appear, 2 particles with 0.3mm2<area<=0.5mm2 are allowed to appear, and 1 particle with 0.5mm2<area<=0.7mm2 is allowed to appear.

[0042] In a possible embodiment, each defect has its corresponding requirement. For different surfaces, the end surface and the PL surface (bottom surface) also have corresponding different requirements, and include a variety of other different requirements, such as no defects are allowed on the long rear end surface, for front defects, two defects with an area of ​​0.2 mm are not allowed within 30 mm, for front defects, two defects with an area of ​​0.2 mm are not allowed within 30 mm, and so on.

[0043] In a possible embodiment, different models and colors of notebook shells may have other different requirements, and a defect alarm may be issued for the appearance of notebooks with color difference or the like.

[0044] In some possible implementations of the present invention, after acquiring the appearance data features and inputting them into the convolutional neural network detection model to obtain the detection data, the following steps are specifically included: Based on the detection data, a plurality of defect target locations of the notebook are determined, wherein the defect target locations at least include defects on a large surface area, an end surface area, and a bottom surface area of ​​the notebook; Based on the multiple defective target locations, overlapping areas of the multiple defective target locations are determined, and redundant overlapping areas are removed from all the defective target locations.

[0045] Specifically, this embodiment provides an implementation method for obtaining appearance data features and inputting them into a convolutional neural network detection model to obtain detection data. After the detection data obtained by the convolutional neural network detection model is processed, there may be overlapping appearance feature data detected in some appearance detection areas, that is, there is an overlapping problem in the detected appearance defects of the notebook. Therefore, it is necessary to determine multiple defect target positions of the notebook based on the detection data, and remove the overlapping defect target positions, thereby improving the detection accuracy.

[0046] In a possible embodiment, the present invention processes the detection data obtained by the convolutional neural network detection model and uses operations such as non-maximum suppression (NMS) to remove redundant detection results to improve detection accuracy. For example, a defect target model detects multiple defect objects and removes redundant targets with overlapping object areas. In addition, according to the different detection requirements of different product models, the detected defects can be further filtered according to specific needs to achieve more accurate detection.

[0047] In some possible implementations of the present invention, after removing redundant overlapping areas in all defect target positions, the following steps are specifically performed: Add a maximum external detection box to each defect target position after removing the overlapping area; Generate detection results based on all defect target positions with the maximum external detection frame added.

[0048] Specifically, this embodiment provides an implementation method after removing redundant overlapping areas within all defect target positions. After processing the detection data obtained by the convolutional neural network detection model, a maximum external detection box is added to each defect target position after removing the overlapping areas, thereby improving the defect recognition and detection accuracy.

[0049] In a possible embodiment, based on the defect target positions to which the maximum external detection frames are added, a maximum external rectangular frame of the obtained defect mask is drawn on the original image for display, so as to improve the recognition of the defect.

[0050] In some possible implementations of the present invention, before acquiring the appearance feature data of the notebook based on the recognition signal, the following steps are specifically included: Based on the input signal, obtaining model characteristic data of the notebook, wherein the input operation corresponding to the input signal includes at least inputting part of the model characteristic data of the notebook; Obtain the classification results of the model feature data input into the convolutional neural network strategy model, which is trained based on the model feature sample data of the notebook; The defect detection strategy of the notebook is determined based on the classification results, and the defect detection strategy is transferred to the convolutional neural network detection model.

[0051] Specifically, this embodiment provides an implementation method before obtaining the appearance feature data of the notebook. During the production process of the notebook, different notebook models will be produced on the same production line according to different business needs, such as origin, customer demand, product iteration, etc. After the notebook model is changed, it is necessary to determine the defect detection strategy according to the new notebook model. The present invention obtains the model feature data of the notebook based on the input signal. It can be understood that the model feature data of the notebook can be a specific model marked with the notebook, or it can be the special needs of the customer, such as the color, surface flatness, special flaws and defects of the notebook, and it can also be different precisions of notebook defects, such as particle size, scratch area, dent area, etc.

[0052] Furthermore, after obtaining the model feature data of the notebook, the model feature data is input into the convolutional neural network strategy model to obtain the corresponding classification result of the notebook, and the defect detection strategy of the current model of the notebook is determined according to the classification result.

[0053] In a possible embodiment, the existing defect detection of multiple models of notebook production lines usually uses manual defect judgment or machine vision technology to detect notebook appearance defects. However, the existing image processing usually obtains results after image segmentation and calculation, which is highly correlated with the notebook model and cannot meet the needs of multi-model notebook defect detection. The present invention, through the setting of a convolutional neural network strategy model, can achieve rapid model classification during the notebook model replacement process, and quickly generate defect detection strategies according to different defect detection needs, meet the needs of switching between multiple notebook models in a short period of time, and achieve rapid change of notebook appearance defect detection without affecting production progress, so that a set of visual equipment system can detect multiple models of notebook shells.

[0054] In a possible embodiment, the entire process of changing the notebook model and determining the policy model, detection model, etc. takes less than 10 minutes.

[0055] In some possible implementations of the present invention, obtaining the model feature data of the notebook based on the input signal specifically includes: Obtaining set detection defects of the notebook with the input operation identifier, where the set detection defects include any one or a combination of length defects, area defects, width defects, particle defects, paint accumulation defects, scratch defects, dent defects and over-wear defects of the notebook; Based on the set detection defects, the model feature data is updated.

[0056] Specifically, this embodiment provides an implementation method for obtaining model feature data of a notebook based on an input signal. By setting detection defects for the notebook identified by the input operation, the main defects that need to be detected for the notebook of the replaced model can be determined, which can be more in line with the detection requirements of the notebook after the model is replaced.

[0057] It should be noted that the detection model of the present invention is seamlessly integrated with the strategy model. The core advantage of the model lies in its versatility and flexibility, and there is no need to retrain or replace the model for each model. The configuration parameters of the notebook defect detection strategy are passed to the detection model through the interface and used as input parameters to guide the defect detection process of the model. Once the user enters a new detection requirement or sets a detection defect on the industrial computer, the system can automatically complete the model change and immediately execute the new defect detection strategy. The model automatically adapts to the current model of the notebook shell according to the new defect detection strategy setting, ensuring that each model change process is fast, accurate and does not require interruption of the production line. This model change process can be achieved only through simple parameter configuration, which greatly reduces the time for manual adjustment and improves production efficiency.

[0058] In a possible embodiment, since different customers may have different requirements for the detection of the same model, in order to achieve flexible response to customer customization needs, the convolutional neural network strategy model of the present invention can quickly respond to the needs of different customers to change models, and make personalized settings for each customer's special requirements to achieve different defect detection strategies. For example, customer A may want to be more strict with surface scratches, while customer B pays more attention to the detection of particle defects. These requirements can be met by adjusting the defect detection strategy without the need for additional algorithm development or model replacement.

[0059] In a possible embodiment, the construction of the convolutional neural network strategy model is carried out in combination with the business rules of notebook appearance defect detection and the deep learning classification algorithm, and quantifiable and regularized defects, as well as defects that cannot be quantified (such as sandpaper marks) are reported and filtered respectively to ensure that defects are reported in accordance with business instructions. The convolutional neural network strategy model is exposed in the form of a configuration page and supports modification. Fast switching between different models only requires changing the defect detection strategy, without retraining the strategy model and the detection model, which can achieve fast model change.

[0060] It can be understood that the convolutional neural network strategy model is a rule-based algorithm that mainly solves the reporting and filtering of quantifiable defects. The rules include: defect length, area, width, defect distance, defect number, etc.

[0061] In a possible embodiment, the present invention also classifies the severity of notebook appearance defects through a convolutional neural network classification model based on deep learning, reports and filters them according to different severities, and solves the problem of reporting and filtering defects that cannot be quantified.

[0062] In some possible implementations of the present invention, the convolutional neural network detection model is trained based on appearance defect sample data, specifically including: Based on the model training signal, obtaining appearance defect sample data of the notebook sample, the appearance defect sample data at least including large surface defect samples, end surface defect samples and bottom surface defect samples of the notebook sample; Preprocessing the appearance defect sample data, including any one or a combination of size standardization, grayscale, denoising, data enhancement and image scaling of the appearance defect sample data; When the appearance defect location data of the notebook sample identified by the appearance defect sample data is less than the set minimum value, the number of current defects at the appearance defect location is expanded based on the AIDG model; The convolutional neural network detection model is trained based on all appearance defect sample data.

[0063] Specifically, this embodiment provides an implementation method for training a convolutional neural network detection model, by acquiring appearance defect sample data of notebook samples, and preprocessing the appearance defect sample data, and then training the convolutional neural network detection model through the preprocessed appearance defect sample data.

[0064] In a possible embodiment, after collecting the sample data of the appearance defects of the notebook sample shell, it is not directly labeled and then trained using the convolutional neural network detection model. Since the defects that need to be detected on the appearance of the notebook are all small targets, if the input size of the model is set too large, it will greatly increase the proportion of the background, resulting in an extremely unbalanced proportion of the detection target and the background, which will further reduce the training effect of the model. Therefore, it is necessary to ensure that the input size of the model is appropriate. Before entering the model training, the image data needs to be scaled to a certain extent to adapt to the input of the model, but this will also reduce the characteristics of the defect information and increase the learning difficulty of the model. Therefore, the global line scan image is cropped into multiple blocks to adapt to the size ratio of the model input. For the end face data, since the length-to-width ratio is too large, it is not conducive to training. The image data is processed by a cutting and splicing strategy before training.

[0065] In a possible embodiment, image data of all existing models of notebook shells with various defects are collected, and the collected image data are classified and labeled, including the location and category of the defects, to form the label data required for training. The image data is also preprocessed as necessary, such as image size standardization, grayscale, denoising, data enhancement, image scaling adjustment, etc., to meet the requirements of model input. All model data are unified for training to lay a solid foundation for rapid model change of product defect detection. Some types of defect samples in the training data are very scarce, resulting in insufficient and extremely unbalanced sample data during model training. In order to solve this problem, the AIDG (Artificial Intelligent Defect Generator) defect image generation large model is introduced. The defect image data generated by the AIDG model greatly expands the number of scarce samples and enhances the diversity of data.

[0066] In some possible implementations of the present invention, a convolutional neural network detection model is obtained based on training of all appearance defect sample data, specifically including: Based on the model segmentation algorithm, the appearance defect sample data is segmented; Based on the segmented appearance defect sample data, transfer training is performed on the convolutional neural network detection model; Based on the back propagation algorithm, the network parameters of the convolutional neural network detection model are adjusted until the convolutional neural network detection model converges.

[0067] Specifically, this embodiment provides an implementation method for obtaining a convolutional neural network detection model based on all appearance defect sample data training, and uses a convolutional neural network model instance segmentation algorithm to perform segmentation training on defects in the image. The sampling model is transferred and trained, and the training of the present invention is secondary training on a model with preliminary learning ability, based on a labeled data set training model, and the network parameters are continuously adjusted through a back propagation algorithm until the model converges, thereby improving the training efficiency.

[0068] In a possible embodiment, the present invention uses a deep learning segmentation network model based on SwinTransformer as the backbone network to learn defect state patterns and achieve the ability to detect pixel-level defect locations. Based on the segmentation algorithm, the defect area can be accurately calculated. Swin Transformer has the characteristics of cross-layer connection and multi-scale feature fusion, which can effectively improve the accuracy of the model. The appearance defects of the notebook are small target defects, such as scratches, dents, particles, etc. Swin Transformer's multi-scale prediction and feature fusion technology enable it to process defects of different sizes, meet the defect detection needs of large-area dents and fine particles, and improve the accuracy of detection.

[0069] In some possible implementations of the present invention, a convolutional neural network detection model is obtained by training based on all appearance defect sample data, specifically including: Based on the segmented appearance defect sample data, the first sub-model of the convolutional neural network detection, the second sub-model of the convolutional neural network detection, the third sub-model of the convolutional neural network detection and the fourth sub-model of the convolutional neural network detection are trained. The training samples of the first sub-model of the convolutional neural network detection are the large surface line scan image data of the cropped notebook sample, the training samples of the second sub-model of the convolutional neural network detection are the large surface array image data of the notebook sample, the training samples of the third sub-model of the convolutional neural network detection are the end surface line scan image data after cropping and splicing, and the training samples of the fourth sub-model of the convolutional neural network detection are the bottom surface line scan image data of the notebook sample; Model transfer training is performed on the first sub-model of convolutional neural network detection, the second sub-model of convolutional neural network detection, the third sub-model of convolutional neural network detection and the fourth sub-model of convolutional neural network detection to obtain a convolutional neural network detection model.

[0070] Specifically, this embodiment provides another implementation method for obtaining a convolutional neural network detection model based on training of all appearance defect sample data. Due to different locations, the defect detection of the notebook appearance will have different sensitivities. The present invention adopts four sub-models, namely, the first sub-model of convolutional neural network detection, the second sub-model of convolutional neural network detection, the third sub-model of convolutional neural network detection, and the third sub-model of convolutional neural network detection, to respectively carry out targeted training on four parts of the notebook appearance where defects are relatively concentrated. The parts where defects are relatively concentrated are the small images after cropping of the large surface line scan image data of the shell, the edge array and the logo area image, the image data after the end face is cropped and spliced, and the bottom line scan image data of the notebook sample. Subdividing different areas and selecting independent models for training is helpful for the overall recognition effect. During reasoning, the image data of the corresponding parts are directly predicted using the corresponding sub-model.

[0071] It should be noted that after summarizing and classifying the four areas where defects are more concentrated on the notebook's appearance, the training samples of the first sub-model of the convolutional neural network detection are the large-surface line scan image data of the cropped notebook samples, the training samples of the second sub-model of the convolutional neural network detection are the large-surface array image data of the notebook samples, the training samples of the third sub-model of the convolutional neural network detection are the end surface line scan image data of the notebook samples after cropping and splicing, and the training samples of the fourth sub-model of the convolutional neural network detection are the bottom surface line scan image data of the notebook samples.

[0072] In a possible embodiment, the present invention also strengthens the training of reflux data, specifically, according to the performance of the model in actual application, the cases with poor performance are picked out and classified, and the causes of their occurrence are analyzed, and solutions are found, and the data with poor detection are collected for model training to iterate the new version of the model, so as to further improve the model's adaptability to scenarios and data and achieve better detection results.

[0073] In a possible embodiment, after the present invention collects all-round data of the entire notebook shell, the program will transmit the data to the server, complete the detection of the detection model on the server and feedback (if only the detection training data is collected, this process is ignored, and the program will only save the collected image data) from the image data of the large surface, end surface and bottom surface detection, and the program on the server will make a final judgment on the detected defects through the strategy model and the detection model. Generate a test report based on the test results, mark the type, location and severity of all detected defects, and provide a qualified / unqualified judgment.

[0074] In some specific embodiments of the present invention, Figure 2 As shown, this solution provides a notebook appearance defect recognition device based on a convolutional neural network, comprising: The feature acquisition module 10 is used to acquire appearance feature data of the notebook based on the defect recognition signal, where the appearance feature data at least includes the large surface data, the end surface data and the bottom surface data of the notebook; A data detection module 20 is used to obtain appearance data features and input them into a convolutional neural network detection model to obtain detection data. The convolutional neural network detection model is trained based on appearance defect sample data; The result determination module 30 is used to determine the detection result of the notebook based on the detection data.

[0075] Optionally, obtaining appearance feature data of the notebook includes: Acquire an appearance inspection area of ​​the notebook and extract regional morphological features of the appearance inspection area, wherein the appearance inspection area at least includes a large surface area, an end surface area and a bottom surface area of ​​the notebook, and the regional morphological features at least include any one or a combination of particles, paint accumulation, scratches, dents and over-abrasion in the large surface area, the end surface area and the bottom surface area; Based on the appearance detection area and regional morphological features, large surface data of the large surface area, end surface data of the end surface area, and bottom surface data of the bottom surface area are extracted respectively.

[0076] Specifically, this embodiment provides an implementation method for obtaining appearance feature data of a notebook.

[0077] Optionally, after obtaining the appearance data features and inputting them into the convolutional neural network detection model to obtain the detection data, the following steps are specifically included: Based on the detection data, a plurality of defect target locations of the notebook are determined, wherein the defect target locations at least include defects on a large surface area, an end surface area, and a bottom surface area of ​​the notebook; Based on the multiple defective target locations, overlapping areas of the multiple defective target locations are determined, and redundant overlapping areas are removed from all the defective target locations.

[0078] Specifically, this embodiment provides an implementation method for obtaining appearance data features and inputting them into a convolutional neural network detection model to obtain detection data.

[0079] Optionally, after removing redundant overlapping areas in all defect target positions, specifically including: Add a maximum external detection box to each defect target position after removing the overlapping area; Generate detection results based on all defect target positions with the maximum external detection frame added.

[0080] Specifically, this embodiment provides an implementation method after removing redundant overlapping areas in all defect target positions.

[0081] Optionally, before acquiring the appearance feature data of the notebook based on the recognition signal, the method specifically includes: Based on the input signal, obtaining model characteristic data of the notebook, wherein the input operation corresponding to the input signal includes at least inputting part of the model characteristic data of the notebook; Obtain the classification results of the model feature data input into the convolutional neural network strategy model, which is trained based on the model feature sample data of the notebook; The defect detection strategy of the notebook is determined based on the classification results, and the defect detection strategy is transferred to the convolutional neural network detection model.

[0082] Specifically, this embodiment provides an implementation method before acquiring appearance feature data of a notebook.

[0083] Optionally, obtaining the model feature data of the notebook based on the input signal specifically further includes: Obtaining set detection defects of the notebook with the input operation identifier, where the set detection defects include any one or a combination of length defects, area defects, width defects, particle defects, paint accumulation defects, scratch defects, dent defects and over-wear defects of the notebook; Based on the set detection defects, the model feature data is updated.

[0084] Specifically, this embodiment provides an implementation method for acquiring model feature data of a notebook based on an input signal.

[0085] Optionally, the convolutional neural network detection model is trained based on appearance defect sample data, specifically including: Based on the model training signal, obtaining appearance defect sample data of the notebook sample, the appearance defect sample data at least including large surface defect samples, end surface defect samples and bottom surface defect samples of the notebook sample; Preprocessing the appearance defect sample data, including any one or a combination of size standardization, grayscale, denoising, data enhancement and image scaling of the appearance defect sample data; When the appearance defect location data of the notebook sample identified by the appearance defect sample data is less than the set minimum value, the number of current defects at the appearance defect location is expanded based on the AIDG model; The convolutional neural network detection model is trained based on all appearance defect sample data.

[0086] Specifically, this embodiment provides an implementation method for training a convolutional neural network detection model.

[0087] Optionally, a convolutional neural network detection model is trained based on all appearance defect sample data, specifically including: Based on the model segmentation algorithm, the appearance defect sample data is segmented; Based on the segmented appearance defect sample data, transfer training is performed on the convolutional neural network detection model; Based on the back propagation algorithm, the network parameters of the convolutional neural network detection model are adjusted until the convolutional neural network detection model converges.

[0088] Specifically, this embodiment provides an implementation method for obtaining a convolutional neural network detection model based on training of all appearance defect sample data.

[0089] Optionally, a convolutional neural network detection model is obtained by training based on all appearance defect sample data, specifically including: Based on the segmented appearance defect sample data, the first sub-model of the convolutional neural network detection, the second sub-model of the convolutional neural network detection, the third sub-model of the convolutional neural network detection and the fourth sub-model of the convolutional neural network detection are trained. The training samples of the first sub-model of the convolutional neural network detection are the large surface line scan image data of the cropped notebook sample, the training samples of the second sub-model of the convolutional neural network detection are the large surface array image data of the notebook sample, the training samples of the third sub-model of the convolutional neural network detection are the end surface line scan image data after cropping and splicing, and the training samples of the fourth sub-model of the convolutional neural network detection are the bottom surface line scan image data of the notebook sample; Model transfer training is performed on the first sub-model of convolutional neural network detection, the second sub-model of convolutional neural network detection, the third sub-model of convolutional neural network detection and the fourth sub-model of convolutional neural network detection to obtain a convolutional neural network detection model.

[0090] Specifically, this embodiment provides another implementation method for training a convolutional neural network detection model based on all appearance defect sample data.

[0091] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the notebook appearance defect recognition method based on the convolutional neural network.

[0092] It should be noted that the electronic device in this embodiment can be a server, a PC, or other devices in specific implementation, as long as its structure includes the following: Figure 3 The processor 810, communication interface 820, memory 830 and communication bus 840 shown in the figure, wherein the processor 810, communication interface 820, memory 830 communicate with each other through the communication bus 840, and the processor 810 can call the logic instructions in the memory 830 to execute the above method. This embodiment does not limit the specific implementation form of the electronic device.

[0093] The server may be a single server or a server group. The server group may be centralized or distributed (for example, the server may be a distributed system).

[0094] In a possible embodiment, the server can be local or remote relative to the terminal. For example, the server can access information stored in the user terminal, database or any combination thereof via a network.

[0095] As another example, the server may be directly connected to at least one of the user terminal and the database to access information and / or data stored therein.

[0096] In a possible embodiment, the server may be implemented on a cloud platform; by way of example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, etc., or any combination thereof.

[0097] In a possible embodiment, the server and the user terminal may be implemented on an electronic device having one or more components in the embodiments of the present invention.

[0098] Furthermore, the network may be used for the exchange of information and / or data.

[0099] In a possible embodiment, one or more components in the interactive scenario (eg, a server, a user terminal, and a database) may send information and / or data to other components.

[0100] In possible embodiments, the network may be any type of wired or wireless network, or a combination thereof. As examples only, the network may include a wired network, a wireless network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, or a near field communication (NFC) network, or any combination thereof.

[0101] In a possible embodiment, the network may include one or more network access points. For example, the network may include a wired or wireless network access point, such as a base station and / or a network switching node, through which one or more components of the interactive scene may connect to the network to exchange data and / or information.

[0102] In addition, the logic instructions in the memory 830 described above may be implemented in the form of software functional units and when sold or used as independent products, may be stored in a computer-readable storage medium.

[0103] Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., various media that can store program codes.

[0104] In a possible embodiment, an embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the notebook appearance defect recognition method based on a convolutional neural network provided in the above embodiments.

[0105] In a possible embodiment, the embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments.

[0106] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0107] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on such an understanding, the above technical solution can essentially or in other words be embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A notebook appearance defect recognition algorithm based on convolutional neural network, characterized in that: include: Based on the defect recognition signal, acquiring appearance feature data of the notebook, the appearance feature data at least including large surface data, end surface data and bottom surface data of the notebook; The appearance data features are obtained and input into a convolutional neural network detection model to obtain detection data, wherein the convolutional neural network detection model is trained based on appearance defect sample data; A detection result of the notebook is determined based on the detection data.

2. The notebook appearance defect recognition algorithm based on convolutional neural network according to claim 1 is characterized in that: The obtaining of the appearance feature data of the notebook specifically includes: Acquire an appearance inspection area of ​​the notebook, and extract regional morphological features of the appearance inspection area, wherein the appearance inspection area at least includes a large surface area, an end surface area, and a bottom surface area of ​​the notebook, and the regional morphological features at least include any one or a combination of particles, paint accumulation, scratches, dents, and over-abrasion in the large surface area, the end surface area, and the bottom surface area; Based on the appearance detection area and the regional morphological features, large surface data of the large surface area, end surface data of the end surface area, and bottom surface data of the bottom surface area are respectively extracted.

3. The notebook appearance defect recognition algorithm based on convolutional neural network according to claim 1 is characterized in that: After obtaining the appearance data features and inputting them into the convolutional neural network detection model to obtain the detection data, the method specifically includes: Based on the detection data, determining a plurality of defect target locations of the notebook, wherein the defect target locations at least include defects on a large surface area, an end surface area, and a bottom surface area of ​​the notebook; Based on the plurality of defective target locations, overlapping areas of the plurality of defective target locations are determined, and redundant overlapping areas are removed from all the defective target locations.

4. The notebook appearance defect recognition algorithm based on convolutional neural network according to claim 3 is characterized in that: After removing the redundant overlapping areas in all the defect target positions, the method specifically includes: Adding a maximum external detection frame to each defect target position after removing the overlapping area; The detection result is generated based on all the defective target positions with the maximum external detection frame added.

5. The notebook appearance defect recognition algorithm based on convolutional neural network according to any one of claims 1 to 4, characterized in that: Before obtaining the appearance feature data of the notebook based on the recognition signal, the method specifically includes: Based on an input signal, acquiring model feature data of the notebook, wherein the input operation corresponding to the input signal at least includes inputting part of the model feature data of the notebook; Obtaining a classification result of the model feature data input into a convolutional neural network strategy model, wherein the convolutional neural network strategy model is trained based on the model feature sample data of the notebook; A defect detection strategy for the notebook is determined based on the classification result, and the defect detection strategy is transmitted to a convolutional neural network detection model.

6. The notebook appearance defect recognition algorithm based on convolutional neural network according to claim 5 is characterized in that: The step of obtaining the model characteristic data of the notebook based on the input signal specifically includes: Acquire the set detection defects of the notebook identified by the input operation, wherein the set detection defects include any one or a combination of length defects, area defects, width defects, particle defects, paint accumulation defects, scratch defects, hole defects and over-wear defects of the notebook; Defects are detected based on the settings, and the model feature data is updated.

7. The notebook appearance defect recognition algorithm based on convolutional neural network according to any one of claims 1 to 4, characterized in that: The convolutional neural network detection model is trained based on appearance defect sample data, specifically including: Based on the model training signal, obtaining appearance defect sample data of the notebook sample, wherein the appearance defect sample data at least includes a large surface defect sample, an end surface defect sample, and a bottom surface defect sample of the notebook sample; Preprocessing the appearance defect sample data, wherein the preprocessing includes performing any one or a combination of size standardization, grayscale conversion, denoising, data enhancement, and image scaling on the appearance defect sample data; When the appearance defect position data of the notebook sample identified by the appearance defect sample data is less than a set minimum value, the number of current defects at the appearance defect position is expanded based on the AIDG model; The convolutional neural network detection model is obtained by training based on all the appearance defect sample data.

8. The notebook appearance defect recognition algorithm based on convolutional neural network according to claim 7 is characterized in that: The convolutional neural network detection model is obtained by training based on all the appearance defect sample data, specifically including: Segmenting the appearance defect sample data based on a model segmentation algorithm; Based on the segmented appearance defect sample data, performing migration training on the convolutional neural network detection model; Based on the back propagation algorithm, the network parameters of the convolutional neural network detection model are adjusted until the convolutional neural network detection model converges.

9. The notebook appearance defect recognition algorithm based on convolutional neural network according to claim 8 is characterized in that: The convolutional neural network detection model is obtained by training based on all the appearance defect sample data, and specifically includes: Based on the segmented appearance defect sample data, a convolutional neural network is trained to detect a first sub-model, a second sub-model, a third sub-model and a fourth sub-model, wherein the training sample of the convolutional neural network for detecting the first sub-model is the cropped large surface line scan image data of the notebook sample, the training sample of the convolutional neural network for detecting the second sub-model is the large surface array image data of the notebook sample, the training sample of the convolutional neural network for detecting the third sub-model is the end surface line scan image data of the notebook sample after cropping and splicing, and the training sample of the convolutional neural network for detecting the fourth sub-model is the bottom surface line scan image data of the notebook sample; Model migration training is performed on the first sub-model of the convolutional neural network detection, the second sub-model of the convolutional neural network detection, the third sub-model of the convolutional neural network detection and the fourth sub-model of the convolutional neural network detection to obtain the convolutional neural network detection model.

10. A notebook appearance defect recognition device based on convolutional neural network, characterized in that: include: A feature acquisition module (10) is used to acquire appearance feature data of the notebook based on the defect recognition signal, wherein the appearance feature data at least includes large surface data, end surface data and bottom surface data of the notebook; A data detection module (20), used for acquiring the appearance data features and inputting them into a convolutional neural network detection model to obtain detection data, wherein the convolutional neural network detection model is trained based on appearance defect sample data; A result determination module (30) is used to determine a detection result of the notebook based on the detection data.

Images