A method, device and electronic device for identifying dust screen defects
Through the automated identification method of dustproof net defects in mechanical factories, the combination of preprocessing and neural network models is used to solve the problems of accuracy and efficiency caused by manual classification, and automatic detection of high accuracy and stability is achieved.
Patent Information
- Application Number
- CN202010769188.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-03
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-08-03
AI Technical Summary
The classification of dustproof net defects in mechanical factories still relies on manual operation, resulting in workers' mental fatigue, reduced accuracy, low efficiency, and unstable production. Dustproof net detection automation is needed to reduce production costs and improve accuracy.
The target image is randomly rotated, mirrored, contrasted and brighter by using preprocessing technology to generate simulated images, and input them into the target image with the neural network model. The recognition base of multiple simulated images is increased, the probability of error is reduced, and dustproof is determined. Is there any flaws in the network?
It realizes automated identification, improves the accuracy and efficiency of dustproof net defect detection, reduces manual errors, and ensures the stability and reliability of production.
Smart Images

Figure CN111862089B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of images, and more specifically, to a method, device and electronic device for identifying dust screen defects. Background Art
[0002] At present, it is imperative to fully realize production line automation in mechanical factories, and production line automation has also become a current research hotspot. For example, during the production process of mobile phones, it is necessary to classify the defects of dust screens. The mobile phone cases with defective dust screens will be reworked, and those without problems will proceed to the next step of production. At present, this work is still classified manually, and workers will encounter many problems when performing a large amount of mechanical repetitive work, such as mental fatigue leading to reduced classification accuracy and efficiency, and different workers have a lot of autonomy in classification, and the stability and reliability of production are low.
[0003] How to automate the dust screen inspection and classify the mobile phone cases according to the prescribed procedures without human intervention, freeing workers from heavy and mechanical physical labor, saving labor and reducing production costs, thus avoiding the problems of low accuracy, low efficiency and unstable production in manual inspection, has become a problem that must be solved now. Summary of the invention
[0004] The purpose of this application is to provide a dust screen defect identification method, device and electronic equipment to solve the above-mentioned problems.
[0005] In order to achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:
[0006] In a first aspect, an embodiment of the present application provides a method for identifying dust screen defects, the method comprising:
[0007] Preprocessing the target image to generate a simulated image, wherein the preprocessing includes random rotation processing, random mirror processing, contrast processing and brightness processing, and the simulated image is an image of the target image after rotation or mirroring or after adjusting the contrast or brightness;
[0008] Using the target image and the simulated image as inputs of a neural network model;
[0009] When the number of the output of the neural network model is a first result that is greater than a predetermined threshold, it is determined that the dust screen in the target image has defects, wherein the first result is a defective recognition result corresponding to the target image or the simulated image.
[0010] In a second aspect, an embodiment of the present application provides a dust screen defect identification device, the device comprising:
[0011] A preprocessing unit, used for preprocessing the target image to generate a simulated image, wherein the preprocessing includes random rotation processing, random mirror processing, contrast processing and brightness processing, and the simulated image is an image of the target image after rotation or mirroring or after adjusting the contrast or brightness;
[0012] An identification unit is used to use the target image and the simulated image as inputs of a neural network model; and is also used to determine that the dust screen in the target image has defects when the number of first results output by the neural network model is greater than a predetermined threshold, wherein the first result is a defective identification result corresponding to the target image or the simulated image.
[0013] In a third aspect, an embodiment of the present application provides a storage medium on which a computer program is stored, and the computer program implements the above method when executed by a processor.
[0014] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, wherein the memory is used to store one or more programs; when the one or more programs are executed by the processor, the above method is implemented.
[0015] Compared with the prior art, the beneficial effects of a dust screen defect recognition method, device and electronic device provided in the embodiments of the present application are as follows: first, the target image is preprocessed to generate a simulated image; then the target image and the simulated image are used as inputs of the neural network model, and multiple simulated images containing the same object to be recognized are recognized to increase the recognition cardinality and reduce the probability of recognition errors; when the output of the neural network model is that the number of first results is greater than a predetermined threshold, it is determined that the dust screen in the target image has defects, and the first result is the defective recognition result corresponding to the target image or the simulated image. Among them, the simulated image is an image of the target image after rotation or mirroring or after adjusting the contrast or brightness, so as to achieve the purpose of enriching the number of recognition samples; by recognizing multiple simulated images containing the same object to be recognized, the recognition cardinality is increased, the probability of recognition errors is reduced, and the accuracy of the model recognition results is guaranteed.
[0016] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0018] Figure 1 A schematic diagram of the structure of an electronic device provided by an embodiment;
[0019] Figure 2 A schematic diagram of a flow chart of a dust screen defect identification method provided by an embodiment;
[0020] Figure 3 A schematic flow chart of a method for identifying dust screen defects provided in another embodiment;
[0021] Figure 4 A schematic diagram of sub-steps of S102 provided in an embodiment;
[0022] Figure 5 A schematic diagram of the structure of a neural network model provided by an embodiment;
[0023] Figure 6 A schematic diagram of a unit of a dust screen defect identification device provided in an embodiment.
[0024] In the figure: 10 - processor; 11 - memory; 12 - bus; 13 - communication interface; 201 - pre-processing unit; 202 - recognition unit; 203 - training unit. DETAILED DESCRIPTION
[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.
[0026] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.
[0027] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0028] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0029] In the description of the present application, it should be noted that the terms "upper", "lower", "inside", "outside", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the product of the application is usually placed when in use. They are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.
[0030] In the description of this application, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "disposed" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.
[0031] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0032] The present application embodiment provides an electronic device, which may be a computer device. Figure 1, a schematic diagram of the structure of an electronic device. The electronic device includes a processor 10, a memory 11, and a bus 12. The processor 10 and the memory 11 are connected via the bus 12, and the processor 10 is used to execute an executable module stored in the memory 11, such as a computer program.
[0033] The processor 10 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the dust screen defect identification method can be completed by the hardware integrated logic circuit in the processor 10 or the instructions in the form of software. The above-mentioned processor 10 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
[0034] The memory 11 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.
[0035] The bus 12 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. Figure 1 Although only one bidirectional arrow is used in the figure, it does not mean that there is only one bus 12 or only one type of bus 12 .
[0036] The memory 11 is used to store programs, such as programs corresponding to the dust screen defect recognition device. The dust screen defect recognition device includes at least one software function module that can be stored in the memory 11 in the form of software or firmware or fixed in the operating system (OS) of the electronic device. After receiving the execution instruction, the processor 10 executes the program to implement the dust screen defect recognition method.
[0037] Possibly, the electronic device provided in the embodiment of the present application further includes a communication interface 13. The communication interface 13 is connected to the processor 10 via a bus. The electronic device can receive picture information and the like transmitted by other terminals or components via the communication interface 13.
[0038] It should be understood that Figure 1 The structure shown is only a schematic diagram of a portion of the electronic device. The electronic device may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.
[0039] The dust screen defect recognition method provided by the embodiment of the present invention can be applied to but not limited to Figure 1 For detailed procedures, please refer to the electronic equipment shown in Figure 2 , the dust screen defect identification methods include:
[0040] S106, preprocessing the target image to generate a simulated image.
[0041] The preprocessing includes random rotation processing, random mirror processing, contrast processing and brightness processing, and the simulated image is an image of the target image after rotation or mirroring or after adjusting the contrast or brightness.
[0042] Specifically, the simulated images all contain the object to be identified in the target image, namely, the dust screen.
[0043] S107, using the target image and the simulated image as inputs of the neural network model.
[0044] Specifically, the neural network model has a certain error when identifying whether the object to be identified in the picture has defects. In order to reduce the error, the target picture is preprocessed to obtain multiple simulated pictures containing the same object to be identified. By increasing the cardinality of identification, the probability of identification error is reduced.
[0045] S108, determining whether the number of the first results output by the neural network model is greater than a predetermined threshold. If so, executing S109; if not, executing S110.
[0046] The first result is a defective recognition result corresponding to the target image or the simulated image.
[0047] Specifically, when the number of the first results output by the neural network model is greater than a predetermined threshold, it indicates that the dust screen in the target image has defects, and S109 is executed. When the number of the first results output by the neural network model is less than or equal to the predetermined threshold, it indicates that the dust screen in the target image has no defects, and S110 is executed. It should be noted that the predetermined threshold is related to the total number of simulated images.
[0048] S109, determining that the dust screen in the target image has defects.
[0049] S110, determining that the dust screen in the target image has no defects.
[0050] In summary, in the dust screen defect recognition method provided in the embodiment of the present application, first, the target image is preprocessed to generate a simulated image; then the target image and the simulated image are used as inputs of the neural network model, and multiple simulated images containing the same object to be recognized are recognized to increase the recognition cardinality and reduce the probability of recognition errors; when the output of the neural network model is that the number of first results is greater than a predetermined threshold, it is determined that the dust screen in the target image has defects, and the first result is the defective recognition result corresponding to the target image or the simulated image. Among them, the simulated image is an image of the target image after rotation or mirroring or after adjusting the contrast or brightness, so as to achieve the purpose of enriching the number of recognition samples; by recognizing multiple simulated images containing the same object to be recognized, the recognition cardinality is increased, the probability of recognition errors is reduced, and the accuracy of the model recognition results is guaranteed.
[0051] exist Figure 2 On the basis of how to train the neural network model, the present application embodiment also provides a possible implementation method, please refer to Figure 3 ,The dust screen defect identification method also includes:
[0052] S101, obtain sample images and corresponding supervision images.
[0053] Among them, both the sample images and the supervision images are labeled, and the labels indicate whether the dust screen in the corresponding image has defects.
[0054] Possibly, the sample images include a first type of training images and a second type of training images, wherein the first type of training images are initially acquired images including dust screens, and the first type of training images are preprocessed to generate the second type of training images.
[0055] The preprocessing includes random rotation processing, random mirror processing, contrast processing and brightness processing. The second type of training pictures are images of the first type of training pictures after rotation or mirroring or after adjusting the contrast or brightness.
[0056] Specifically, the richness of the training samples determines the accuracy of the model recognition results to a certain extent. Often, the richer the training samples, the higher the accuracy of the recognition results of the neural network model that is finally trained. In reality, the number of sample images (first-class training images) collected is limited. The first-class training images are preprocessed to generate the second-class training images, which are images of the first-class training images that are rotated or mirrored or adjusted in contrast or adjusted in brightness. Both contain images corresponding to the dust net, thereby achieving the purpose of enriching the number of samples.
[0057] Possibly, the dust screen pictures are randomly divided into the first type of training pictures and supervision pictures according to a preset ratio, wherein the preset ratio may be 8:2.
[0058] S102, using the sample image as an input of the neural network model to train the neural network model until the neural network model converges.
[0059] Specifically, the first type of training pictures and the second type of training pictures are used as the input of the neural network model, which improves the sample pictures for training the neural network model. The neural network model is trained using the gradient descent optimization algorithm, which can effectively improve the training efficiency of the neural network model. When the neural network model converges, it means that the training is completed, and the current dust screen defect recognition model can be used to identify dust screen defects.
[0060] S103, using the supervised image as the input of the converged neural network model to perform supervised testing on the neural network model.
[0061] Specifically, the accuracy of the output results of the neural network model is verified by using the supervision images as the input of the neural network model. It should be noted that the supervision images are labeled. For example, when the dust screen in a supervision image has defects, the label of the supervision image is 1; when the dust screen in a supervision image has no defects, the label of the supervision image is 0. The content and form of the labels are not limited here. The examples are only for ease of understanding.
[0062] S104, determine whether the accuracy of the test output result is less than the test threshold. If yes, execute S102; if no, execute S105.
[0063] By comparing the output result with the label of the corresponding supervision image, the accuracy of the output result can be obtained. When the accuracy of the output result is greater than or equal to the preset test threshold, it means that the dust screen defect recognition model has a good recognition effect. At this time, the training can be ended and S105 is executed. When the accuracy of the output result is less than the test threshold, it means that the dust screen defect recognition model still has major defects and needs to be repeatedly trained. At this time, S102 is executed to repeatedly train the neural network model.
[0064] S105, end the training.
[0065] Specifically, the training is completed and the neural network model is saved.
[0066] exist Figure 3 On this basis, regarding how to obtain sample images and target images, the embodiment of the present application also provides a possible implementation method, please refer to the following.
[0067] Crop the device image to obtain the dust filter image.
[0068] Specifically, when collecting images, because the shooting range is very wide, the device parts and environmental parts other than the dust net will be captured. The image content other than the dust net may interfere with the training and testing of the neural network model. In order to reduce interference, improve training efficiency and the accuracy of test results, the device image can be cropped and the image content other than the dust net can be deleted, so as to accurately obtain the dust net image. When the dust net image is labeled, it is a sample image; when the dust net image is directly used for recognition, it is a target image.
[0069] Possibly, the dust screen part can be accurately cropped out from the device image by using grayscale, binarization and contour detection algorithms.
[0070] exist Figure 3 Based on the content in S102, the present application embodiment also provides a possible implementation method, please refer to Figure 4 , S102 includes:
[0071] S102-1, using the sample image as the input of the neural network model, and using the gradient descent optimization algorithm to train the neural network model.
[0072] Specifically, the training speed is improved by using a gradient descent optimization algorithm to train the neural network model.
[0073] S102-2, calculate the loss function of the neural network model.
[0074] The loss function may be a cross entropy loss function. It should be noted that the sample image carries a label indicating whether the dust screen has defects. Thus, the loss function can be calculated in combination with the training results. The loss function represents the error level of the recognition result of the neural network model.
[0075] S102-3, when the loss function is less than a preset loss threshold, the neural network model is considered to have converged.
[0076] Specifically, when the loss function is less than a preset loss threshold, it indicates that the neural network model has stably converged, and the neural network model is considered to have converged, and the training is stopped, and S103 can be executed.
[0077] exist Figure 2 On the basis of how to construct a neural network model, the embodiment of the present application also provides a possible implementation method, and the dustproof net defect recognition method also includes:
[0078] Construct a neural network model based on parameters, where the parameters include learning force, momentum, and weight decay.
[0079] Among them, the learning rate (lr) refers to the magnitude of the update of the network weights in the optimization algorithm. The learning rate can be constant, gradually decreasing, momentum-based or adaptive. Different optimization algorithms determine different learning rates. When the learning rate is too large, the model may not converge, and the loss loss will continue to fluctuate up and down; if the learning rate is too small, the model converges slowly and requires longer training time. In the embodiment of this application, the learning rate is set to 0.01.
[0080] Momentum comes from Newton's law. The basic idea is to find the optimal effect of adding "inertia". When there is a flat area in the error surface, the gradient descent method (SGD) can learn faster. In the embodiment of the present application, the momentum is set to 0.9.
[0081] Weight decay, i.e. L2 regularization, can avoid model overfitting to a certain extent. In the embodiment of the present application, the weight decay is set to 0.0005.
[0082] Possibly, the present application embodiment provides a possible neural network model structure, please refer to Figure 5The neural network model includes: 3D comprehensive layer [abbreviated as Conv(3*3) / BN / Relu], 5D comprehensive layer [abbreviated as Conv(5*5) / BN / Relu], concatenation layer (abbreviated as Concat), maximum pooling layer [abbreviated as MaxPool(2*2)], 9D comprehensive layer [abbreviated as Conv(9*9) / BN / Relu], 1D comprehensive layer [abbreviated as Conv(1*1) / BN], flattening layer (abbreviated as Reduce_mean / Squeeze) and fully connected layer (abbreviated as FullConnect).
[0083] The 3D synthesis layer includes a 3*3 convolution layer, a batch normalization layer, and an activation layer;
[0084] The 5D comprehensive layer includes a 5*5 convolutional layer, a batch normalization layer, and an activation layer;
[0085] The 9D comprehensive layer includes a 9*9 convolutional layer, a batch normalization layer, and an activation layer;
[0086] The 1D comprehensive layer includes a 1*1 convolution layer and a batch normalization layer.
[0087] Please continue to refer to Figure 5 The neural network model includes a first branch, a second branch, a third branch and a fourth branch, and the first branch, the second branch, the third branch and the fourth branch are connected in sequence.
[0088] The first branch includes the first batch of 3D synthesis layers, the first batch of 5D synthesis layers, the first batch of splicing layers, and the first batch of maximum pooling layers.
[0089] The second branch includes two second batches of 3D synthesis layers, a second batch of 5D synthesis layers, a second batch of splicing layers, and a second batch of maximum pooling layers.
[0090] The third branch includes four third batches of 3D integration layers, a third batch of splicing layers, and a third batch of maximum pooling layers.
[0091] The fourth branch includes a 9D synthesis layer, a 1D synthesis layer, a fourth batch of maximum pooling layers, a flattening layer, and a fully connected layer.
[0092] Please continue to refer to Figure 5 , Input is used to input data, and its data dimension can be (n*178*178*3), where n represents the number of input images and 178*178 represents the size of the input image.
[0093] After the input is input, the first batch of 3D synthesis layers are connected, and the convolution layer with 3*3 convolution kernel, batch normalization, and ReLu activation function processing are performed. The convolution layer extracts features and normalization is performed after convolution to make the features converge. The activation function adds nonlinear factors to enrich the dimension of the features.
[0094] The first batch of 3D integrated layers are connected to the first batch of 5D integrated layers and the first batch of splicing layers respectively.
[0095] The first batch of 5D synthesis layers are used to process the outputs of the first batch of 3D synthesis layers through a 5*5 convolution kernel convolution layer, batch normalization, and ReLu activation function.
[0096] The first batch of 5D synthesis layers are also connected to the first batch of stitching layers.
[0097] The first batch of concatenation layers is used to concatenate the output of the first batch of 5D synthesis layers with the output of the first batch of 3D synthesis layers. Through tensor concatenation, the convolution features of the previous layer can be obtained, making the features more comprehensive and improving the network accuracy; secondly, the number of convolution kernels can be reduced, the redundancy of the network can be reduced, and the network operation speed can be improved without reducing the accuracy.
[0098] The first batch of concatenation layers are connected to the first batch of maximum pooling layers. The stride of the maximum pooling layer is 2. The maximum pooling layer can reduce the deviation of the estimated mean caused by the error of the convolution layer parameters and retain more texture information.
[0099] Convolution kernels of different sizes can obtain different receptive fields, which helps improve accuracy. However, a convolution kernel that is too large will lead to greater computational complexity. Therefore, choosing 3 and 5 is more appropriate.
[0100] The first batch of max pooling layers are connected with the second branch.
[0101] After the first batch of maximum pooling layers, two second batch 3D comprehensive layer structures are connected in series. [Conv(3*3) / BN / Relu]*2 represents two second batch 3D comprehensive layer structures connected in series.
[0102] The structures of the two second batch 3D comprehensive layers are connected in series, and the outputs of the first batch of maximum pooling layers are respectively connected to the structures of the two second batch 3D comprehensive layers and the second batch of splicing layers.
[0103] The structures of the two second batches of 3D integrated layers are connected in series, and the second batches of 5D integrated layers and the second batches of splicing layers are connected in sequence.
[0104] The second batch of concatenation layers is used to concatenate the outputs of the second batch of 5D synthesis layers and the outputs of the first batch of maximum pooling layers.
[0105] The second batch of concatenation layers is connected to the second batch of max pooling layers.
[0106] The second batch of max pooling layers is connected to the third branch.
[0107] The second batch of maximum pooling layers is connected to the fourth third batch of 3D integration layer structures in series and the third batch of concatenation layers.
[0108] The third batch of concatenation layers is used to concatenate the outputs of the four third batch 3D comprehensive layer structures in series and the outputs of the second batch of maximum pooling layers.
[0109] The third batch of concatenation layers are connected to the third batch of maximum pooling layers to perform 2*2 maximum pooling.
[0110] The third batch of max pooling layers is connected to the 9D synthesis layer.
[0111] Since the previous network uses a stack of smaller convolution kernels and the number of convolution kernels is also small, the network complexity is very low. By using a 9*9 convolution kernel, the receptive field is expanded, the network's ability to represent features is enhanced, and the accuracy is further improved.
[0112] The 9D comprehensive layer is connected with the 1D comprehensive layer and the fourth batch of maximum pooling layers in sequence to perform dimensionality reduction.
[0113] The fourth batch of maximum pooling layers, flattening layers and fully connected layers follow.
[0114] It is used to calculate the average value of the tensor along the specified axis and remove the dimension with a value of 1. Finally, the fully connected layer is connected to output the prediction result (Output).
[0115] BN stands for batch normalization, Conv stands for convolutional layer, ReLu stands for ReLu activation layer, MaxPool stands for maximum pooling layer, Reduce_mean stands for calculating the average value of the tensor along the specified axis (a certain dimension of the tensor), Squeeze stands for deleting the dimension with a value of 1 in the tensor, FullConnect stands for fully connected layer, and Concat stands for tensor concatenation.
[0116] The neural network model structure in the embodiment of the present application is simple, easy to understand and suitable for the task of identifying dustproof net defects. The batch normalization layer is used multiple times in the network to greatly improve the training speed and improve the gradient flow of the entire network. In addition, the features of the deep convolutional layer of the network are spliced with the features of the previous convolutional layer and then input into the next network layer, that is, a higher accuracy can be achieved with fewer convolution kernels, which means fewer training parameters and faster running speed. After multiple training and debugging, the final network structure is determined. The recognition accuracy during training can reach 99%, and the test recognition accuracy is about 97%. And based on Intel Corei7-7800X CPU @ 3.5GHz * 12, the running speed of each picture is about 30ms, which can fully meet the real-time requirements, and the recognition accuracy is also relatively high. And the model can also be accelerated through openvino, which can meet the needs of real-time detection in the factory.
[0117] Possibly, the dust screen defect recognition method in the embodiment of the present application can also be used for training models such as glue overflow defect classification, sound hole defect classification, and shell defect classification.
[0118] See also Figure 6 , Figure 6 A dust screen defect recognition device is provided in an embodiment of the present application. Optionally, the dust screen defect recognition device is applied to the electronic device described above.
[0119] The dust screen defect recognition device includes: a pre-processing unit 201 and a recognition unit 202 .
[0120] The preprocessing unit 201 is used to preprocess the target image to generate a simulated image, wherein the preprocessing includes random rotation processing, random mirror processing, contrast processing and brightness processing, and the simulated image is an image of the target image after rotation or mirroring or after adjusting the contrast or brightness. Specifically, the preprocessing unit 201 can perform the above S106.
[0121] The recognition unit 202 is used to use the target image and the simulated image as inputs of the neural network model; and is also used to determine that the dust screen in the target image has defects when the number of the outputs of the neural network model is a first result greater than a predetermined threshold, wherein the first result is a defective recognition result corresponding to the target image or the simulated image. Specifically, the recognition unit 202 can execute the above S107 to S110.
[0122] Furthermore, the dust screen defect recognition device further includes: a training unit 203 .
[0123] The preprocessing unit 201 is further used to obtain a sample image and a corresponding supervision image, wherein both the sample image and the supervision image are labeled, and the label indicates whether the dust screen in the corresponding image has defects. Specifically, the preprocessing unit 201 can execute the above S101.
[0124] The training unit 203 is used to use the sample image as the input of the neural network model to train the neural network model until the neural network model converges; use the supervision image as the input of the converged neural network model to perform a supervision test on the neural network model; when the accuracy of the test output result is less than the test threshold, repeatedly use the sample image as the input of the neural network model to train the neural network model until the neural network model converges; when the accuracy of the test output result is greater than or equal to the test threshold, end the training. Specifically, the training unit 203 can execute the above S102 to S105.
[0125] Furthermore, the training unit 203 is specifically used to use the sample image as the input of the neural network model, and use the gradient descent optimization algorithm to train the neural network model; calculate the loss function of the neural network model, wherein the loss function represents the error level of the recognition result of the neural network model; when the loss function is less than a preset loss threshold, the neural network model is considered to have converged. Specifically, the training unit 203 can execute the above S102-1 to S102-3.
[0126] It should be noted that the dust screen defect identification device provided in this embodiment can execute the method flow shown in the above method flow embodiment to achieve the corresponding technical effect. For the sake of brief description, for the parts not mentioned in this embodiment, please refer to the corresponding contents in the above embodiment.
[0127] The embodiment of the present invention further provides a storage medium, which stores computer instructions and programs, which, when read and executed, execute the dust screen defect identification method of the above embodiment. The storage medium may include a memory, a flash memory, a register, or a combination thereof.
[0128] An electronic device is provided below, which may be a computer device. Figure 1 As shown, the above-mentioned dust screen defect recognition method can be implemented; specifically, the electronic device includes: a processor 10, a memory 11, and a bus 12. The processor 10 can be a CPU. The memory 11 is used to store one or more programs. When the one or more programs are executed by the processor 10, the dust screen defect recognition method of the above-mentioned embodiment is executed.
[0129] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0130] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
[0131] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0132] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
[0133] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present application. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.
Claims
1. A method for identifying dust screen defects, characterized in that: The method comprises: Preprocessing the target image to generate a simulated image, wherein the preprocessing includes random rotation processing, random mirror processing, contrast processing and brightness processing, and the simulated image is an image of the target image after rotation or mirroring or after adjusting the contrast or brightness; Using the target image and the simulated image as inputs of a neural network model; When the number of the output of the neural network model is a first result that is greater than a predetermined threshold, it is determined that the dust screen in the target image has a defect, wherein the first result is a defective recognition result corresponding to the target image or the simulated image; The neural network model includes: a 3D synthesis layer, a 5D synthesis layer, a splicing layer, a maximum pooling layer, a 9D synthesis layer, a 1D synthesis layer, a flattening layer and a fully connected layer; The 3D synthesis layer includes a 3*3 convolution layer, a batch normalization layer and an activation layer; The 5D comprehensive layer includes a 5*5 convolution layer, a batch normalization layer and an activation layer; The 9D comprehensive layer includes a 9*9 convolutional layer, a batch normalization layer and an activation layer; The 1D comprehensive layer includes a 1*1 convolution layer and a batch normalization layer; The neural network model includes a first branch, a second branch, a third branch and a fourth branch, and the first branch, the second branch, the third branch and the fourth branch are connected in sequence; The first branch includes the first batch of 3D synthesis layers, the first batch of 5D synthesis layers, the first batch of splicing layers and the first batch of maximum pooling layers; The second branch includes two second batches of 3D synthesis layers, a second batch of 5D synthesis layers, a second batch of splicing layers, and a second batch of maximum pooling layers; The third branch includes four third batches of 3D synthesis layers, a third batch of splicing layers, and a third batch of maximum pooling layers; The fourth branch includes the 9D comprehensive layer, the 1D comprehensive layer, the fourth batch of maximum pooling layers, the flattening layer and the fully connected layer; The first branch is connected to the second branch; The first batch of 3D integrated layers are respectively connected with the first batch of 5D integrated layers and the first batch of splicing layers; The first batch of 5D comprehensive layers is used to process the output of the first batch of 3D comprehensive layers through a convolution layer with a 5*5 convolution kernel, batch normalization, and ReLu activation function; The first batch of 5D integrated layers are connected to the first batch of splicing layers; The first batch of splicing layers are used to perform tensor splicing on the outputs of the first batch of 5D synthesis layers and the outputs of the first batch of 3D synthesis layers; The first batch of concatenation layers are connected to the first batch of maximum pooling layers; The first batch of maximum pooling layers are connected to the second branch.
2. The dust screen defect identification method according to claim 1, characterized in that: The method further comprises: Obtain a sample image and a corresponding supervision image, wherein both the sample image and the supervision image are labeled, and the label indicates whether the dust screen in the corresponding image has defects; Using the sample image as an input of a neural network model to train the neural network model until the neural network model converges; Using the supervised image as input of the converged neural network model to perform supervised testing on the neural network model; When the accuracy of the test output result is less than the test threshold, repeatedly using the sample image as the input of the neural network model to train the neural network model until the neural network model converges; When the accuracy of the test output result is greater than or equal to the test threshold, the training ends.
3. The dust screen defect identification method according to claim 2, characterized in that: The step of using the sample image as an input of the neural network model to train the neural network model until the neural network model converges includes: Using the sample image as input of the neural network model, and training the neural network model using a gradient descent optimization algorithm; Calculating a loss function of the neural network model, wherein the loss function represents the level of error in the recognition result of the neural network model; When the loss function is less than a preset loss threshold, the neural network model is considered to have converged.
4. The dust screen defect identification method according to claim 1, characterized in that: The second branch is connected to the third branch; The structures of the two second batches of 3D comprehensive layers are connected in series, and the outputs of the first batch of maximum pooling layers are respectively connected to the structures of the two second batches of 3D comprehensive layers and the second batch of splicing layers; The structures of the two second batches of 3D integrated layers are connected in series, and the second batch of 5D integrated layers and the second batch of splicing layers are connected in sequence; The second batch of concatenation layers are used to concatenate the outputs of the second batch of 5D comprehensive layers and the outputs of the first batch of maximum pooling layers; The second batch of concatenation layers are connected to the second batch of maximum pooling layers; The second batch of maximum pooling layers is connected to the third branch.
5. A dust screen defect identification device, characterized in that: The device comprises: A preprocessing unit, used for preprocessing the target image to generate a simulated image, wherein the preprocessing includes random rotation processing, random mirror processing, contrast processing and brightness processing, and the simulated image is an image of the target image after rotation or mirroring or after adjusting the contrast or brightness; an identification unit, configured to use the target image and the simulated image as inputs of a neural network model; and further configured to determine that the dust screen in the target image has defects when the number of first results output by the neural network model is greater than a predetermined threshold, wherein the first result is an identification result of defects corresponding to the target image or the simulated image; The neural network model includes: a 3D synthesis layer, a 5D synthesis layer, a splicing layer, a maximum pooling layer, a 9D synthesis layer, a 1D synthesis layer, a flattening layer and a fully connected layer; The 3D synthesis layer includes a 3*3 convolution layer, a batch normalization layer and an activation layer; The 5D comprehensive layer includes a 5*5 convolution layer, a batch normalization layer and an activation layer; The 9D comprehensive layer includes a 9*9 convolutional layer, a batch normalization layer and an activation layer; The 1D comprehensive layer includes a 1*1 convolution layer and a batch normalization layer; The neural network model includes a first branch, a second branch, a third branch and a fourth branch, and the first branch, the second branch, the third branch and the fourth branch are connected in sequence; The first branch includes the first batch of 3D synthesis layers, the first batch of 5D synthesis layers, the first batch of splicing layers and the first batch of maximum pooling layers; The second branch includes two second batches of 3D synthesis layers, a second batch of 5D synthesis layers, a second batch of splicing layers, and a second batch of maximum pooling layers; The third branch includes four third batches of 3D synthesis layers, a third batch of splicing layers, and a third batch of maximum pooling layers; The fourth branch includes the 9D comprehensive layer, the 1D comprehensive layer, the fourth batch of maximum pooling layers, the flattening layer and the fully connected layer; The first branch is connected to the second branch; The first batch of 3D integrated layers are respectively connected with the first batch of 5D integrated layers and the first batch of splicing layers; The first batch of 5D comprehensive layers is used to process the output of the first batch of 3D comprehensive layers through a convolution layer with a 5*5 convolution kernel, batch normalization, and ReLu activation function; The first batch of 5D integrated layers are connected to the first batch of splicing layers; The first batch of splicing layers are used to perform tensor splicing on the outputs of the first batch of 5D synthesis layers and the outputs of the first batch of 3D synthesis layers; The first batch of concatenation layers are connected to the first batch of maximum pooling layers; The first batch of maximum pooling layers are connected to the second branch.
6. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.
7. An electronic device, characterized in that: include: A processor and a memory, the memory being used to store one or more programs; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 4 is implemented.
Citation Information
Cited By
Building construction green dustproof net identification method and system, computer equipment and readable storage medium
CN122157003A
