Method, device, equipment and readable storage medium for detecting key paint uniformity

Abstract

The present invention discloses a method, apparatus, device, and readable storage medium for detecting the uniformity of key paint spraying, belonging to the field of machine vision technology. The method comprises collecting images of the key to be inspected from different viewing angles; inputting the images of the key to be inspected from different viewing angles into a key paint detection neural network model, sequentially performing key view classification and key paint defect classification to obtain paint defect classification results for each key view; and outputting key paint uniformity detection results based on the paint defect classification results for each key view. The present invention can detect the uniformity of key paint spraying with high overall efficiency and accurate defect classification.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision technology, and more specifically, relates to a method, device, equipment and readable storage medium for detecting the uniformity of key paint spraying. Background Art

[0002] Computer power buttons are exterior components. To achieve aesthetic appeal and meet market demand, they need to be painted, often using a paint jig. Existing button paint uniformity testing relies primarily on visual inspection, which is subjective and difficult to distinguish between unclear painted areas. Furthermore, if workers work for a certain period of time, visual fatigue can easily occur, affecting detection stability and making missed inspections more likely.

[0003] Corresponding improvements have also been made to the above-mentioned problems. For example, Chinese patent application number CN202010371104.9, published on September 1, 2020, discloses a method and system for rapid detection of cracks on product surfaces, including: step S1, classifying the collected high-resolution images, and using an improved sliding window to manually amplify the data set after cropping the collected original images; step S2, using a Mask R-CNN network with good feature extraction effect for calculation and processing; constructing a product surface crack data set, and then constructing a data set to train a Mask R-CNN model so that the model can identify cracks on the product surface; step S4, using a thresholding algorithm to solve the global segmentation threshold of the product surface crack image, and accurately extract the crack features in the crack surface element; and step S5, arranging the pixel points in the crack, dividing them into the upper left corner and lower right corner coordinates, and marking the rectangular area determined by the two coordinate points as a bounding box to mark the position of the crack on the product surface. The method, when faced with product surface cracks, expands them through image processing algorithms (such as horizontal flipping, vertical flipping, linear changes, and spatial filtering), and divides them into several categories, such as images containing obstacles (such as stains, etc.) and simple background images, and uses the above algorithms to expand the data set. A product surface crack image segmentation model based on semantic segmentation is used to extract features from high-resolution images, so that the product surface crack detection method can maintain a high recognition success rate; Chinese patent application number CN202010540227.0, published on September 22, 2020, discloses a surface defect detection method, model training method, device, equipment and medium, involving the fields of artificial intelligence computer vision, deep learning and cloud computing, and specifically image recognition technology. The specific implementation scheme is as follows: inputting the surface image of the object to be tested into a defect detection model for defect detection, and obtaining the defect detection results output by the defect detection model; based on the defect detection results, inputting the surface image of the defective object determined to have defects into an image discrimination model to discriminate whether the surface image of the defective object has defects; wherein the image discrimination model is a trained generative adversarial network model, which is trained using the surface images of defect-free good objects; and adjusting the defect detection results of the surface image of the defective object based on the discrimination results of the image discrimination model. This embodiment of the application can effectively reduce the over-kill rate of the surface image of the object to be tested.

[0004] The shortcomings of the above patent are: since the computer power button includes multiple surfaces, all painted surfaces of the button need to be tested for paint uniformity one by one, the overall efficiency is low and the classification of defect types is not accurate enough. Summary of the Invention

[0005] 1. Problems to be solved

[0006] In response to the problems in the prior art of low overall efficiency in detecting the uniformity of each surface of a key and inaccurate classification of defect types, the present invention provides a method, device, equipment and readable storage medium for detecting the uniformity of key paint. By utilizing machine vision to detect the uniformity of key paint, a camera is used to take pictures of the key to be inspected, and a segmentation algorithm is used to extract the effective area of ​​the key paint. Then, the defects of each surface are detected, and the paint uniformity of the key is determined based on the defect conditions of each surface.

[0007] 2. Technical solution

[0008] To solve the above problems, the present invention adopts the following technical solutions.

[0009] A method for detecting the uniformity of key paint spraying comprises the following steps:

[0010] Collect images of the key to be detected from different perspectives;

[0011] Input images of the key to be inspected from different perspectives into a key paint detection neural network model, perform key view classification and key paint defect classification in sequence, and obtain paint defect classification results for each key view; wherein the key paint detection neural network model is obtained by training a Mask R-CNN neural network using a first preset sample set and a second preset sample set, wherein the first preset sample set includes multiple labeled image data of effective paint areas, and the second preset sample set includes multiple labeled image data of paint defects;

[0012] Based on the paint defect classification results in each view of the button, the detection results of the button paint uniformity are output.

[0013] Preferably, the step of collecting images of the key to be detected from different viewing angles includes:

[0014] Images are captured for each surface of the key to be inspected at a preset image capture distance. The image capture distances for the surfaces that are always seen, the surfaces that may be faced, and the surfaces that are rarely seen are 250 cm, 450 cm, and 450 cm, respectively.

[0015] Preferably, the view classification results include view categories that are always seen, view categories that may be encountered, and view categories that are rarely seen; the key paint defect classification includes one or more of orange peel, paint accumulation, bubbles, ruptures, cracks, and trachoma.

[0016] Preferably, the button paint detection neural network model includes a button view classification network, and the button view classification network is used to perform button view classification on the button to be detected.

[0017] Preferably, the specific steps of performing key view classification on the key to be detected include:

[0018] Preprocess the images of the key to be detected from different perspectives;

[0019] The processed image is passed into the trained key view classification network to obtain the corresponding feature map;

[0020] Set ROI for each point in the feature map to obtain multiple ROI candidate boxes;

[0021] Input multiple ROI candidate boxes into the RPN network and perform binary classification and BB regression in sequence;

[0022] Perform ROI Align operation on the remaining ROIs;

[0023] The ROI after ROI Align operation is classified, BB regression and MASK generated in sequence;

[0024] Output the view classification result of the image.

[0025] Preferably, the button paint detection neural network model includes a button paint defect classification network, and the button paint defect classification network is used to classify defects existing in the buttons to be detected.

[0026] Preferably, outputting the detection result of the paint uniformity of the key according to the paint defect classification result in each view of the key specifically includes:

[0027] If no defective area is detected in the inspection results, the original image of each view and the inspection results are output as "qualified";

[0028] If a defective area is detected in the inspection result, an identification image marking the defective area, an enlarged image of each defective area and the defect type are output, and the inspection result is "failed".

[0029] To solve the above technical problems, the present invention further provides a device for detecting the uniformity of key paint spraying, which is used to implement any of the above methods, including:

[0030] An acquisition module, used to acquire images of the key to be detected from different viewing angles;

[0031] A surface extraction tool is used to classify the key views of the key to be detected collected by the acquisition module from different perspectives;

[0032] Defect extraction tool, used to classify key paint defects in each key view based on the key view classification results of the surface extraction tool;

[0033] And a uniformity evaluation module is used to output the detection results of the uniformity of the key painting according to the paint defect classification results under each view of the key.

[0034] In order to solve the above technical problems, the present invention also provides a device for detecting the uniformity of key painting, comprising:

[0035] memory for storing computer programs;

[0036] A processor is configured to implement the steps of a method for detecting the uniformity of key painting as described in any one of the above items when executing the computer program.

[0037] In order to solve the above technical problems, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for detecting the uniformity of key paint spraying as described in any one of the above items are implemented.

[0038] 3. Beneficial effects

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The present invention provides a method for detecting the uniformity of key paint spraying. When detecting the uniformity of key paint spraying, the use of machine vision methods effectively improves the speed and accuracy of detection and reduces labor costs. Since the surface defects of the keys can be quickly output, the cause of unqualified key paint spraying can be quickly found.

[0041] The present invention also provides a job control device, equipment and computer-readable storage medium, which have the same beneficial effects as the above job control method. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic flow chart of a method for detecting the uniformity of key paint spraying provided by the present invention. DETAILED DESCRIPTION

[0043] The core of the present invention is to provide a method for detecting the uniformity of key paint spraying. By using machine vision to perform key paint uniformity detection, a camera is used to take pictures of the key to be detected, and a segmentation algorithm is used to extract the effective area of ​​the key paint spraying, and then the defects of each surface are detected. According to the defects of each surface, the uniformity of the key paint spraying is determined; another core of the present invention is to provide an apparatus, equipment and computer-readable storage medium for detecting the uniformity of key paint spraying. A deep learning method is used to construct a surface extraction tool and a defect extraction tool to perform key paint uniformity detection, which effectively improves the speed and accuracy of detection and reduces labor costs.

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0045] Please refer to Figure 1 , Figure 1 A flow chart of a method for detecting the uniformity of key paint spraying provided by the present invention includes:

[0046] Step S100: collecting images of the key to be detected from different viewing angles;

[0047] Specifically, considering the technical problems in the above background technology, since all painted surfaces of the keys need to be tested for paint uniformity one by one, while improving the efficiency of the test and improving the accuracy of the classification of the detected defect types as much as possible, the embodiment of the present invention intends to improve its detection efficiency and accuracy through deep learning. Therefore, it is first necessary to collect images of the keys to be tested from different perspectives, so as to facilitate the surface classification and defect classification in the subsequent deep learning stage. The collection of images of the keys to be tested from different perspectives can be completed by taking pictures of the keys to be tested using a variety of image acquisition devices such as cameras, which is not limited in this application.

[0048] Step S200: Input images of the key to be inspected from different perspectives into a key paint detection neural network model, perform key view classification and key paint defect classification in sequence, and obtain paint defect classification results for each key view; wherein the key paint detection neural network model is obtained by training a Mask R-CNN neural network using a first preset sample set and a second preset sample set, wherein the first preset sample set includes multiple labeled image data of effective paint areas, and the second preset sample set includes multiple labeled image data of paint defects;

[0049] Specifically, in this embodiment, the button paint detection neural network model includes a button view classification network and a button paint defect classification network. The button paint detection neural network model is obtained by training the Mask R-CNN neural network through a first preset sample set and a second preset sample set. Mask-RCNN is an instance segmentation algorithm used for target instance segmentation; Mask R-CNN upgrades the ROIPooling layer of Fast R-CNN to an ROI Align layer, and adds a branch FCN layer, namely the mask layer, on the basis of bounding box recognition for semantic mask recognition, generates target candidate boxes through the RPN network, and then classifies and judges each target candidate box and regresses the border, while using a fully convolutional network to predict the segmentation of each target candidate box; Mask R-CNN is essentially an instance segmentation algorithm. Compared with conventional neural network training models, in this embodiment, the Mask R-CNN neural network is selected as the basic architecture of the button paint detection neural network model, which can better segment the valid area and defect area in the image. Both the button view classification network and the button paint defect classification network need to be trained before use. Since the button view classification network and the button paint defect classification network share the same basic architecture, their training processes are largely the same, differing only in the choice of input dataset. Taking the button paint defect classification network training process as an example, it generally includes three steps: sample dataset processing, model framework construction, and model training. The details are as follows:

[0050] Acquire training samples and preprocess the training samples; the training samples include multiple labeled effective spray paint area image data; when labeling, first perform classification setting; then select the target area in the image; and finally set the classification of the target box;

[0051] The process of building the model framework is as follows: the first stage builds the RPN layer that scans the image and generates proposals; the second stage builds the prediction category, bbox regression and mask for prediction and segmentation.

[0052] The model training process is as follows: download the pre-trained weight file, modify the file parameters and dataset path; create and execute the training script file.

[0053] The convolutional neural network is trained and optimized. The training dataset is applied to the convolutional network established in the previous step for training to obtain a button paint defect classification network based on the Mask R-CNN neural network.

[0054] In some implementations of this embodiment, when classifying the acquired key view images, the following implementation can be performed: the view classification results include a view category that is always seen, a view category that may be encountered, and a view category that is rarely seen. In other implementations of this embodiment, when classifying the acquired key paint defect images, the following implementation can be performed: the view classification results include a key paint defect category that includes one or more of orange peel, paint accumulation, bubbles, ruptures, cracks, and trachoma. It is worth noting that for different view classification categories, in order to further improve the accuracy of the output results of the image instance segmentation model and enhance the detection accuracy of each key view and defect, the acquired key view images can be processed and then used as input data. In a preferred embodiment, images of each key surface to be inspected can be captured at a preset image capture distance. The image capture distances corresponding to the always seen surface, the likely encountered surface, and the rarely seen surface can be set to 250 cm, 450 cm, and 450 cm, respectively. By classifying different views into different categories, different preset methods are adopted to capture images when constructing data sample sets. Since different views are subject to the different attention paid by users to different areas of the keys and the limited field of view when capturing images of surfaces that are always seen, surfaces that may be faced, and surfaces that are rarely seen, different uniformity requirements are placed on different areas of the keys during use. The above method can maximize the key qualification rate while meeting the uniformity standards.

[0055] After training the button view classification network and the button paint defect classification network using the above method, the button view classification network can be used to classify the button view of the button to be inspected, and the button paint defect classification network can be used to classify the defects of the button to be inspected. Here, taking the button view classification of the button to be inspected as an example, the specific steps include:

[0056] Preprocess the images of the key to be detected from different perspectives;

[0057] The processed image is passed into the trained key view classification network to obtain the corresponding feature map;

[0058] Set ROI for each point in the feature map to obtain multiple ROI candidate boxes;

[0059] Input multiple ROI candidate boxes into the RPN network and perform binary classification and BB regression in sequence;

[0060] Perform ROI Align operation on the remaining ROIs;

[0061] The ROI after ROI Align operation is classified, BB regression and MASK generated in sequence;

[0062] Output the view classification result of the image.

[0063] Step S300: Output the detection result of the key painting uniformity according to the painting defect classification result in each view of the key.

[0064] Specifically, the above-mentioned technical solution can be used to obtain the classification results of paint defects in each view of the key. When outputting the detection results of the uniformity of the key paint, it specifically includes:

[0065] If no defective area is detected in the inspection results, the original image of each view and the inspection results are output as "qualified";

[0066] If a defective area is detected in the inspection result, an identification image marking the defective area, an enlarged image of each defective area and the defect type are output, and the inspection result is "failed".

[0067] By adopting the above-mentioned method for detecting the uniformity of key paint spraying, the use of machine vision methods effectively improves the speed and accuracy of detection when detecting the uniformity of key paint spraying, and reduces labor costs; since the surface defects of the keys can be quickly output, the reason for the unqualified key paint spraying can be quickly found.

[0068] It should be noted that there is no strict order in which the steps in this application are performed. As long as they comply with the logical order, these steps can be performed simultaneously or in a predetermined order. Figure 1 This is just a schematic and does not mean that this is the only execution order.

[0069] The embodiments of the present invention also provide a corresponding device for detecting the uniformity of key paint spraying, further enhancing the practicality of the method. The device can be described from the perspective of functional modules and hardware. The following describes the device for detecting the uniformity of key paint spraying provided by the embodiments of the present invention. The device described below can be used in conjunction with the method for detecting the uniformity of key paint spraying described above.

[0070] From the perspective of functional modules, one embodiment of the present invention further provides a device for detecting the uniformity of key painting, for implementing the above method, and the device may include:

[0071] An acquisition module, used to acquire images of the key to be detected from different viewing angles;

[0072] The surface extraction tool is used to classify the key views of the key to be detected collected by the acquisition module from different perspectives. Taking a conventional key as an example, it generally includes 4 surfaces. Please refer to Figure 1 , at this time, the surface extraction tool can be set to A surface extraction tool, B surface extraction tool, C surface extraction tool, and D surface extraction tool respectively;

[0073] Defect extraction tool, used to classify key paint defects in each key view based on the key view classification results of the surface extraction tool;

[0074] And the uniformity evaluation module is used to output the detection results of the key painting uniformity according to the paint defect classification results under each view of the key; for more details, please refer to Figure 1 Among them, the A-side paint defect detection tool, B-side paint defect detection tool, C-side paint defect detection tool and D-side paint defect detection tool all include defect extraction tools and uniformity evaluation modules to detect and classify the corresponding button paint surface defects and output the results.

[0075] The functions of the various functional modules of the device for detecting the uniformity of key painting described in the embodiment of the present invention can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description of the above method embodiment, which will not be repeated here.

[0076] The device for detecting the uniformity of key paint spraying mentioned above is described from the perspective of functional modules. Furthermore, this application also provides a device for detecting the uniformity of key paint spraying, described from the perspective of hardware. The device includes a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the method for detecting the uniformity of key paint spraying as described in any of the above embodiments.

[0077] The processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0078] The memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory is at least used to store the following computer program, wherein, after the computer program is loaded and executed by the processor, it can implement the relevant steps of the method for detecting the uniformity of key paint spraying disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory may also include an operating system and data, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system may include Windows, Unix, Linux, etc. The data may include but is not limited to data corresponding to the test results, etc.

[0079] The functions of the various functional modules of the device for detecting the uniformity of key painting described in the embodiment of the present invention can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description of the above method embodiment, which will not be repeated here.

[0080] From the above, it can be seen that the device for detecting the uniformity of key paint spraying in the embodiment of the present invention uses the method of machine vision to effectively improve the speed and accuracy of detection and reduce labor costs when detecting the uniformity of key paint spraying; since it can quickly output the surface defects of the keys, the reason for the unqualified key paint spraying can be quickly found.

[0081] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0082] Those skilled in the art may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0083] The above is a detailed introduction to a method for detecting the uniformity of key painting provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A method for detecting the uniformity of key painting, characterized by: The following steps are involved: Collect images of the key to be detected from different perspectives; Input images of the key to be inspected from different perspectives into a key paint detection neural network model, perform key view classification and key paint defect classification in sequence, and obtain paint defect classification results for each key view; wherein the key paint detection neural network model is obtained by training a Mask R-CNN neural network using a first preset sample set and a second preset sample set, wherein the first preset sample set includes multiple labeled image data of effective paint areas, and the second preset sample set includes multiple labeled image data of paint defects; Output the detection result of the key painting uniformity according to the paint defect classification results under each view of the key; and The collecting of images of the key to be detected from different viewing angles includes: Capture images of each surface of the key to be inspected at a preset image capture distance. The image capture distances for surfaces that are always visible, surfaces that may be encountered, and surfaces that are rarely seen are 250 cm, 450 cm, and 450 cm, respectively. The view classification results include view categories that are always seen, view categories that may be encountered, and view categories that are rarely seen; the key paint defect classification includes one or more of orange peel, paint accumulation, bubbles, ruptures, cracks, and trachoma.

2. The method for detecting the uniformity of key painting according to claim 1, characterized in that: The button paint detection neural network model includes a button view classification network, and the button view classification network is used to perform button view classification on the button to be detected.

3. The method for detecting the uniformity of key painting according to claim 2, characterized in that: The specific steps for key view classification of the key to be detected include: Preprocess the images of the key to be detected from different perspectives; The processed image is passed into the trained key view classification network to obtain the corresponding feature map; Set ROI for each point in the feature map to obtain multiple ROI candidate boxes; Input multiple ROI candidate boxes into the RPN network and perform binary classification and BB regression in sequence; Perform ROIAlign operation on the remaining ROIs; The ROI after ROIAlign operation is classified, BB regression and MASK generated in sequence; Output the view classification result of the image.

4. The method for detecting the uniformity of key painting according to claim 1, characterized in that: The key paint detection neural network model includes a key paint defect classification network, and the key paint defect classification network is used to classify defects existing in the key to be detected.

5. The method for detecting the uniformity of key painting according to claim 1, characterized in that: Outputting the detection result of the paint uniformity of the key according to the paint defect classification result in each view of the key specifically includes: If no defective area is detected in the inspection results, the original image of each view and the inspection results are output as "qualified"; If a defective area is detected in the inspection results, an identification image of the defective area, an enlarged image of each defective area and the defect type will be output, and the inspection result will be "unqualified".

6. A device for detecting the uniformity of paint spraying on a key, for implementing the method according to any one of claims 1 to 5, characterized in that: include: An acquisition module, used to acquire images of the key to be detected from different viewing angles; A surface extraction tool is used to classify the key views of the key to be detected collected by the acquisition module from different perspectives; Defect extraction tool, used to classify key paint defects in each key view based on the key view classification results of the surface extraction tool; And a uniformity evaluation module is used to output the detection results of the uniformity of the key painting according to the paint defect classification results under each view of the key.

7. A device for detecting the uniformity of key paint spraying, characterized by: include: Memory for storing computer programs; A processor, configured to implement the steps of a method for detecting the uniformity of key painting as described in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for detecting the uniformity of key painting as described in any one of claims 1 to 5.

Images