Appearance defect detection method and device for electrical equipment

Abstract

The invention discloses an electrical equipment appearance defect detection method and device. The method comprises the following steps: acquiring a first panel image and a second panel image of an injection molding panel of target electrical equipment; inputting the first panel image and the second panel image into a panel appearance defect detection model, and processing the first panel image and the second panel image by using the panel appearance defect detection model to obtain an appearance defect detection result of the target electrical equipment; and determining defect information of the injection molding panel of the target electrical equipment according to the appearance defect detection result. According to the defect detection method and device, the technical problem that the defect detection accuracy of the air conditioner is low due to the fact that the unknown defect detection capacity is poor in the defect detection of the air conditioner in the related technology is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image target detection, and in particular to a method and device for detecting appearance defects of electrical equipment. Background Art

[0002] In the air conditioner production line, the quality of injection molded panels directly affects the appearance and user experience of the final product. In the quality inspection of air conditioner injection molded panels, the focus includes surface defect detection, which usually refers to the judgment of whether the surface is uneven, dirty, or scratched. Through target detection, the position and size of the defect are located. In actual production, the defect rate is relatively low, so the data of negative samples is relatively small, and the formation of defects varies, resulting in complex and changeable actual forms of defects. Traditional defect detection methods rely on manual visual inspection or basic automated visual systems. These methods have obvious deficiencies in detection efficiency, accuracy, and consistency. Although the existing deep learning technology has made significant progress in the field of image processing, it requires a large number of specific samples when facing the specific defect detection task of air conditioner injection molded panels. In reality, there are often problems such as a small number of samples, complex and changeable defect types, and difficulty in model migration. When using deep learning models, a large number of samples are required for training, and in similar scenarios, sufficient data is usually collected for model fine-tuning to adapt to new scenarios. If traditional image processing methods are used for processing, feature extractors need to be manually designed for defect morphology, which makes it difficult to cover various types of defects and has poor detection capabilities for unknown defects.

[0003] For example, in the prior art, dual-path depth residual threshold features are used to remove the influence of background noise in aerial images. The present invention is based on a visual prompt mechanism and a dual-path detection network to determine whether there are bumps, dirt, or scratches on the air-conditioning panel, thereby solving the difficulties in acquiring data sets in target detection and the poor transferability of models for different scenarios.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for detecting appearance defects of electrical equipment, so as to at least solve the technical problem that the detection capability of unknown defects in defect detection of air conditioners in the related art is poor, resulting in low accuracy of defect detection of air conditioners.

[0006] According to one aspect of an embodiment of the present invention, a method for detecting appearance defects of an electrical device is provided, comprising: obtaining a first panel image and a second panel image of an injection molding panel of a target electrical device, wherein the first panel image is a standard panel image without defects, and the second panel image is an image to be subjected to defect detection; inputting the first panel image and the second panel image into a panel appearance defect detection model, so as to process the first panel image and the second panel image using the panel appearance defect detection model to obtain an appearance defect detection result of the target electrical device, wherein the panel appearance defect detection model is a model obtained by machine learning training using multiple sets of training data, each of the multiple sets of training data includes: a sample panel image and a sample appearance defect detection result corresponding to the sample panel image, the sample panel image includes: a first sample panel image, a second sample panel image and a sample defect image, and the sample defect image is a panel defect image generated according to the first sample panel image; and determining defect information of the injection molding panel of the target electrical device according to the appearance defect detection result.

[0007] Optionally, the first panel image and the second panel image are input into a panel appearance defect detection model so as to use the panel appearance defect detection model to process the first panel image and the second panel image to obtain the appearance defect detection result of the target electrical device, including: performing a standardization operation on the first panel image and the second panel image to obtain the first panel image and the second panel image after standardization, wherein the standardization operation includes: a scaling operation and a filling operation; and inputting the first panel image and the second panel image after standardization into the panel appearance defect detection model so as to use the panel appearance defect detection model to process the first panel image and the second panel image after standardization to obtain the appearance defect detection result of the target electrical device.

[0008] Optionally, before inputting the first panel image and the second panel image into a panel appearance defect detection model to process the first panel image and the second panel image using the panel appearance defect detection model to obtain an appearance defect detection result of the target electrical device, the appearance defect detection method of the electrical device further includes: acquiring the multiple sets of training data; performing standardization processing on the multiple sets of training data to obtain the multiple sets of standardized training data; and training the panel appearance defect detection model using the multiple sets of standardized training data.

[0009] Optionally, obtaining the multiple sets of training data includes: obtaining the first sample panel image and the second sample panel image of the injection molding panels of multiple electrical equipment including the target electrical equipment, and annotating defects on the second sample panel image; generating the sample defect image based on the first sample panel image, and annotating defects on the sample defect image.

[0010] Optionally, generating the sample defect image according to the first sample panel image includes: determining the defect type to be generated of the target electrical equipment; setting defect information of each defect type to be generated according to the first sample panel image, wherein the defect information is used to describe the defects to be generated corresponding to each defect type to be generated; generating each defect to be generated according to the defect information; and performing pixel processing on each defect to be generated to obtain the sample defect image.

[0011] Optionally, the defect types to be generated include: dirt type defects, scratch type defects and concave-convex type defects, and the defect information of each defect type to be generated is set according to the first sample panel image, including: first defect information including the maximum number of dirty areas of the dirt type defects, the size of each dirty area and the dirty color range defined according to the first sample panel image; second defect information including the maximum number of scratches of the scratch type defects, the size of each scratch defined according to the first sample panel image; third defect information including the maximum number of concave-convex areas of the concave-convex type defects, the size of each concave-convex area and the degree of concave-convex feeling defined according to the first sample panel image.

[0012] Optionally, each defect to be generated is generated according to the defect information, including: generating the dirt type defect on the first sample panel image according to the first defect information through a random noise algorithm or a random walk algorithm; randomly selecting a starting point and a scratch direction on the first sample panel image, and generating the scratch type defect according to the second defect information; generating the concave-convex type defect of a predetermined shape on the first sample panel image according to the third defect information through an elliptical or random walk algorithm.

[0013] Optionally, pixel processing is performed on each of the defects to be generated to obtain the sample defect image, including: filling the dirt-type defect by randomly selecting a color; filling the scratch-type defect with a color according to the brightness deviation of the color within a predetermined range of the scratch-type defect; determining a pixel variation range corresponding to the pixel value when simulating the concave-convex defect according to the depth variation range of the concave-convex defect, and performing pixel adjustment on the concave-convex defect according to the pixel variation range.

[0014] Optionally, the panel appearance defect detection model is trained using the multiple groups of training data after standardization, including: obtaining the occluded part and the unoccluded part of the first sample panel image and the sample defect image by randomly occluding predetermined parts of the first sample panel image and the sample defect image; inputting the unoccluded part into the encoder in the panel appearance defect detection model to train the encoder until the encoder can reconstruct the first sample panel image and the sample defect image; training the two-way detection model and the decoder in the panel appearance defect detection model by the real defects marked in the second sample panel image and the generated defects corresponding to the sample defect image until the two-way detection model and the decoder can generate and identify the real defects and the generated defects and generate defect images according to the real defects and the generated defects; splicing the encoder with the two-way detection model and the decoder to obtain the panel appearance defect detection model.

[0015] Optionally, the encoder is spliced ​​with the dual-path detection model and the decoder to obtain the panel appearance defect detection model, including: classifying the sample defect images according to dirt-type defects, scratch-type defects and concave-convex defects to obtain three different types of sample defect image subsets; using each of the sample defect image subsets to train the visual cue network and the decoder in the panel appearance defect detection model, respectively, until the panel appearance defect detection model can identify the defect type corresponding to each of the sample defect images, to obtain the panel appearance defect detection model.

[0016] According to another aspect of an embodiment of the present invention, there is also provided an appearance defect detection device for an electrical device, comprising: a first acquisition module, used to acquire a first panel image and a second panel image of an injection molding panel of a target electrical device, wherein the first panel image is a standard panel image without defects, and the second panel image is an image to be subjected to defect detection; a first processing module, used to input the first panel image and the second panel image into a panel appearance defect detection model, so as to process the first panel image and the second panel image using the panel appearance defect detection model to obtain an appearance defect detection result of the target electrical device, wherein the panel appearance defect detection model is a model obtained by machine learning training using multiple sets of training data, each of the multiple sets of training data includes: a sample panel image and a sample appearance defect detection result corresponding to the sample panel image, the sample panel image includes: a first sample panel image, a second sample panel image and a sample defect image, and the sample defect image is a panel defect image generated according to the first sample panel image; a first determination module, used to determine defect information of the injection molding panel of the target electrical device according to the appearance defect detection result.

[0017] Optionally, the first processing module includes: a first processing unit, used to perform a standardization processing operation on the first panel image and the second panel image to obtain the first panel image and the second panel image after standardization, wherein the standardization processing operation includes: a scaling operation and a filling operation; a second processing unit, used to input the first panel image and the second panel image after standardization into the panel appearance defect detection model, so as to use the panel appearance defect detection model to process the first panel image and the second panel image after standardization to obtain the appearance defect detection result of the target electrical equipment.

[0018] Optionally, before the first processing module, the appearance defect detection device of the electrical equipment also includes: a second acquisition module, used to acquire the multiple groups of training data; a second processing module, used to standardize the multiple groups of training data to obtain the multiple groups of training data after standardization; and a third processing module, used to train the panel appearance defect detection model using the multiple groups of training data after standardization.

[0019] Optionally, the second acquisition module includes: a third processing unit, used to acquire the first sample panel image and the second sample panel image of the injection molding panels of multiple electrical equipment including the target electrical equipment, and to mark defects on the second sample panel image; a fourth processing unit, used to generate the sample defect image based on the first sample panel image, and to mark defects on the sample defect image.

[0020] Optionally, the fourth processing unit includes: a first determination subunit, used to determine the defect type to be generated of the target electrical equipment; a first processing subunit, used to set defect information of each defect type to be generated according to the first sample panel image, wherein the defect information is used to describe the defects to be generated corresponding to each defect type to be generated; a second processing subunit, used to generate each defect to be generated according to the defect information; and a pixel processing subunit, used to perform pixel processing on each defect to be generated to obtain the sample defect image.

[0021] Optionally, the first processing subunit includes: a first definition subunit, which is used to define first defect information including the maximum number of dirty areas of the dirt type defect, the size of each of the dirty areas, and the dirty color range according to the first sample panel image; a second definition subunit, which is used to define second defect information including the maximum number of scratches of the scratch type defect, and the size of each of the scratches according to the first sample panel image; and a third definition subunit, which is used to define third defect information including the maximum number of concave-convex areas of the concave-convex type defect, the size of each of the concave-convex areas, and the degree of concave-convex feeling according to the first sample panel image.

[0022] Optionally, the second processing subunit includes: a third processing subunit, used to generate the dirt type defect on the first sample panel image according to the first defect information through a random noise algorithm or a random walk algorithm; a fourth processing subunit, used to randomly select a starting point and a scratch direction on the first sample panel image, and generate the scratch type defect according to the second defect information; a fifth processing subunit, used to generate the concave-convex type defect of a predetermined shape on the first sample panel image according to the third defect information through an elliptical or random walk algorithm.

[0023] Optionally, the pixel processing subunit includes: a sixth processing subunit, used to fill the dirt type defect by randomly selecting a color; a seventh processing subunit, used to fill the scratch type defect with color according to the brightness deviation of the color within a predetermined range of the scratch type defect; an eighth processing subunit, used to determine the pixel variation range corresponding to the pixel value when simulating the concave-convex type defect according to the depth variation range of the concave-convex type defect, and perform pixel adjustment on the concave-convex type defect according to the pixel variation range.

[0024] Optionally, the third processing module includes: a fifth processing unit, used to obtain the occluded part and the unoccluded part of the first sample panel image and the sample defect image by randomly occluding predetermined parts of the first sample panel image and the sample defect image; a sixth processing unit, used to input the unoccluded part into the encoder in the panel appearance defect detection model to train the encoder until the encoder can reconstruct the first sample panel image and the sample defect image; a seventh processing unit, used to train the two-way detection model and the decoder in the panel appearance defect detection model through the real defects marked in the second sample panel image and the generated defects corresponding to the sample defect image, until the two-way detection model and the decoder can generate and identify the real defects and the generated defects and generate defect images according to the real defects and the generated defects; a splicing unit, used to splice the encoder with the two-way detection model and the decoder to obtain the panel appearance defect detection model.

[0025] Optionally, the stitching unit includes: a classification subunit, used to classify the sample defect images according to dirt-type defects, scratch-type defects and concave-convex defects, and obtain three different types of sample defect image subsets; a ninth processing subunit, used to respectively use each of the sample defect image subsets to train the visual cue network and the decoder in the panel appearance defect detection model until the panel appearance defect detection model can identify the defect type corresponding to each of the sample defect images, and obtain the panel appearance defect detection model.

[0026] According to another aspect of an embodiment of the present invention, there is further provided a system for detecting appearance defects of an electrical device, wherein the system for detecting appearance defects of an electrical device uses any one of the above-mentioned methods for detecting appearance defects of an electrical device.

[0027] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein the program executes any one of the above-mentioned methods for detecting appearance defects of electrical equipment.

[0028] According to another aspect of an embodiment of the present invention, a processor is further provided, the processor being used to run a program, wherein the program, when running, executes any one of the above-mentioned methods for detecting appearance defects of electrical equipment.

[0029] According to another aspect of an embodiment of the present invention, a computer program product is provided, comprising computer instructions, wherein when the computer instructions are executed by a processor, any one of the above-mentioned methods for detecting appearance defects of an electrical device is executed.

[0030] In an embodiment of the present invention, a first panel image and a second panel image of an injection molding panel of a target electrical device are obtained, wherein the first panel image is a standard panel image without defects, and the second panel image is an image to be subjected to defect detection; the first panel image and the second panel image are input into a panel appearance defect detection model, so as to process the first panel image and the second panel image using the panel appearance defect detection model to obtain an appearance defect detection result of the target electrical device, wherein the panel appearance defect detection model is a model obtained by machine learning training using multiple sets of training data, each of the multiple sets of training data includes: a sample panel image and a sample appearance defect detection result corresponding to the sample panel image, the sample panel image includes: a first sample panel image, a second sample panel image and a sample defect image, and the sample defect image is a panel defect image generated according to the first sample panel image; and defect information of the injection molding panel of the target electrical device is determined according to the appearance defect detection result. Through the technical solution provided by the present invention, a network structure combining dual-path detection and a decoder is achieved, and the features of the two paths are fused through cross-attent ion to introduce visual cues. At the same time, the mask image generated by the dual-path detection network and the decoder network specific to the injection molding panel can efficiently and accurately identify the defect location, thereby achieving the technical effects of reducing the demand for negative samples, efficiently detecting defect features, and quickly adapting to new scenarios, improving the migration ability and accuracy of the detection model, and further solving the technical problem in the related art that the detection ability of unknown defects in the defect detection of the air conditioner is poor, resulting in low accuracy of the defect detection of the air conditioner. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0032] Figure 1 It is a hardware structure block diagram of a mobile terminal of a method for detecting appearance defects of an electrical device according to an embodiment of the present invention;

[0033] Figure 2 is a flow chart of a method for detecting appearance defects of an electrical device according to an embodiment of the present invention;

[0034] Figure 3 is a flow chart of the reasoning phase of the appearance defect detection method of an electrical device according to an embodiment of the present invention;

[0035] Figure 4 is a flowchart of training a MAE image encoder according to a method for detecting appearance defects of an electrical device according to an embodiment of the present invention;

[0036] Figure 5 is a flowchart of the main network training of the appearance defect detection method of the electrical equipment according to the embodiment of the present invention;

[0037] Figure 6 is a flow chart of visual prompt training of a method for detecting appearance defects of an electrical device according to an embodiment of the present invention;

[0038] Figure 7 is a flow chart of a migration phase of a method for detecting appearance defects of an electrical device according to an embodiment of the present invention;

[0039] Figure 8 is a flow chart of an alternative implementation scheme of a method for detecting appearance defects of an electrical device according to an embodiment of the present invention;

[0040] Fig. 9 is a schematic diagram of an appearance defect detection device for an electrical device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0042] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0043] As introduced in the background technology, the related art has a poor ability to detect unknown defects in the defect detection of air conditioners, resulting in a technical problem of low accuracy in the defect detection of air conditioners. In view of the above defects, an appearance defect detection method and device for electrical equipment, an appearance defect detection system for electrical equipment, a computer-readable storage medium, a processor, and a computer program product are provided in the embodiments of the present invention.

[0044] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0045] The method embodiments provided in the embodiments of the present invention can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a method for detecting appearance defects of an electrical device according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0046] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the appearance defect detection method of the electrical equipment in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0047] According to an embodiment of the present invention, a method embodiment of a method for detecting appearance defects of electrical equipment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] Figure 2 is a flow chart of a method for detecting appearance defects of an electrical device according to an embodiment of the present invention. Figure 2 As shown, the method comprises the following steps:

[0049] Step S202 , obtaining a first panel image and a second panel image of an injection molding panel of a target electrical device, wherein the first panel image is a standard panel image without defects, and the second panel image is an image to be inspected for defects.

[0050] Optionally, the target electrical equipment may include, but is not limited to: an air conditioner, a refrigerator, and a water heater. In the embodiment of the present invention, the target electrical equipment is an air conditioner as an example for description.

[0051] In this embodiment, image data of the injection-molded panel of the air conditioner can be collected, including defect-free standard images (ie, first panel images) and various defective images (ie, second panel images), which can provide a data basis for subsequent model training.

[0052] Step S204, inputting the first panel image and the second panel image into a panel appearance defect detection model, so as to use the panel appearance defect detection model to process the first panel image and the second panel image to obtain an appearance defect detection result of the target electrical equipment, wherein the panel appearance defect detection model is a model trained by machine learning using multiple sets of training data, each of the multiple sets of training data includes: a sample panel image and a sample appearance defect detection result corresponding to the sample panel image, the sample panel image includes: a first sample panel image, a second sample panel image and a sample defect image, and the sample defect image is a panel defect image generated based on the first sample panel image.

[0053] Optionally, the first sample panel image is a panel image of an electrical device without defects, the second sample panel image is a panel image of an electrical device with defects, and the sample defect images are defect sample panel images generated randomly, such as scratch-type defect images, dirty-type defect images, and concave-convex defect images.

[0054] It should be noted that the above-mentioned panel appearance defect detection model is a model that uses computer vision and machine learning technology to automatically detect panel appearance defects. It can detect various defects by analyzing the image of the panel, and can quickly and accurately detect defects in the panel, thereby improving production efficiency and product quality.

[0055] In this embodiment, the first panel image and the second panel image can be processed by the panel appearance defect detection model, so that the appearance defects of the target electrical device can be effectively detected.

[0056] Step S206, determining defect information of the injection molding panel of the target electrical device according to the appearance defect detection result.

[0057] In this embodiment, by detecting and determining the defect information of the injection-molded panel, quality problems existing in the product production process can be discovered and resolved in a timely manner, the risk of product recall can be reduced, user experience can be guaranteed, and production costs can be reduced.

[0058] As can be seen from the above, in an embodiment of the present invention, a first panel image and a second panel image of an injection molding panel of a target electrical device are obtained, wherein the first panel image is a standard panel image without defects, and the second panel image is an image to be subjected to defect detection; the first panel image and the second panel image are input into a panel appearance defect detection model, so as to process the first panel image and the second panel image using the panel appearance defect detection model to obtain an appearance defect detection result of the target electrical device, wherein the panel appearance defect detection model is a model obtained by machine learning training using multiple sets of training data, each of the multiple sets of training data includes: a sample panel image and a sample appearance defect detection result corresponding to the sample panel image, the sample panel image includes: a first sample panel image, a second sample panel image and a sample defect image, and the sample defect image is a panel defect image generated according to the first sample panel image; the defect information of the injection molding panel of the target electrical device is determined according to the appearance defect detection result, thereby achieving a network structure combining dual-path detection and a decoder, and through cross-attent ion fuses the features of the two paths and introduces visual cues. At the same time, the mask image generated by the dual-path detection network and decoder network specific to injection molding panels can efficiently and accurately identify the defect location, thereby achieving the technical effects of reducing the demand for negative samples, efficiently detecting defect features, and quickly adapting to new scenarios, and improving the migration ability and accuracy of the detection model.

[0059] The technical solution provided by the embodiment of the present invention solves the technical problem in the related art that the detection capability of unknown defects in the defect detection of the air conditioner is poor, resulting in low accuracy of the defect detection of the air conditioner.

[0060] According to the above embodiment of the present invention, the first panel image and the second panel image are input into the panel appearance defect detection model, so as to use the panel appearance defect detection model to process the first panel image and the second panel image to obtain the appearance defect detection result of the target electrical equipment, including: performing a standardization processing operation on the first panel image and the second panel image to obtain the first panel image and the second panel image after standardization processing, wherein the standardization processing operation includes: a scaling operation and a filling operation; inputting the first panel image and the second panel image after standardization processing into the panel appearance defect detection model, so as to use the panel appearance defect detection model to process the first panel image and the second panel image after standardization processing to obtain the appearance defect detection result of the target electrical equipment.

[0061] Optionally, the above-mentioned standardization processing operation may include: grayscale adjustment, contrast enhancement, histogram equalization and other operations. In the embodiment of the present invention, the standardization processing operation is described by taking a scaling operation and a filling operation as an example.

[0062] In this embodiment, the first panel image and the second panel image after the standardized processing can be input into the panel appearance defect detection model to obtain the appearance defect detection result of the target electrical equipment, thereby ensuring that the model can accurately detect the appearance defects of the electrical equipment in practical applications, and can quickly and accurately detect possible appearance defects, thereby improving product quality and production efficiency. Among them, the standardized processing can improve the stability and accuracy of the model.

[0063] It should be noted that the above standardization operation uses scaling + padding to process the image into a size of 448X448 and then standardizes it: Where I is the original image pixel value, μ and σ are the mean and standard deviation of all pixel values ​​in the entire data set, respectively. norm is the normalized pixel value.

[0064] Figure 3 FIG. 1 is a flow chart of the reasoning phase of the appearance defect detection method for an electrical device according to an embodiment of the present invention. Figure 3 As shown, the standard image of the input model (i.e., the first panel image) and the image to be tested (i.e., the second panel image) are subjected to standardization processing operations. For different defect detection types, the corresponding visual cues are used for splicing and then input into the network, and all network model parameters are frozen. Finally, the corresponding decoder network is used to output the defect mask, and the results of the three output masks are unioned. The output mask is post-processed, including threshold processing, morphological operations, and mask refinement, to obtain the final result mask.

[0065] According to the above embodiment of the present invention, before the first panel image and the second panel image are input into the panel appearance defect detection model to process the first panel image and the second panel image using the panel appearance defect detection model to obtain the appearance defect detection result of the target electrical device, the appearance defect detection method of the electrical device also includes: acquiring multiple sets of training data; performing standardization processing on the multiple sets of training data to obtain multiple sets of standardized training data; and training the panel appearance defect detection model using the multiple sets of standardized training data.

[0066] In this embodiment, multiple sets of training data can be obtained and standardized so that the data have the same scale and distribution, which is conducive to model training and performance improvement. The panel appearance defect detection model trained with multiple sets of training data can be used to automatically detect defects on the product surface, improve product quality and production efficiency, and reduce manual inspection costs and error rates.

[0067] According to the above-mentioned embodiment of the present invention, multiple groups of training data are obtained, including: obtaining a first sample panel image and a second sample panel image of an injection-molded panel of a plurality of electrical equipment including a target electrical equipment, and annotating defects on the second sample panel image; generating a sample defect image based on the first sample panel image, and annotating defects on the sample defect image.

[0068] In this embodiment, a first sample panel image and a second sample panel image of injection-molded panels of multiple electrical devices including a target electrical device can be acquired and annotated, and a sample defect image can be generated and annotated, which can be used to detect and identify defects on the electrical device panel.

[0069] In the above-mentioned embodiment of the present invention, a sample defect image is generated according to a first sample panel image, including: determining the defect type to be generated of the target electrical equipment; setting defect information of each defect type to be generated according to the first sample panel image, wherein the defect information is used to describe the defects to be generated corresponding to each defect type to be generated; generating each defect to be generated according to the defect information; and performing pixel processing on each defect to be generated to obtain a sample defect image.

[0070] Optionally, the above defect types may include: dirty type, scratch type and concave-convex type; the above defect information may include: defect position, size, shape, color, etc.; the above pixel processing may include: adjusting the transparency, color, edge blur and other operations of the defect.

[0071] In this embodiment, the type of target electrical equipment and common defect types can be determined, and corresponding defect information can be set for each defect type to be generated; based on the set defect information, each defect to be generated is generated, and image processing software or a programming language can be used for generation; pixel processing is performed on each generated defect to be generated to merge it with the original image to generate a sample defect image; after pixel processing, a sample defect image containing each defect to be generated is obtained, that is, a specific mask is designed according to the common defect types of the air-conditioning injection molding panel, and defective data is generated according to the designed mask, and these images can be used to train the model or test the accuracy and robustness of the detection algorithm.

[0072] It should be noted that the above-mentioned specially designed masks may include: generating a dirty mask, generating a scratch mask and generating a concave-convex mask. Generating the defect mask can be used for data enhancement, image processing, imitating dirt and the diversity of training data.

[0073] Among them, the dirty mask simulates local pollution caused by oil stains, dust, etc. The dirt usually has irregular shapes and sizes, and the color may contrast with the background color of the injection-molded panel; the scratch mask simulates the slender straight or curved damage caused by mechanical action. The scratches can be single or multiple, and the length, width and direction are different; the concave-convex mask simulates the depression or convexity caused by physical impact or pressure. This defect can usually include circular, elliptical and irregular shaped boundaries.

[0074] In the above embodiment of the present invention, the defect types to be generated include: dirt type defects, scratch type defects and concave-convex type defects, and the defect information of each defect type to be generated is set according to the first sample panel image, including: first defect information including the maximum number of dirty areas of dirt type defects, the size of each dirty area and the dirty color range defined according to the first sample panel image; second defect information including the maximum number of scratches of scratch type defects and the size of each scratch defined according to the first sample panel image; third defect information including the maximum number of concave-convex areas of concave-convex type defects, the size of each concave-convex area and the degree of concave-convex feeling defined according to the first sample panel image.

[0075] In this embodiment, defect information of dirt-type, scratch-type and concave-convex-type defects can be set according to the first sample panel image, and such defect information can help operators quickly identify defects on the surface of the equipment.

[0076] That is, the first defect information defines the maximum number of dirty areas N_dirty, the minimum size min_size of each dirty area, the maximum size max_size, and the color range of dirt color_range; the second defect information defines the maximum number of scratches N_scratch, the minimum length min_length, the maximum length max_length, the minimum width min_width, and the maximum width max_width of the scratches; and the third defect information defines the maximum number of bump areas N_bump, the minimum size min_size of each area, the maximum size max_size, and the degree of bumpiness depth_range.

[0077] In the above-mentioned embodiment of the present invention, each defect to be generated is generated according to the defect information, including: generating a dirt-type defect on the first sample panel image according to the first defect information through a random noise algorithm or a random walk algorithm; randomly selecting a starting point and a scratch direction on the first sample panel image, and generating a scratch-type defect according to the second defect information; generating a concave-convex defect of a predetermined shape on the first sample panel image according to the third defect information through an elliptical or random walk algorithm.

[0078] In this embodiment, dirty areas (i.e., dirty type defects), scratches (i.e., scratch type defects) and concave-convex areas (i.e., concave-convex type defects) can be generated according to the defect information, that is, irregular shaped areas are generated on the first sample panel image according to the set quantity and size restrictions using Perl in noise (i.e., random noise algorithm) or random walk algorithm; starting points and directions are randomly selected on the first sample panel image, and straight lines or Bezier curves are drawn according to the set quantity, length and width; circular, elliptical or irregular shaped areas are generated using elliptical or random walk algorithms.

[0079] It should be noted that the above-mentioned Perlin noise is an algorithm for generating continuous, smooth and natural random noise; the above-mentioned random walk algorithm is a path planning algorithm based on randomness, which is used to perform random movement in a given space; the above-mentioned Bezier curve is a mathematical model used to describe a curve, which has good smoothness and controllability and can well express complex curve shapes.

[0080] In the above-mentioned embodiment of the present invention, pixel processing is performed on each defect to be generated to obtain a sample defect image, including: filling dirty type defects by randomly selecting colors; filling scratch type defects with colors according to the brightness deviation of colors in a predetermined range of scratch type defects; determining the pixel variation range corresponding to the pixel value when simulating the concave-convex type defect according to the depth variation range of the concave-convex type defect, and adjusting the pixels of the concave-convex type defect according to the pixel variation range.

[0081] In this embodiment, pixel processing can be performed according to dirt type defects, scratch type defects and concave-convex type defects to obtain a sample defect image, that is, randomly selecting colors to fill the dirty area, filling the color according to the brightness deviation (10%-30%) of the color around the scratch, and depth simulation.

[0082] Among them, the depth simulation includes the following operations: determining the depth variation range, first, determining the variation range of pixel values ​​when simulating the concave and convex effect based on the depth_range parameter; randomly selecting a lighting direction, simulating the concave and convex feeling by changing the pixel values ​​in the area, for the simulation of convexities, increasing the pixel value (making it brighter) on one side of the convexity (close to the light source), and reducing the pixel value (making it darker) on the other side (away from the light source) to simulate the shadow effect; for the simulation of concave, the operation is opposite, that is, increasing the pixel value on one side of the concave (away from the light source), and reducing the pixel value on the other side (close to the light source).

[0083] It should be noted that the above depth_range is a positive or negative value, indicating the degree to which the pixel value should be increased or decreased relative to the background color.

[0084] According to the above embodiment of the present invention, a panel appearance defect detection model is obtained by training multiple groups of training data after standardization, including: obtaining the occluded part and the unoccluded part of the first sample panel image and the sample defect image by randomly occluding predetermined parts of the first sample panel image and the sample defect image; inputting the unoccluded part into the encoder in the panel appearance defect detection model to train the encoder until the encoder can reconstruct the first sample panel image and the sample defect image; training the two-way detection model and the decoder in the panel appearance defect detection model by the real defects marked in the second sample panel image and the generated defects corresponding to the sample defect image, until the two-way detection model and the decoder can generate and identify the real defects and the generated defects and generate defect images according to the real defects and the generated defects; splicing the encoder with the two-way detection model and the decoder to obtain the panel appearance defect detection model.

[0085] In this embodiment, a panel appearance defect detection model can be obtained by training multiple groups of training data after standardization, wherein the model training can include three stages: the first stage training is to train the image encoder; the second stage training is to train the dual-path detection network and the decoder network; the third stage training is to train the visual cues of the three types of air-conditioning panel defects separately.

[0086] Figure 4 FIG. 4 is a flowchart of training a MAE image encoder according to a method for detecting appearance defects of an electrical device according to an embodiment of the present invention. Figure 4As shown in the figure, the first stage of training is to train the MAE (Masked Autoencoder) image encoder, which uses a self-supervised learning method to block part of the input image information. 1 to 16 are image blocks, the gray part represents the randomly masked image block, and the white part represents the image block input to the encoder. The original input is reconstructed by training the MAE encoder-decoder network parameters. Both the original image and the generated defect image are used as training data for MAE training.

[0087] Figure 5 FIG. 4 is a flowchart of the main network training of the appearance defect detection method of the electrical equipment according to the embodiment of the present invention. Figure 5 As shown in the figure, the second stage training uses annotated data (real defects and artificially generated defects) to train the two-way detection network and the decoder network so that the model can identify defects and generate corresponding defect masks. The two-way network detector compares the difference between the standard image and the image to be tested in the implicit feature space, and obtains the difference in the image by comparison, so that the network learns to obtain the difference from the spatial information. On the one hand, it can reduce and adapt to the influence of illumination and angle; on the other hand, it does not depend on the specific form of the defect. The decoder network can restore the difference of implicit spatial features into a mask image, thereby intuitively obtaining the location and size information of the defect. Each pair of images contains a standard image and a defect image (real or generated defects), as well as the mask corresponding to the defect image. The two-way detection network receives a pair of images as input, one for processing the standard image and the other for processing the defect image. The backbone network of the two-way network is the encoder in the MAE network in the first stage above, and the parameters of this part are frozen during training. At the same time, visual cues are introduced, and the visual cues are initialized as randomly generated noise images with the same image resolution size, which are spliced ​​with the original image as new channels and input into the two-way network. The purpose of introducing visual cues is to control the adaptability of the network to specific tasks through a small number of parameters, but the main network remains unchanged to retain the model's ability to learn from a large amount of data. The cross-attent ion mechanism is used to fuse the two-way features, and finally input them into the decoder network to generate the defect mask, where the parameters of the decoder network can be initialized using the parameters of the MAE decoder. During the training process, the goal of the model is to minimize the error of the generated mask.

[0088] It should be noted that the above-mentioned real defect data requires manual annotation of real defect images to create accurate defect masks; the above-mentioned generated defect data is for defect images generated by the algorithm, and the masks when they are generated are used as annotations; the above-mentioned cross-attention mechanism is a method for fusing different features. By establishing associations between different features, a weighted representation of another feature in a feature space is achieved. By using the cross-attention mechanism to fuse two-way features, the association information between the two-way features can be effectively utilized to improve the performance and generalization ability of the model.

[0089] Figure 6 FIG. 1 is a flow chart of visual prompt training of a method for detecting appearance defects of an electrical device according to an embodiment of the present invention. Figure 6 As shown, the third stage of training mentioned above is to train visual cues for three types of defects separately. Targeted training for each defect category can further improve the accuracy of the network in identifying each type of defect.

[0090] In the above embodiment of the present invention, the encoder is spliced ​​with the dual-path detection model and the decoder to obtain a panel appearance defect detection model, including: classifying the sample defect images according to dirt-type defects, scratch-type defects and concave-convex defects to obtain three different types of sample defect image subsets; using each sample defect image subset to train the visual cue network and the decoder in the panel appearance defect detection model respectively until the panel appearance defect detection model can identify the defect type corresponding to each sample defect image, thereby obtaining the panel appearance defect detection model.

[0091] In this embodiment, the data set can be divided into three types: dirty type, scratch type, and bump type. The three types of data sets are used to train the network respectively. During training, all network parameters except visual cues and decoders are frozen until the training is completed. In this way, three visual cues and decoder heads suitable for different defect detection types can be obtained.

[0092] In an embodiment of the present invention, in order to judge whether there are defects such as bumps, dirt and scratches on the air conditioner, a network structure combining dual-path detection and a decoder is adopted, and the features of the two paths are fused through cross-attent ion to achieve more detailed air conditioner defect feature comparison and detection, and visual cues are added to the surface defect detection model of the air conditioner injection molding panel body. The model can be quickly trained using a small amount of data, which solves the problems of lack of model samples and poor migration ability in the surface defect detection of the air conditioner injection molding panel body, and can include the following steps: 1) data preparation; 2) defect mask generation; 3) detection model construction and training; 4) inference stage, the same preprocessing steps as the training stage are performed on the new input images (standard images and images to be tested), which may include size adjustment, standardization, etc., and defect flooding is output for different defects; 5) model migration, and various defects of the panel are detected.

[0093] Figure 7 1 is a flow chart of the migration phase of the appearance defect detection method of an electrical device according to an embodiment of the present invention. Figure 7 To illustrate this embodiment, model migration is performed. For a new type of injection-molded panel, when the detection accuracy is poor, for defects that need to be improved, 20 to 30 pictures of this defect type are prepared, and defect masks are annotated. All network parameters except visual cues are frozen for fine-tuning training to obtain new visual cues, which can then be used for model reasoning.

[0094] Figure 8 is a flow chart of an alternative implementation scheme of the method for detecting appearance defects of an electrical device according to an embodiment of the present invention, such as Figure 8 As shown, in the main network structure, the MAE encoder used is the same in structure and parameters, so a single network structure can be shared.

[0095] Through the technical solution provided by the above-mentioned embodiment of the present invention, specific masks are designed to generate sample pairs for the bumps, dirt and scratches of the air-conditioning panel, including dirt type, scratch type and bump type masks, and all samples (original samples and defect samples) are input into MAE (Mask Auto Encoder) for training; the sample pairs (one original sample and one defect sample) and the mask are used as the input and output of the main network for training, wherein the mask is used as the label (defect mask) output by the final network; a network structure combining dual-path detection and decoder is adopted, and the features of the two paths are fused through cross-attent ion to achieve more detailed feature comparison and defect detection, thereby improving the model's ability to detect the presence or absence of unknown defects in the air-conditioning panel. At the same time, visual cues are added to the surface defect detection model of the air-conditioning injection molding panel body, that is, before the image is input into the encoder, the visual cues are superimposed to participate in the image encoding; when migrating to other injection molding panel body detection targets, only the parameters of the visual cues are learned, so that the model can be quickly trained using a small amount of data to achieve the presence or absence of various defects in the air-conditioning panel.

[0096] Therefore, the technical solution provided by the above embodiment of the present invention solves the following problems: 1) Solve the problem that the demand for negative samples in the existing air conditioner defect detection is difficult to meet: for the judgment of the presence or absence of defects of the air conditioner, a network structure combining dual-path detection and decoder is adopted, and the features of the two paths are fused through cross-attent ion to achieve more detailed air conditioner defect feature comparison and detection, and visual cues are added to the air conditioner injection molding panel surface defect detection model. Before the image is input to the encoder, the visual cues are superimposed to participate in the image encoding. When migrating to other injection molding panel detection targets, only the parameters of the visual cues are learned, and the model can be quickly trained using a small amount of data, which solves the problem of fewer negative samples in the air conditioner injection molding panel surface defect detection task and the lack of demand for negative sample data in the model. 2) Solve the problem of poor unknown defect detection capability in air conditioner defect detection: A network structure combining dual-path detection and decoder is used, and the features of the two paths are fused through cross-attent ion to achieve more detailed feature comparison and defect detection. For the complex and changeable model defects in the air conditioner injection molding panel surface defect detection task, the model's detection capability for unknown defects is improved. 3) Solve the problem of poor migration ability of detection model in air conditioner defect detection: add visual cues to the surface defect detection model of air conditioner injection molding panel, superimpose visual cues before the image is input into the encoder to participate in image encoding, and only learn the parameters of visual cues when migrating to other injection molding panel detection targets, so that the model can be quickly trained with a small amount of data. When the surface defect detection model of air conditioner injection molding panel migrates between different detection targets, it is not necessary to collect a large amount of data and model training again, which improves the migration efficiency of model ability and solves the problem of poor model migration ability in air conditioner defect detection. At the same time, it has the following beneficial effects: 1) Using a detection method based on visual cues, by designing a specific mask to generate sample pairs, simulated defect samples are generated, and the model's demand for negative samples is reduced; 2) Using a network structure combining a dual-path detection network and a decoder network, efficient detection of defect features is achieved through feature layer comparison and self-learning; 3) Introducing a visual cue mechanism to further improve the accuracy of the network in specific defect tasks, and by adjusting the visual cues to quickly migrate the model ability, it can quickly adapt to new scenarios.

[0097] In addition, the mask image generated by the dual-path inspection network and decoder network specific to injection molding panels can efficiently and accurately identify defect locations, providing strong support for production quality control while improving production efficiency and reducing costs.

[0098] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0099] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0100] According to an embodiment of the present invention, there is also provided an appearance defect detection device for electrical equipment for implementing the appearance defect detection method for electrical equipment. Fig. 9 is a schematic diagram of an appearance defect detection device for an electrical device according to an embodiment of the present invention. Fig. 9 As shown, the device includes: a first acquisition module 901, a first processing module 903 and a first determination module 905. The appearance defect detection device of the electrical equipment is described below.

[0101] The first acquisition module 901 is used to acquire a first panel image and a second panel image of an injection molding panel of a target electrical device, wherein the first panel image is a standard panel image without defects, and the second panel image is an image to be inspected for defects.

[0102] The first processing module 903 is used to input the first panel image and the second panel image into the panel appearance defect detection model, so as to use the panel appearance defect detection model to process the first panel image and the second panel image to obtain the appearance defect detection result of the target electrical equipment, wherein the panel appearance defect detection model is a model trained by machine learning using multiple sets of training data, each of the multiple sets of training data includes: a sample panel image and a sample appearance defect detection result corresponding to the sample panel image, the sample panel image includes: a first sample panel image, a second sample panel image and a sample defect image, and the sample defect image is a panel defect image generated based on the first sample panel image.

[0103] The first determination module 905 is used to determine defect information of the injection molding panel of the target electrical device according to the appearance defect detection result.

[0104] It should be noted here that the above-mentioned first acquisition module 901, first processing module 903 and first determination module 905 correspond to steps S202 to S206 in the above-mentioned embodiments, and the three modules have the same instances and application scenarios implemented by the corresponding steps, but are not limited to the contents disclosed in the above-mentioned embodiments.

[0105] As can be seen from the above, in the scheme recorded in the above-mentioned embodiment of the present invention, firstly, the first acquisition module can be used to acquire the first panel image and the second panel image of the injection molding panel of the target electrical equipment, wherein the first panel image is a standard panel image without defects, and the second panel image is an image to be subjected to defect detection; then, the first processing module can be used to input the first panel image and the second panel image into the panel appearance defect detection model, so as to process the first panel image and the second panel image using the panel appearance defect detection model to obtain the appearance defect detection result of the target electrical equipment, wherein the panel appearance defect detection model is a model obtained by machine learning training using multiple sets of training data, each of the multiple sets of training data includes: a sample panel image and a sample appearance defect detection result corresponding to the sample panel image, the sample panel image includes: a first sample panel image, a second sample panel image and a sample defect image, and the sample defect image is a panel defect image generated according to the first sample panel image; finally, the first determination module can be used to determine the defect information of the injection molding panel of the target electrical equipment according to the appearance defect detection result, thereby achieving a network structure combining dual-path detection and a decoder, and through cross-attent ion fuses the features of the two paths and introduces visual cues. At the same time, the mask image generated by the dual-path detection network and decoder network specific to injection molding panels can efficiently and accurately identify the defect location, thereby achieving the technical effects of reducing the demand for negative samples, efficiently detecting defect features, and quickly adapting to new scenarios, and improving the migration ability and accuracy of the detection model.

[0106] The technical solution provided by the embodiment of the present invention solves the technical problem in the related art that the detection capability of unknown defects in the defect detection of the air conditioner is poor, resulting in low accuracy of the defect detection of the air conditioner.

[0107] In an optional embodiment, the first processing module includes: a first processing unit, which is used to perform a standardization processing operation on the first panel image and the second panel image to obtain the first panel image and the second panel image after standardization processing, wherein the standardization processing operation includes: a scaling operation and a filling operation; a second processing unit, which is used to input the first panel image and the second panel image after standardization processing into a panel appearance defect detection model, so as to use the panel appearance defect detection model to process the first panel image and the second panel image after standardization processing to obtain an appearance defect detection result of the target electrical equipment.

[0108] In an optional embodiment, before the first processing module, the appearance defect detection device of the electrical equipment also includes: a second acquisition module, used to acquire multiple groups of training data; a second processing module, used to standardize the multiple groups of training data to obtain multiple groups of standardized training data; a third processing module, used to train the panel appearance defect detection model using the multiple groups of standardized training data.

[0109] In an optional embodiment, the second acquisition module includes: a third processing unit, used to acquire a first sample panel image and a second sample panel image of an injection molding panel of multiple electrical devices including a target electrical device, and to mark defects on the second sample panel image; a fourth processing unit, used to generate a sample defect image based on the first sample panel image, and to mark defects on the sample defect image.

[0110] In an optional embodiment, the fourth processing unit includes: a first determination subunit, used to determine the defect type to be generated of the target electrical equipment; a first processing subunit, used to set defect information of each defect type to be generated according to the first sample panel image, wherein the defect information is used to describe the defects to be generated corresponding to each defect type to be generated; a second processing subunit, used to generate each defect to be generated according to the defect information; and a pixel processing subunit, used to perform pixel processing on each defect to be generated to obtain a sample defect image.

[0111] In an optional embodiment, the first processing subunit includes: a first definition subunit, which is used to define first defect information including the maximum number of dirty areas of dirt-type defects, the size of each dirty area and the dirty color range according to the first sample panel image; a second definition subunit, which is used to define second defect information including the maximum number of scratches of scratch-type defects and the size of each scratch according to the first sample panel image; and a third definition subunit, which is used to define third defect information including the maximum number of concave-convex areas of concave-convex type defects, the size of each concave-convex area and the degree of concave-convex feeling according to the first sample panel image.

[0112] In an optional embodiment, the second processing subunit includes: a third processing subunit, used to generate a dirt-type defect on the first sample panel image according to the first defect information through a random noise algorithm or a random walk algorithm; a fourth processing subunit, used to randomly select a starting point and a scratch direction on the first sample panel image, and generate a scratch-type defect according to the second defect information; a fifth processing subunit, used to generate a concave-convex defect of a predetermined shape on the first sample panel image according to the third defect information through an elliptical or random walk algorithm.

[0113] In an optional embodiment, the pixel processing subunit includes: a sixth processing subunit, used to fill the dirt type defect by randomly selecting colors; a seventh processing subunit, used to fill the scratch type defect with color according to the brightness deviation of the color in a predetermined range of the scratch type defect; an eighth processing subunit, used to determine the pixel change range corresponding to the pixel value when simulating the concave-convex type defect according to the depth change range of the concave-convex type defect, and perform pixel adjustment on the concave-convex type defect according to the pixel change range.

[0114] In an optional embodiment, the third processing module includes: a fifth processing unit, which is used to obtain the occluded part and the unoccluded part of the first sample panel image and the sample defect image by randomly occluding predetermined parts of the first sample panel image and the sample defect image; a sixth processing unit, which is used to input the unoccluded part into the encoder in the panel appearance defect detection model to train the encoder until the encoder can reconstruct the first sample panel image and the sample defect image; a seventh processing unit, which is used to train the two-way detection model and the decoder in the panel appearance defect detection model by using the real defects marked in the second sample panel image and the generated defects corresponding to the sample defect image, until the two-way detection model and the decoder can generate and identify the real defects and the generated defects and generate defect images according to the real defects and the generated defects; a splicing unit, which is used to splice the encoder with the two-way detection model and the decoder to obtain the panel appearance defect detection model.

[0115] In an optional embodiment, the stitching unit includes: a classification subunit, used to classify the sample defect images according to dirt-type defects, scratch-type defects and concave-convex defects, and obtain three different types of sample defect image subsets; a ninth processing subunit, used to use each sample defect image subset to train the visual cue network and decoder in the panel appearance defect detection model, until the panel appearance defect detection model can identify the defect type corresponding to each sample defect image, and obtain the panel appearance defect detection model.

[0116] According to another aspect of an embodiment of the present invention, a system for detecting appearance defects of an electrical device is provided. The system for detecting appearance defects of an electrical device uses any one of the above-mentioned methods for detecting appearance defects of an electrical device.

[0117] According to another aspect of an embodiment of the present invention, a processor is provided, and the processor is used to run a program, wherein when the program is run, any one of the above-mentioned methods for detecting appearance defects of an electrical device is executed.

[0118] According to another aspect of an embodiment of the present invention, a computer program product is provided, including computer instructions, which, when executed by a processor, execute any one of the above-mentioned methods for detecting appearance defects of electrical equipment.

[0119] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein the program executes any one of the above-mentioned methods for detecting appearance defects of an electrical device.

[0120] Optionally, in this embodiment, the computer-readable storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the communication devices in a communication device group.

[0121] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: obtaining a first panel image and a second panel image of an injection molding panel of a target electrical device, wherein the first panel image is a standard panel image without defects, and the second panel image is an image to be subjected to defect detection; inputting the first panel image and the second panel image into a panel appearance defect detection model, so as to process the first panel image and the second panel image using the panel appearance defect detection model to obtain an appearance defect detection result of the target electrical device, wherein the panel appearance defect detection model is a model trained by machine learning using multiple sets of training data, each of the multiple sets of training data includes: a sample panel image and a sample appearance defect detection result corresponding to the sample panel image, the sample panel image includes: a first sample panel image, a second sample panel image and a sample defect image, and the sample defect image is a panel defect image generated based on the first sample panel image; and determining defect information of the injection molding panel of the target electrical device based on the appearance defect detection result.

[0122] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: performing a standardization operation on the first panel image and the second panel image to obtain the first panel image and the second panel image after standardization, wherein the standardization operation includes: a scaling operation and a filling operation; inputting the first panel image and the second panel image after standardization into a panel appearance defect detection model to process the first panel image and the second panel image after standardization using the panel appearance defect detection model to obtain an appearance defect detection result of the target electrical device.

[0123] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: obtaining multiple sets of training data; performing standardization processing on the multiple sets of training data to obtain multiple sets of standardized training data; and training a panel appearance defect detection model using the multiple sets of standardized training data.

[0124] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: obtaining a first sample panel image and a second sample panel image of an injection molding panel of a plurality of electrical devices including a target electrical device, and annotating defects on the second sample panel image; generating a sample defect image based on the first sample panel image, and annotating defects on the sample defect image.

[0125] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: determining a type of defect to be generated of the target electrical device;

[0126] Defect information of each defect type to be generated is set according to the first sample panel image, wherein the defect information is used to describe the defects to be generated corresponding to each defect type to be generated; each defect to be generated is generated according to the defect information; and pixel processing is performed on each defect to be generated to obtain a sample defect image.

[0127] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: defining first defect information including the maximum number of dirty areas of dirt-type defects, the size of each dirty area, and the range of dirty colors according to the first sample panel image; defining second defect information including the maximum number of scratches of scratch-type defects, the size of each scratch according to the first sample panel image; defining third defect information including the maximum number of concave-convex areas of concave-convex type defects, the size of each concave-convex area, and the degree of concave-convex feeling according to the first sample panel image.

[0128] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for performing the following steps: generating a dirt-type defect on the first sample panel image according to the first defect information through a random noise algorithm or a random walk algorithm; randomly selecting a starting point and a scratch direction on the first sample panel image, and generating a scratch-type defect according to the second defect information; generating a concave-convex defect of a predetermined shape on the first sample panel image according to the third defect information through an elliptical or random walk algorithm.

[0129] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for performing the following steps: filling dirty type defects by randomly selecting colors; filling scratch type defects with colors according to the brightness deviation of colors in a predetermined range of scratch type defects; determining a pixel variation range corresponding to pixel values ​​when simulating concave-convex type defects according to the depth variation range of concave-convex type defects, and performing pixel adjustments on the concave-convex type defects according to the pixel variation range.

[0130] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for performing the following steps: obtaining the occluded and unoccluded parts of the first sample panel image and the sample defect image by randomly occluding predetermined parts of the first sample panel image and the sample defect image; inputting the unoccluded parts into the encoder in the panel appearance defect detection model to train the encoder until the encoder can reconstruct the first sample panel image and the sample defect image; training the dual-path detection model and decoder in the panel appearance defect detection model through the real defects marked in the second sample panel image and the generated defects corresponding to the sample defect image until the dual-path detection model and the decoder can generate and identify real defects and generated defects and generate defect images based on the real defects and generated defects; splicing the encoder with the dual-path detection model and the decoder to obtain the panel appearance defect detection model.

[0131] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for performing the following steps: classifying the sample defect images according to dirt-type defects, scratch-type defects, and concave-convex defects to obtain three different types of sample defect image subsets; using each sample defect image subset to train the visual cue network and decoder in the panel appearance defect detection model until the panel appearance defect detection model can identify the defect type corresponding to each sample defect image, thereby obtaining the panel appearance defect detection model.

[0132] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0133] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0135] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0136] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0138] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting appearance defects of electrical equipment, characterized in that: include: Acquire a first panel image and a second panel image of an injection-molded panel of a target electrical device, wherein the first panel image is a standard panel image without defects, and the second panel image is an image to be inspected for defects; Inputting the first panel image and the second panel image into a panel appearance defect detection model, so as to process the first panel image and the second panel image using the panel appearance defect detection model to obtain an appearance defect detection result of the target electrical device, wherein the panel appearance defect detection model is a model trained by machine learning using multiple sets of training data, each of the multiple sets of training data includes: a sample panel image and a sample appearance defect detection result corresponding to the sample panel image, the sample panel image includes: a first sample panel image, a second sample panel image and a sample defect image, and the sample defect image is a panel defect image generated according to the first sample panel image; Defect information of the injection molding panel of the target electrical device is determined according to the appearance defect detection result.

2. The appearance defect detection method of electrical equipment according to claim 1, characterized in that: Inputting the first panel image and the second panel image into a panel appearance defect detection model, and processing the first panel image and the second panel image using the panel appearance defect detection model to obtain an appearance defect detection result of the target electrical device, including: Performing a standardization operation on the first panel image and the second panel image to obtain the first panel image and the second panel image after standardization, wherein the standardization operation includes: a scaling operation and a filling operation; The first panel image and the second panel image after the standardized processing are input into the panel appearance defect detection model, so as to use the panel appearance defect detection model to process the first panel image and the second panel image after the standardized processing to obtain the appearance defect detection result of the target electrical equipment.

3. The appearance defect detection method of electrical equipment according to claim 1, characterized in that: Before inputting the first panel image and the second panel image into a panel appearance defect detection model to process the first panel image and the second panel image using the panel appearance defect detection model to obtain an appearance defect detection result of the target electrical device, the method further includes: Acquire the multiple sets of training data; Performing standardization processing on the multiple sets of training data to obtain the multiple sets of training data after standardization processing; The panel appearance defect detection model is obtained by training the multiple sets of training data after standardization.

4. The method for detecting appearance defects of electrical equipment according to claim 3, characterized in that: Acquiring the multiple sets of training data includes: Acquire the first sample panel image and the second sample panel image of injection-molded panels of multiple electrical devices including the target electrical device, and perform defect annotation on the second sample panel image; The sample defect image is generated according to the first sample panel image, and defects are annotated on the sample defect image.

5. The method for detecting appearance defects of electrical equipment according to claim 4, characterized in that: Generating the sample defect image according to the first sample panel image includes: Determining the type of defect to be generated of the target electrical device; Setting defect information of each defect type to be generated according to the first sample panel image, wherein the defect information is used to describe the defect to be generated corresponding to each defect type to be generated; generating each of the defects to be generated according to the defect information; Pixel processing is performed on each of the defects to be generated to obtain the sample defect image.

6. The method for detecting appearance defects of electrical equipment according to claim 5, characterized in that: The defect types to be generated include: dirt type defects, scratch type defects and concave-convex type defects, and setting defect information of each defect type to be generated according to the first sample panel image includes: Defining first defect information including the maximum number of dirty regions of the dirt-type defect, the size of each of the dirty regions, and the range of dirt color according to the first sample panel image; defining second defect information including a maximum number of scratches of the scratch-type defect and a size of each scratch according to the first sample panel image; The third defect information including the maximum number of concave-convex areas of the concave-convex defect, the size of each concave-convex area and the degree of concave-convex feeling is defined according to the first sample panel image.

7. The method for detecting appearance defects of electrical equipment according to claim 6, characterized in that: Generating each defect to be generated according to the defect information includes: Generating the dirt-type defect on the first sample panel image according to the first defect information by using a random noise algorithm or a random walk algorithm; Randomly selecting a starting point and a scratch direction on the first sample panel image, and generating the scratch type defect according to the second defect information; The concave-convex defect of a predetermined shape is generated on the first sample panel image according to the third defect information by using an elliptical or random walk algorithm.

8. The method for detecting appearance defects of electrical equipment according to claim 6, characterized in that: Performing pixel processing on each of the defects to be generated to obtain the sample defect image includes: Filling the dirt-type defect by randomly selecting a color; Filling the scratch-type defect with color according to the brightness deviation of the color within the predetermined range of the scratch-type defect; The pixel variation range corresponding to the pixel value when simulating the concave-convex defect is determined according to the depth variation range of the concave-convex defect, and the pixel of the concave-convex defect is adjusted according to the pixel variation range.

9. The method for detecting appearance defects of electrical equipment according to claim 3, characterized in that: The panel appearance defect detection model is obtained by training the plurality of sets of training data after standardization, including: Obtaining an obstructed portion and an unobstructed portion of the first sample panel image and the sample defect image by randomly obstructing predetermined portions of the first sample panel image and the sample defect image; Inputting the unobstructed portion into an encoder in the panel appearance defect detection model to train the encoder until the encoder can reconstruct the first sample panel image and the sample defect image; Training a two-way detection model and a decoder in the panel appearance defect detection model by using the real defects annotated in the second sample panel image and the generated defects corresponding to the sample defect image, until the two-way detection model and the decoder are capable of generating and identifying the real defects and the generated defects and generating defect images according to the real defects and the generated defects; The encoder is spliced ​​with the dual-path detection model and the decoder to obtain the panel appearance defect detection model.

10. The appearance defect detection method of electrical equipment according to claim 9, characterized in that: The encoder is connected with the dual-path detection model and the decoder to obtain the panel appearance defect detection model, including: Classifying the sample defect images according to dirt-type defects, scratch-type defects, and concave-convex defects to obtain three different types of sample defect image subsets; The visual cue network and the decoder in the panel appearance defect detection model are trained using each subset of the sample defect images respectively until the panel appearance defect detection model can identify the defect type corresponding to each of the sample defect images, thereby obtaining the panel appearance defect detection model.

11. An appearance defect detection device for electrical equipment, characterized in that: include: A first acquisition module is used to acquire a first panel image and a second panel image of an injection molding panel of a target electrical device, wherein the first panel image is a standard panel image without defects, and the second panel image is an image to be inspected for defects; a first processing module, configured to input the first panel image and the second panel image into a panel appearance defect detection model, so as to process the first panel image and the second panel image using the panel appearance defect detection model to obtain an appearance defect detection result of the target electrical device, wherein the panel appearance defect detection model is a model obtained by machine learning training using multiple sets of training data, each of the multiple sets of training data includes: a sample panel image and a sample appearance defect detection result corresponding to the sample panel image, the sample panel image includes: a first sample panel image, a second sample panel image and a sample defect image, and the sample defect image is a panel defect image generated according to the first sample panel image; The first determination module is used to determine the defect information of the injection molding panel of the target electrical device according to the appearance defect detection result.

12. An appearance defect detection system for electrical equipment, characterized in that: The appearance defect detection system of the electrical equipment uses the appearance defect detection method of the electrical equipment described in any one of claims 1 to 10 above.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the appearance defect detection method of an electrical device according to any one of claims 1 to 10.

14. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by the processor, the appearance defect detection method of the electrical equipment as described in any one of claims 1 to 10 is performed.

Images