Defect sample expansion method and device of industrial product, electronic equipment and medium

Abstract

The invention provides an industrial product defect sample expansion method and device, electronic equipment and a medium, and relates to the technical field of industrial manufacturing, the method comprises the following steps: obtaining process parameters corresponding to a first industrial product, the first industrial product being an actually produced industrial product with defects; simulating the production process of the industrial product through a simulation tool according to the process parameters, and simulating a second industrial product; a plurality of process defect images in the process of simulating the second industrial product are obtained, defect samples are obtained according to the process defect images, the defect samples are used for training a defect detection model for defect detection of the industrial products, and the defect detection model is used for detecting the defects of the industrial products by simulating the forming process of the defects on the first industrial product. The defect samples are obtained according to the multiple process defect images, one defect sample of an original first industrial product is expanded and filled into the multiple defect samples, sample expansion is achieved, the defect detection model is trained through more samples, and therefore the performance of the model is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of industrial manufacturing technology, and in particular to a method, device, electronic device, and medium for expanding defect samples of industrial products. Background Art

[0002] Industrial defect detection refers to the process of detecting defects in industrial products during the industrial production process, so that the products can be repaired or eliminated in a timely manner to ensure product quality and production efficiency. In related technologies, AI (Artificial Intelligence) technology can be used for the above-mentioned defect detection. However, in the actual production process, defective industrial products account for a small number of products, resulting in a small number of defective samples, which leads to poor performance of the trained AI defect detection model. Summary of the Invention

[0003] To overcome the problems existing in the related art, the present disclosure provides a method, device, electronic device and medium for expanding defect samples of industrial products.

[0004] According to a first aspect of an embodiment of the present disclosure, a method for expanding defect samples of industrial products is provided, the method comprising: obtaining process parameters corresponding to a first industrial product, wherein the first industrial product is an actually produced industrial product with defects; simulating the production process of the industrial product according to the process parameters using a simulation tool to simulate a second industrial product; obtaining multiple process defect images in the process of simulating the second industrial product, and obtaining defect samples based on the multiple process defect images, wherein the defect samples are used to train a defect detection model for defect detection of industrial products.

[0005] Optionally, before obtaining multiple process defect images in the process of simulating the second industrial product, it also includes: shooting a real defect image corresponding to the first industrial product, and obtaining a simulated defect image corresponding to the second industrial product; obtaining multiple process defect images in the process of simulating the second industrial product includes: if the real defect image and the simulated defect image are consistent, then obtaining the multiple process defect images in the process of simulating the second industrial product; if the real defect image and the simulated defect image are inconsistent, then adjusting the process parameters, and executing the step of simulating the production process of the industrial product according to the process parameters through the simulation tool to simulate the second industrial product, until the real defect image and the simulated defect image are consistent, and obtaining the multiple process defect images in the process of simulating the second industrial product.

[0006] Optionally, the method also includes: extracting real image features of the real defect image, and extracting simulated image features of the simulated defect image; if the real defect image and the simulated defect image are consistent, obtaining the multiple process defect images in the process of simulating the second industrial product, including: obtaining the similarity between the real image features and the simulated image features; if the similarity is greater than a preset similarity, determining that the real defect image and the simulated defect image are consistent, and obtaining the multiple process defect images in the process of simulating the second industrial product.

[0007] Optionally, obtaining defect samples based on the multiple process defect images includes: segmenting a defect position image corresponding to the defect on each process defect image from the multiple process defect images; obtaining captured historical images from historical data, and combining at least one defect position image with the historical images to obtain the defect samples.

[0008] Optionally, segmenting out a defect position image corresponding to the defect on each process defect image among the multiple process defect images includes: determining the defect position of the defect on each process defect image among the multiple process defect images; and segmenting out a defect position image corresponding to the defect on each process defect image from each process defect image among the multiple process defect images according to the defect position.

[0009] Optionally, determining the defect position of the defect on each of the multiple process defect images includes: determining the defect position of the defect on each of the multiple process defect images through the simulation tool.

[0010] Optionally, segmenting a defect position image corresponding to the defect on each process defect image from each of the multiple process defect images based on the defect position includes: segmenting a defect position image corresponding to the defect on each process defect image from each of the multiple process defect images based on the defect position through a segmentation model.

[0011] Optionally, combining at least one defect location image with the historical image to obtain the defect sample includes: performing style transfer on the at least one defect location image to obtain a defect location image corresponding to the style of the historical image; and combining the defect location image after style transfer with the historical image to obtain the defect sample.

[0012] Optionally, performing style transfer on the at least one defect location image to obtain a defect location image corresponding to the style of the historical image includes: performing style transfer on the at least one defect location image to obtain a defect location image corresponding to the style of the historical image through a style transfer model.

[0013] According to a second aspect of an embodiment of the present disclosure, a defect sample expansion device for an industrial product is provided, the device comprising: an acquisition module for acquiring process parameters corresponding to a first industrial product, wherein the first industrial product is an actually produced industrial product with defects; a simulation module for simulating the production process of the industrial product according to the process parameters using a simulation tool to simulate a second industrial product; an expansion module for acquiring multiple process defect images in the process of simulating the second industrial product, and obtaining defect samples based on the multiple process defect images, wherein the defect samples are used to train a defect detection model for defect detection of industrial products.

[0014] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein, when the processor executes the instructions, the steps of the method described in the first aspect are implemented.

[0015] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the method for expanding defect samples of industrial products provided in the first aspect of the present disclosure are implemented.

[0016] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0017] The present disclosure provides a method, device, electronic device and medium for expanding defect samples of industrial products. The method first obtains process parameters corresponding to a first industrial product, wherein the first industrial product is an industrial product with defects actually produced; then, a simulation tool is used to simulate the production process of the industrial product according to the process parameters to simulate a second industrial product; then, multiple process defect images of the second industrial product are obtained, and defect samples are obtained based on the multiple process defect images, wherein the defect samples are used to train a defect detection model for defect detection of industrial products. Through the simulation process, the formation process of defects on the first industrial product is simulated, and the defects in the formation process can be considered as process defects. Defect samples are obtained based on the multiple process defect images, and the original defect sample of the first industrial product is expanded into multiple defect samples to achieve sample expansion. The defect detection model is trained with more samples, thereby improving the performance of the defect detection model.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0020] Figure 1 is a flow chart showing a method for expanding defect samples of industrial products according to an exemplary embodiment;

[0021] Figure 2 yes Figure 1 Schematic diagram of sub-steps of step S130;

[0022] Figure 3 is a flow chart showing a method for expanding defect samples of industrial products according to another exemplary embodiment;

[0023] Figure 4 is a flow chart showing a method for expanding defect samples of industrial products according to another exemplary embodiment;

[0024] Figure 5 is a flow chart showing a method for expanding defect samples of industrial products according to another exemplary embodiment;

[0025] Figure 6 is a block diagram of a device for expanding defect samples of industrial products according to an exemplary embodiment;

[0026] Figure 7 The figure is a block diagram of an electronic device showing a method for expanding defect samples of industrial products according to an exemplary embodiment. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0028] The embodiments described in the following examples of the present disclosure do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0029] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the corresponding data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0030] Smart manufacturing refers to the use of advanced information technology and intelligent technologies to digitize, automate, and intelligentize the manufacturing process, thereby improving production efficiency, reducing costs, and enhancing product quality and flexibility. In recent years, with the continuous development of technologies such as artificial intelligence, the Internet of Things, and cloud computing, the field of smart manufacturing has also made significant progress. The application of AI technology is particularly prominent in all aspects of smart manufacturing. For example, in production planning and scheduling, AI technology can optimize production plans and scheduling by analyzing and predicting production data, thereby improving production efficiency and resource utilization. In quality control, AI technology can promptly identify and correct production problems and improve product quality through real-time monitoring and analysis of the production process. In equipment maintenance, AI technology can proactively detect equipment failures by analyzing and predicting equipment operating data, reducing downtime and repair costs.

[0031] Smart manufacturing also involves the application of digital technologies, such as digital design, digital manufacturing, and digital services. Digital design uses virtual simulation to proactively identify design issues, reducing the cost and time of trial and error. Digital manufacturing uses digital factories and smart manufacturing equipment to automate and intelligentize production processes. Digital services leverage cloud computing and the Internet of Things (IoT) to enable remote monitoring and maintenance of products, improving product reliability and service levels.

[0032] In short, intelligent manufacturing is a comprehensive field involving multiple technologies and processes. AI technology currently has some practical applications in production process optimization, including casting production, metal heat treatment technology, and defect detection.

[0033] In intelligent manufacturing, industrial defect detection refers to the process of detecting defects in industrial products during the production process, so that the products can be repaired or eliminated in a timely manner to ensure product quality and production efficiency. With the development of industrial automation and intelligence, industrial defect detection technology has also developed rapidly. Industrial defect detection has the following significance:

[0034] 1. Improvement of production efficiency: Industrial defect detection can help companies detect defects in products in a timely manner, avoid production stagnation and losses caused by defects, and improve production efficiency.

[0035] 2. Product quality assurance: Industrial defect detection can help companies detect defects in products in a timely manner, prevent defective products from entering the market, and ensure product quality.

[0036] 3. Manual inspection is difficult to meet the needs: With the expansion of production scale, manual inspection can no longer meet the needs, and automation and AI technology are needed to improve inspection efficiency and accuracy.

[0037] 4. Technological development: With the development of technologies such as computer vision, machine learning, and deep learning, industrial defect detection technology has also developed rapidly, and can detect defects in products more accurately.

[0038] In the related art, there are several ways to do defect detection: outlier detection method, target detection method, GAN (Generative Adversarial Networks). Among them, GAN uses the error of the intermediate layer to locate defects, or uses the method of repairing the image first and then comparing it with the original image to locate the defect position (including using the Pix2pix method for reconstruction), and uses a newly generated image that is very close to the defect-free part of the original image, and then subtracts it from the original image to locate the defect. In the related art, by taking images of defective industrial products, defect samples are obtained and used for the training of the above-mentioned AI models (or networks). However, in the actual production process, defective industrial products account for a minority, which makes the number of defective samples small, and limits the accuracy of defect detection in the above-mentioned methods.

[0039] When using AI technology for defect detection in smart manufacturing, a significant issue arises: the aforementioned issue of a small number of defect samples. A factory's production goal is to achieve a high yield at a low cost and quickly optimize production processes based on defect data. This further reduces the defect rate, and the number of defect samples will naturally decrease accordingly. AI models rely on a large sample size to ensure performance. Generating a large number of defective samples violates production goals, and production will inevitably be halted when there are a large number of defective samples. High yield is the ultimate goal of factory production. This is especially true for new parts, as production time is short, making it impossible to accumulate a large number of negative samples in the early stages of production. Consequently, a small number of defective samples can lead to poor performance in trained AI defect detection models.

[0040] To solve the above problems, the present disclosure provides a method for expanding defect samples of industrial products. Figure 1 The defect sample expansion method of industrial products can be applied to Figure 6 The defect sample expansion device 300 of the industrial product shown, Figure 7 The electronic device 800, computer readable storage medium and computer program product shown in FIG. In this embodiment, the electronic device is used as an example, and the electronic device can be an industrial computer. Figure 1The process shown in FIG. 1 is described in detail. The method for expanding defect samples of industrial products may specifically include the following steps:

[0041] Step S110: Acquire process parameters corresponding to a first industrial product, wherein the first industrial product is an industrial product actually produced with defects.

[0042] The industrial product in this embodiment can be as large as a complete device. For example, the industrial product can be a smartphone, tablet computer, smart bracelet, CNC equipment, household appliances, energy-saving devices, internal combustion engines, pharmaceutical equipment, hardware tools, measuring instruments, boiler power equipment, papermaking equipment, etc. The industrial product in this embodiment can also be as small as a specific component on the equipment. For example, the industrial product can be a screw, nut, etc. The first industrial product in this embodiment refers to an industrial product with defects in actual production. Defects can take the form of pores, cracks, sand inclusions, sand holes, folds, scars, faults, misalignment, bends, etc.

[0043] A first industrial product with a defect is selected from the produced industrial products, and process parameters corresponding to the first industrial product are obtained. For example, the process parameters set during the production of the first industrial product are retrieved from an industrial computer. The process parameters may include at least one of air entrainment, air pressure, liquid percentage in the mold, solidified liquid percentage, material age, mold filling temperature, mold filling speed, mold filling time, and cooling rate.

[0044] In one embodiment, an image of the produced industrial product is captured by a camera device, and then the image is detected using a preset defect detection algorithm to detect a first industrial product with defects among the industrial products.

[0045] In another embodiment, among the produced industrial products, a first industrial product with defects is detected through manual screening.

[0046] It should be noted that the image of the first industrial product obtained by shooting can be used as a defect sample. For example, for a first industrial product with only one defect, the image obtained by shooting can be used as a defect sample.

[0047] Step S120: Using a simulation tool, simulate the production process of the industrial product according to the process parameters to simulate a second industrial product.

[0048] In the related art, the final defect sample is obtained by photographing the first industrial product that has been formed. However, since the number of first industrial products with defects is limited, the number of defect samples is limited. Therefore, in the present disclosure, a simulation tool is installed in the electronic device, and the simulation tool is used to simulate the production process of the industrial product according to the process parameters, simulate the production process of the first industrial product, and simulate the second industrial product. It should be noted that the second industrial product is simulated by the simulation tool, and is not a product produced in the actual production process. It is understandable that by using the simulation tool, according to the process parameters corresponding to the first industrial product, the production process of the first industrial product is simulated, the production process of the first industrial product is reproduced, and naturally the defect formation process is also reproduced. In this process, the defects continue to change and develop to form the final defects. The final defects on the second industrial product are consistent with the defects on the first industrial product. For the sake of distinction, the defects in the development process are called process defects.

[0049] Among them, simulation tools can be used to simulate and analyze the behavior and performance of various systems, and are widely used in various fields such as engineering design, product development, risk assessment, education, and research. They can help users better understand and master the behavior and performance of various systems and improve work efficiency and work quality. For example, simulation tools include Magma, SageMath, Mathematica, Maple, GAP, Singular, etc., among which Magma is a software engaged in injection molding simulation, focusing on the simulation and optimization of thermoplastics, silicone rubber, metal / ceramic powders and semi-solid products. It is a computer algebra system used in fields such as algebraic geometry, algebraic number theory and computational group theory. Magmat casting simulation software is increasingly widely used in casting production. Its main function is to predict casting defects and ensure product quality. Especially in the product development stage, it can greatly shorten the development cycle and is one of the indispensable tools for casting technicians. Therefore, the simulation tool of this embodiment can be Magma.

[0050] Step S130: Acquire multiple process defect images during the simulation of the second industrial product, and obtain defect samples based on the multiple process defect images, wherein the defect samples are used to train a defect detection model for defect detection of industrial products.

[0051] Multiple process defect images corresponding to process defects during the simulation process are obtained, for example, by automatically exporting multiple process defect images through simulation software. Defect samples are then obtained based on the multiple process defect images, thereby expanding the defect sample. For example, in related art, photographing a first industrial product with only one defect only yields one defect sample. In this embodiment, by reproducing the defect formation process on the first industrial product, multiple intermediate process defects are reproduced. Each image corresponding to the multiple process defects can be used as a defect sample, resulting in multiple defect samples and expanding the sample.

[0052] In one embodiment, an image corresponding to each of the multiple process defects is used as a defect sample, thereby obtaining rich defect samples.

[0053] In another embodiment, among multiple process defects, process defects at adjacent moments have a high degree of similarity. If process defect images at two consecutive moments are selected as defect samples and used to train a defect detection model, the resulting defect detection model may suffer from shortcomings such as overfitting, poor generalization, and insufficient robustness. Therefore, if process defect images obtained at adjacent moments have a high degree of defect similarity, some process defect images can be discarded. For example, one process defect image can be selected as a defect sample at every preset time interval, thereby increasing the difference between each defect sample and obtaining a defect detection model with better performance.

[0054] After the above sample expansion, more defect samples are obtained. Using the defect samples to train the defect detection model can improve the performance of the defect detection model, so that the trained defect detection model can more accurately identify defects on industrial products.

[0055] This embodiment provides a method for expanding defect samples of industrial products. The method first obtains process parameters corresponding to a first industrial product, wherein the first industrial product is an industrial product with defects actually produced; then, a simulation tool is used to simulate the production process of the industrial product according to the process parameters to simulate a second industrial product; then, multiple process defect images in the process of simulating the second industrial product are obtained, and defect samples are obtained based on the multiple process defect images, wherein the defect samples are used to train a defect detection model for defect detection of industrial products. Through the simulation process, the formation process of defects on the first industrial product is simulated. The defects in the formation process can be considered as process defects. Defect samples are obtained based on the multiple process defect images, and the original defect sample of the first industrial product is expanded into multiple defect samples to achieve sample expansion. The defect detection model is trained with more samples, thereby improving the performance of the defect detection model.

[0056] Optionally, before step S130, the defect sample expansion method of the industrial product also includes: shooting a real defect image corresponding to the first industrial product, wherein the real defect image includes the defects on the first industrial product, for example, taking a picture of the position where the defect of the first industrial product image is located by a camera device to obtain a real defect image; and obtaining a simulated defect image corresponding to the second industrial product, wherein the simulated defect image includes the defects on the second industrial product, for example, through a simulation tool, exporting the simulated defect image at the defect position of the second industrial product obtained by the final simulation.

[0057] Based on this, in one embodiment, acquiring multiple process defect images during the simulation of the second industrial product in step S130 includes: if the real defect image and the simulated defect image are consistent, acquiring the multiple process defect images during the simulation of the second industrial product. Alternatively, if the real defect image and the simulated defect image are inconsistent, adjusting the process parameters and executing step S120 until the real defect image and the simulated defect image are consistent, thereby acquiring the multiple process defect images during the simulation of the second industrial product.

[0058] Whether the real defect image and the simulated defect image are consistent can be determined by extracting real image features of the real defect image and extracting simulated image features of the simulated defect image in the following manner. A similarity between the real image features and the simulated image features is obtained; if the similarity is greater than a preset similarity, the real defect image and the simulated defect image are determined to be consistent, and the multiple process defect images of the process of simulating the second industrial product are obtained. Conversely, if the similarity is less than or equal to the preset similarity, the real defect image and the simulated defect image are determined to be inconsistent, and simulation parameters are adjusted to execute step S120.

[0059] In this embodiment, the real defect image corresponding to the real first industrial product is compared with the simulated defect image corresponding to the simulated second industrial product. If the two are consistent, then the multiple process defect images generated in the process of simulating the second industrial product can be considered to be consistent with the images in the real production process, and then the multiple process defect images are used as defect samples, thereby improving the authenticity of the defect samples.

[0060] In one embodiment, in step S130, defect samples are obtained based on the plurality of process defect images. Figure 2 , including the following steps:

[0061] Step S131 : Segmenting each of the plurality of process defect images to obtain a defect position image corresponding to the defect on each process defect image.

[0062] As one approach, a defect position of a defect on each of the plurality of process defect images is determined, and then, based on the defect position, a defect position image corresponding to the defect on each of the plurality of process defect images is segmented from each of the plurality of process defect images.

[0063] Exemplarily, the simulation tool is used to determine a defect position of a defect on each of the plurality of process defect images.

[0064] Exemplarily, a segmentation model is deployed in the electronic device. The segmentation model segments, based on the defect location, each of the multiple process defect images into a defect location image corresponding to the defect in each process defect image. The segmentation model is pre-trained and may be, but is not limited to, an FCN (Fully Convolutional Networks), a PSPNet (Pyramid Scene Parsing Network), or a U-Net (U-shaped Network).

[0065] Step S132: Acquire captured historical images from historical data, and combine at least one defect location image with the historical images to obtain the defect sample.

[0066] The historical data stores captured images. The historical images may be images of defective industrial products, such as the first industrial product captured, or images of other defective industrial products. The historical images may also be images of non-defective industrial products.

[0067] Since the defect location image obtained by simulation includes the image style of the simulation tool, for example, the defect location image contains a grid, a black background, a white background, color annotations, etc., there are certain differences between the defect location image and the actual image. Therefore, the defect location image can be subjected to image migration processing. As a method, the style of the at least one defect location image is transferred to a defect location image corresponding to the style of the historical image, so that the style of the transferred defect location image is closer to the style of the actual image; the defect location image after style transfer is combined with the historical image to obtain the defect sample. The style of the defect location image after style transfer is consistent with the style of the historical image obtained by shooting. After the two are combined, a more realistic defect sample can be obtained.

[0068] Exemplarily, a style transfer model is deployed in the electronic device, and the style transfer model is used to perform style transfer on the at least one defect location image to obtain a defect location image corresponding to the style of the historical image.

[0069] Optionally, the simulated defect image corresponding to the second industrial product obtained by simulation can be used as the input of the initial model, and the real defect image corresponding to the first industrial product can be taken as the output of the initial model. The initial model is trained to obtain a style transfer model.

[0070] In another embodiment, the defect samples are obtained according to the multiple process defect images in step S130 in the following manner: the multiple process defect images are directly used as defect samples.

[0071] Optionally, the present disclosure also provides a method for expanding defect samples of industrial products, see Figure 3 , the method comprises the following steps:

[0072] Step S210: Perform simulation reconstruction based on actual defects.

[0073] Step S220: Generate a process defect image.

[0074] Step S230: synthesize defective samples.

[0075] Optionally, the present disclosure also provides a method for expanding defect samples of industrial products, see Figure 4 The method includes the following: the historical data includes historical images, which refer to images of manufactured industrial products, including images of actual defects corresponding to the first industrial product. The historical data also includes industrial parameters corresponding to the historical images. Defect features are extracted for the first industrial product in the historical data to obtain actual image features. Parameters are set based on the historical data, for example, based on the industrial parameters in the historical images. Based on these parameters, a simulation test is conducted using a simulation tool to simulate a second industrial product. Simulation process extraction is then performed to extract process defect images and obtain simulated defect images corresponding to the second industrial product. Feature extraction is performed on the simulated defect images to obtain simulated image features. The two features are then compared to determine whether they are consistent. If they are consistent, a defect sample is generated. For example, multiple process defect images are obtained during the simulation of the second industrial product, and a defect sample is generated based on at least one process defect image. Conversely, if the two features are inconsistent, the parameter settings are readjusted.

[0076] Optionally, the present disclosure also provides a method for expanding defect samples of industrial products, see Figure 5 , the method comprises the following:

[0077] Simulation tools are used to obtain simulation process data. Process defect images are extracted from the simulation process data and then defect location is performed, for example, using the simulation tool. Defect segmentation is then performed to obtain defect location images. Style transfer is then performed on the defect location images to obtain style-transferred defect location images. The historical image and the style-transferred defect location images are combined to obtain defect samples.

[0078] In the defect sample expansion method for industrial products provided in this embodiment, real experimental results are used for re-simulation, and the data of the defect formation process in the simulation process is used to supplement the defect sample volume. The real results are used to supervise the simulation parameter configuration, reproduce the defect growth process, and form defects of different forms. Based on the style transfer technology, the simulation results are used to generate real defect forms, providing a large number of defect samples for defect detection.

[0079] in, Figures 3 to 5 For the detailed description of the steps, please refer to the above embodiments and will not be repeated here.

[0080] Based on the same inventive concept, the present disclosure provides a defect sample expansion device for industrial products, see Figure 6 The defect sample expansion device 300 of the industrial product includes:

[0081] An acquisition module 310 is configured to acquire process parameters corresponding to a first industrial product, wherein the first industrial product is an industrial product actually produced with defects;

[0082] A simulation module 320 is configured to simulate the production process of the industrial product according to the process parameters using a simulation tool to simulate a second industrial product;

[0083] The expansion module 330 is used to obtain multiple process defect images in the process of simulating the second industrial product, and obtain defect samples based on the multiple process defect images, wherein the defect samples are used to train a defect detection model for defect detection of industrial products.

[0084] Optionally, the defect sample expansion device 300 for industrial products further includes:

[0085] a shooting module, configured to shoot a real defect image corresponding to the first industrial product, and obtain a simulated defect image corresponding to the second industrial product;

[0086] The expansion module 330 includes:

[0087] A first expansion module is configured to obtain the plurality of process defect images during the process of simulating the second industrial product if the real defect image is consistent with the simulated defect image;

[0088] The second expansion module is used to adjust the process parameters if the real defect image and the simulated defect image are inconsistent, and execute the steps of simulating the production process of the industrial product according to the process parameters through the simulation tool to simulate the second industrial product until the real defect image and the simulated defect image are consistent, and obtain the multiple process defect images in the process of simulating the second industrial product.

[0089] Optionally, the defect sample expansion device 300 for industrial products further includes:

[0090] The feature extraction module is used to extract the real image features of the real defect image and the simulated image features of the simulated defect image.

[0091] The first expansion module includes:

[0092] A similarity acquisition module, configured to acquire the similarity between the real image features and the simulated image features;

[0093] A similarity judgment module is used to determine that the real defect image and the simulated defect image are consistent if the similarity is greater than a preset similarity, and obtain the multiple process defect images in the process of simulating the second industrial product.

[0094] Optionally, the expansion module 330 includes:

[0095] a segmentation module, configured to segment, from each of the plurality of process defect images, a defect position image corresponding to the defect on each process defect image;

[0096] The combining module is used to obtain the captured historical images from the historical data, and combine at least one defect location image with the historical images to obtain the defect sample.

[0097] Optionally, the segmentation module includes:

[0098] a defect locating module, configured to determine a defect position of a defect on each of the plurality of process defect images;

[0099] The defect segmentation module is used to segment, from each of the multiple process defect images, a defect location image corresponding to the defect on each process defect image according to the defect location.

[0100] Optionally, the defect location module includes:

[0101] The simulation positioning module is used to determine the defect position of the defect on each of the multiple process defect images by using the simulation tool.

[0102] Optionally, the defect segmentation module includes:

[0103] The model segmentation module is used to segment the defect position image corresponding to the defect on each process defect image from each process defect image of the multiple process defect images according to the defect position by using the segmentation model.

[0104] Optionally, the combination module includes:

[0105] a migration module, configured to perform style migration on the at least one defect location image to obtain a defect location image corresponding to the style of the historical image;

[0106] The defect sample acquisition module is used to combine the defect location image after style transfer with the historical image to obtain the defect sample.

[0107] Optionally, the migration module includes:

[0108] The style transfer module is used to perform style transfer on the at least one defect location image through a style transfer model to transfer the style into a defect location image corresponding to the style of the historical image.

[0109] Regarding the defect sample expansion device 300 for industrial products in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0110] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon. When the program instructions are executed by a processor, the steps of the defect sample expansion method for industrial products provided by the present disclosure are implemented.

[0111] Figure 7 This is a block diagram of an electronic device illustrating a method for expanding defect samples for industrial products according to an exemplary embodiment. For example, electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0112] Please refer to Figure 7 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output interface 812 , a sensor component 814 , and a communication component 816 .

[0113] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

[0114] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0115] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0116] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0117] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0118] The input / output interface 812 provides an interface between the processing component 802 and peripheral interface modules, such as a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0119] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor assembly 814 can also detect changes in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and temperature changes of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0120] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0121] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0122] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the electronic device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0123] In another exemplary embodiment, a computer program product is provided. The computer program product includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above-mentioned defect sample expansion method for an industrial product when executed by the programmable device.

[0124] It should be understood that, unless otherwise specifically noted, the features of the various embodiments of the present disclosure described herein may be combined with each other. As used herein, the term "and / or" includes any one of the relevant listed items and any combination of any two or more thereof; similarly, "at least one of" includes any one of the relevant listed items and any combination of any two or more thereof.

[0125] Although terms such as "first", "second" and "third" may be used herein to describe various components, parts, regions, layers or sections, these components, parts, regions, layers or sections are not limited to these terms. On the contrary, these terms are only used to distinguish one component, part, region, layer or section from another component, part, region, layer or section. Therefore, without departing from the teachings of each example, the first component, part, region, layer or section mentioned in the examples described herein may also be referred to as the second component, part, region, layer or section. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" can explicitly or implicitly include at least one such feature. In the description herein, the meaning of "multiple" is at least two, for example, two, three, etc., unless otherwise clearly and specifically defined.

[0126] Furthermore, the word "exemplary" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as advantageous over other aspects or designs. Rather, the use of the word exemplary is intended to present concepts in a concrete manner. As used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X applies to A or B" is intended to mean any of the natural inclusive permutations. That is, if X applies to A; X applies to B; or X applies to both A and B, then "X applies to A or B" satisfies any of the aforementioned instances. Furthermore, the articles "a" and "an," as used in this application and the appended claims, are generally understood to mean "one or more," unless otherwise specified or clear from the context to refer to the singular form.

[0127] Likewise, although the present disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art after reading and understanding the specification and drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terms used to describe such components are intended to correspond to any component (functionally equivalent) that performs the specific functions of the described components, even if structurally not equivalent to the disclosed structures. In addition, although specific features of the present disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations as may be desired and beneficial for any given or specific application. In addition, with respect to the terms "including," "having," "having," "having," or variations thereof used in the specific embodiments or claims, such terms are intended to be inclusive in a manner similar to the term "comprising."

[0128] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the appended claims.

[0129] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for expanding defect samples of industrial products, characterized in that: The method comprises: Obtaining process parameters corresponding to a first industrial product, wherein the first industrial product is an industrial product actually produced with defects; Simulating the production process of the industrial product according to the process parameters using a simulation tool to simulate a second industrial product; A plurality of process defect images in a process of simulating the second industrial product are acquired, and defect samples are obtained based on the plurality of process defect images, wherein the defect samples are used to train a defect detection model for defect detection of industrial products.

2. The method according to claim 1, characterized in that Before acquiring a plurality of process defect images during the simulation of the second industrial product, the method further includes: Taking a real defect image corresponding to the first industrial product, and obtaining a simulated defect image corresponding to the second industrial product; The acquiring of a plurality of process defect images during the simulation of the second industrial product includes: If the real defect image is consistent with the simulated defect image, acquiring the plurality of process defect images during the simulation of the second industrial product; If the real defect image and the simulated defect image are inconsistent, the process parameters are adjusted, and the steps of simulating the production process of the industrial product according to the process parameters through the simulation tool to simulate the second industrial product are executed until the real defect image and the simulated defect image are consistent, and the multiple process defect images in the process of simulating the second industrial product are obtained.

3. The method according to claim 2, characterized in that The method further comprises: extracting real image features of the real defect image, and extracting simulated image features of the simulated defect image; If the real defect image is consistent with the simulated defect image, obtaining the plurality of process defect images during the simulation of the second industrial product includes: Obtaining similarity between the real image features and the simulated image features; If the similarity is greater than a preset similarity, it is determined that the real defect image and the simulated defect image are consistent, and the multiple process defect images in the process of simulating the second industrial product are obtained.

4. The method according to claim 1, wherein The obtaining of defect samples according to the plurality of process defect images includes: Segmenting, from each of the plurality of process defect images, a defect position image corresponding to the defect on each process defect image; The captured historical images are acquired from the historical data, and at least one defect location image is combined with the historical images to obtain the defect sample.

5. The method according to claim 4, characterized in that The step of segmenting each of the plurality of process defect images to obtain a defect position image corresponding to the defect on each process defect image includes: determining a defect position of a defect on each of the plurality of process defect images; According to the defect position, a defect position image corresponding to the defect on each process defect image is segmented from each process defect image of the plurality of process defect images.

6. The method according to claim 5, characterized in that The determining a defect position of a defect on each of the plurality of process defect images includes: The simulation tool is used to determine a defect position of a defect on each of the plurality of process defect images.

7. The method according to claim 5, characterized in that The step of segmenting, from each of the plurality of process defect images, a defect location image corresponding to the defect on each process defect image according to the defect location, includes: By using the segmentation model, according to the defect position, a defect position image corresponding to the defect on each process defect image is segmented from each process defect image of the plurality of process defect images.

8. The method according to claim 4, characterized in that The combining of at least one defect location image with the historical image to obtain the defect sample includes: performing style migration on the at least one defect location image to obtain a defect location image corresponding to the style of the historical image; The defect location image after style transfer is combined with the historical image to obtain the defect sample.

9. The method according to claim 8, characterized in that The performing style transfer on the at least one defect location image to obtain a defect location image corresponding to the style of the historical image includes: The style of the at least one defect location image is transferred using a style transfer model to obtain a defect location image corresponding to the style of the historical image.

10. A defect sample expansion device for industrial products, characterized in that: The device comprises: an acquisition module, configured to acquire process parameters corresponding to a first industrial product, wherein the first industrial product is an industrial product actually produced with defects; a simulation module, configured to simulate the production process of the industrial product according to the process parameters using a simulation tool to simulate a second industrial product; An expansion module is used to obtain multiple process defect images in the process of simulating the second industrial product, and obtain defect samples based on the multiple process defect images, wherein the defect samples are used to train a defect detection model for defect detection of industrial products.

11. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, when the processor executes the instruction, the steps of the method described in any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the method according to any one of claims 1 to 9 when the computer program is executed by a processor.

Images