Product defect identification method and device and electronic equipment

Abstract

The invention relates to a product defect identification method and device and electronic equipment, and the method comprises the steps: obtaining a product image collected after processing and process parameter information collected based on a product processing process, and the process parameter information is temperature or sound; inputting the product image into a set first defect detection model to obtain a first processing defect detection result, and inputting the process parameter information into a set second defect detection model to obtain a second processing defect detection result; and determining a product defect identification result according to the first processing defect detection result and the second processing defect detection result.

Description

Technical Field

[0001] This disclosure relates to the field of product defect detection technology, and more specifically, to a product defect identification method, apparatus, and electronic device. Background Technology

[0002] In modern factory production, product quality inspection is crucial. Traditional product quality inspection relies heavily on manual inspection or simple automated inspection equipment. Manual inspection is not only inefficient but also easily influenced by worker subjectivity, leading to inconsistent and inaccurate results. While simple automated inspection equipment can improve inspection speed to some extent, it lacks flexibility and adaptability in the face of complex and diverse product defects and constantly changing production environments, making it unable to accurately identify new or complex defect types.

[0003] With the continuous expansion of factory production scale and the increasing demands for product quality, existing product defect detection methods are no longer sufficient to meet actual production needs. Traditional detection methods struggle to identify these problems in a timely and accurate manner, leading to defective products entering the market and impacting corporate reputation and economic benefits. Therefore, developing an efficient, accurate, and adaptable product defect identification method is an urgent practical need. Summary of the Invention

[0004] One objective of this invention is to provide a new technical solution for a product defect identification method.

[0005] According to a first aspect of the present invention, a product defect identification method is provided, comprising:

[0006] Acquire product images after processing and process parameter information collected during product processing, wherein the process parameter information is temperature or sound;

[0007] The product image is input into a set first defect detection model to obtain a first processing defect detection result, and the process parameter information is input into a set second defect detection model to obtain a second processing defect detection result;

[0008] Based on the first processing defect detection result and the second processing defect detection result, the product defect identification result is determined.

[0009] Optionally, the process parameter information is time-series parameters collected during the product processing.

[0010] Optionally, the method further includes:

[0011] When the product type corresponding to the product image is a newly introduced product type, the first defect detection model is retrained using the first image training sample set to obtain a first defect detection model for identifying defects in products corresponding to the newly introduced product type.

[0012] When the process parameter information is temperature and the product type corresponding to the process parameter information is a newly introduced product type, the set second defect detection model is retrained using the first temperature training sample set to obtain a second defect detection model for identifying whether an abnormality occurs in the processing process based on the newly introduced product type.

[0013] When the process parameter information is sound and the product type corresponding to the process parameter information is a newly introduced product type, the set second defect detection model is retrained using the first sound training sample set to obtain a second defect detection model for identifying whether an abnormality occurs in the processing process based on the newly introduced product type.

[0014] Optionally, before acquiring the product image collected after processing and the process parameter information collected based on the product processing process, the method further includes:

[0015] Obtain a second image training sample set, wherein each sample in the second image training sample set includes a product image corresponding to the product after processing and first annotation information. The first annotation information includes whether the product has defects and, if the product has defects, the location information and type information of the product defects.

[0016] Each sample in the second image training sample set is input into the first defect detection model to be trained for training, and the product defect detection information corresponding to each sample is obtained.

[0017] Based on the first annotation information corresponding to each sample and the product defect detection information corresponding to each sample, the defect identification accuracy value of the first defect detection model to be trained is determined.

[0018] If the defect identification accuracy value of the first defect detection model to be trained is not greater than the first preset threshold, the parameters of the first defect detection model to be trained are adjusted until the defect identification accuracy value of the first defect detection model to be trained is greater than the first preset threshold.

[0019] Optionally, before acquiring the product image collected after processing and the process parameter information collected based on the product processing, if the process parameter information is temperature, the method further includes:

[0020] Obtain a second temperature training sample set, wherein each sample in the second temperature training sample set includes temperature collected based on the processing process and second annotation information, the second annotation information includes first actual information of the processing process status, the first actual information of the processing process status being either an abnormality in the processing process or no abnormality in the processing process;

[0021] Each sample in the second temperature training sample set is input into the second defect detection model to be trained for training, and the first processing state detection information corresponding to each sample is obtained.

[0022] Based on the actual information of the first processing state corresponding to each sample and the detection information of the first processing state corresponding to each sample, the detection accuracy value of the second defect detection model to be trained is determined.

[0023] If the detection accuracy value of the second defect detection model to be trained is not greater than the second preset threshold, the parameters of the second defect detection model to be trained are adjusted until the detection accuracy value of the second defect detection model to be trained is greater than the second preset threshold.

[0024] Optionally, before acquiring the product image collected after processing and the process parameter information collected based on the product processing process, if the process parameter information is sound, the method further includes:

[0025] Obtain a second sound training sample set, wherein each sample in the second sound training sample set includes sound collected based on the processing process and third annotation information, wherein the third annotation information includes actual information on the state of the second processing process, and the actual information on the state of the second processing process indicates that the processing process is abnormal or that the processing process is not abnormal.

[0026] Each sample in the second sound training sample set is input into the second defect detection model to be trained to obtain the second processing state detection information corresponding to each sample;

[0027] Based on the actual information of the second processing state corresponding to each sample and the detection information of the second processing state corresponding to each sample, the detection accuracy value of the second defect detection model to be trained is determined.

[0028] If the detection accuracy value of the second defect detection model to be trained is not greater than the second preset threshold, the parameters of the second defect detection model to be trained are adjusted until the detection accuracy value of the second defect detection model to be trained is greater than the second preset threshold.

[0029] Optionally, determining the product defect identification result based on the first processing defect detection result and the second processing defect detection result includes:

[0030] If the first processing defect detection result indicates that the product has a defect and / or the first processing defect detection result indicates that the processing process is abnormal, then the product defect identification result is determined to be that the product has a defect.

[0031] According to a second aspect of the present invention, a product defect identification device is provided, comprising:

[0032] The acquisition module is used to acquire product images collected after processing and process parameter information collected based on the product processing process, wherein the process parameter information is temperature or sound;

[0033] The defect detection model detection result determination module is used to input the product image into a set first defect detection model to obtain a first processing defect detection result, and to input the process parameter information into a set second defect detection model to obtain a second processing defect detection result;

[0034] The product defect identification result determination module is used to determine the product defect identification result based on the first processing defect detection result and the second processing defect detection result.

[0035] According to a third aspect of the present invention, a product defect identification device is provided, comprising a memory and a processor, the memory storing a computer program for controlling the processor to operate in order to perform the method according to any one of the first aspects.

[0036] According to a fourth aspect of the present invention, an electronic device is provided, comprising: a product defect identification device as provided in the second or third aspect, a camera for acquiring product images, and a temperature sensor for acquiring temperature, wherein the camera for acquiring product images and the temperature sensor for acquiring temperature are both connected to the product defect identification device; or, comprising: a product defect identification device as provided in the second or third aspect, a camera for acquiring product images, and a sound sensor for acquiring sound, wherein the camera for acquiring product images and the sound sensor for acquiring sound are both connected to the product defect identification device.

[0037] This invention provides a product defect identification method. It utilizes a first defect detection model to obtain a first processing defect detection result based on product image recognition, and a second defect detection model to obtain a second processing defect detection result based on process parameter information collected during product processing. Then, based on the first and second processing defect detection results, the final product defect identification result is determined. Both the first and second defect detection models are artificial intelligence models trained with a large amount of sample data, enabling rapid processing of large volumes of data and real-time rapid product detection. Compared to manual inspection, the detection speed is significantly improved, effectively increasing the inspection efficiency of products on the production line and meeting the needs of large-scale production. Furthermore, the first and second defect detection models, trained with a large amount of sample data, can learn different defect characteristics of different products, overcoming the subjectivity of manual inspection and the limitations of simple inspection equipment, greatly improving the accuracy and consistency of the detection results, and enhancing overall detection accuracy.

[0038] The features and advantages of the embodiments of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0039] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of these embodiments.

[0040] Figure 1 This is a flowchart illustrating a product defect identification method according to an embodiment of the present invention.

[0041] Figure 2 This is a schematic block diagram of a product defect identification device according to an embodiment of the present invention.

[0042] Figure 3 This is a schematic diagram of a product defect identification device according to an embodiment of the present invention. Detailed Implementation

[0043] Various exemplary embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0044] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the embodiments of this specification or their application or use.

[0045] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0046] To address the aforementioned technical problems, this invention provides a product defect identification method. The method utilizes a first defect detection model to obtain a first processing defect detection result based on product image recognition, and a second defect detection model to obtain a second processing defect detection result based on process parameter information collected during product processing. Then, based on the first and second processing defect detection results, the final product defect identification result is determined. Both the first and second defect detection models are artificial intelligence models trained with a large amount of sample data, enabling rapid processing of large volumes of data and real-time rapid product detection. Compared to manual inspection, the detection speed is significantly improved, effectively increasing the inspection efficiency of products on the production line and meeting the needs of large-scale production. Furthermore, the first and second defect detection models, trained with a large amount of sample data, can learn different defect characteristics of different products, overcoming the subjectivity of manual inspection and the limitations of simple inspection equipment, greatly improving the accuracy and consistency of the detection results, and enhancing overall detection accuracy.

[0047] In one embodiment of the present invention, a product defect identification method is provided. According to... Figure 1 As shown, the product defect identification method of this embodiment includes the following steps S110 to S130.

[0048] Step S110: Obtain the product image after processing and the process parameter information collected based on the product processing process, wherein the process parameter information is temperature or sound.

[0049] The product images captured after processing are obtained by a camera.

[0050] The product image acquired after processing can be one or multiple images. If there are multiple product images, the different product images are taken from different shooting angles for the same product.

[0051] In this embodiment, the process parameter information is based on time-series parameters collected during the product processing, that is, a parameter sequence is generated based on the process parameters collected at continuous acquisition times. Since the process parameter information is a parameter sequence obtained based on time-series acquisition, this parameter sequence records the parameters throughout the entire product processing process. These parameters throughout the product processing process are related to product quality. Therefore, identifying whether a product has defects based on the process parameter information achieves defect detection from another inspection perspective, improving the accuracy of product quality inspection.

[0052] When the process parameter information is temperature, the temperature collected during the product processing is obtained by a temperature sensor. The temperature collected during the product processing is a temperature sequence.

[0053] When the process parameter information is sound, the sound collected during the product processing is obtained by acoustic sensors. The sound collected during the product processing is a sound sequence.

[0054] Step S120: Input the product image into the set first defect detection model to obtain the first processing defect detection result, and input the process parameter information into the set second defect detection model to obtain the second processing defect detection result.

[0055] The first defect detection model is a model trained on a large number of training samples, used to identify whether there are defects in the product image and, if there are defects, to identify the location and category of the defects.

[0056] The first defect detection model can be trained based on a convolutional neural network (CNN) model.

[0057] The second defect detection model is a model trained on a large number of training samples, which is used to identify whether there are defects in the product based on process parameter information.

[0058] The second defect detection model can be trained based on a recurrent neural network (RNN) model or a long short-term memory (LSTM) model.

[0059] Step S130: Determine the product defect identification result based on the first processing defect detection result and the second processing defect detection result.

[0060] In some embodiments, step S130 specifically includes: if the first processing defect detection result is that the product has a defect and / or the first processing defect detection result is that the processing process is abnormal, determining that the product defect identification result is that the product has a defect.

[0061] In this embodiment, if the first processing defect detection result is that the product has no defects and the second processing defect detection result is that no abnormality occurred in the processing, the product defect identification result is determined to be that the product has no defects.

[0062] If the product defect identification result confirms that the product has a defect, an early warning signal is issued to notify relevant personnel to adjust and optimize the production process to avoid similar defects in subsequent products.

[0063] Based on the first and second processing defect detection results, the product defect identification results are determined, realizing the detection of whether the product has defects from two detection angles. The first processing defect detection result is detected from the product itself, and the second processing defect detection result is detected from the product processing process. Combining the two detection angles can improve the product detection accuracy.

[0064] In some embodiments, prior to step S110, the method further includes training a predetermined first defect detection model. The training process of the predetermined first defect detection model includes the following steps S201 to S204.

[0065] Step S201: Obtain a second image training sample set, wherein each sample in the second image training sample set includes a product image corresponding to the processed product and first annotation information. The first annotation information includes whether the product has defects and, if the product has defects, the location information and type information of the product defects.

[0066] The finished products can be of the same type or different types. The product type can be determined based on the product's size or the processing method. Processing methods include welding, stamping, grinding, etc.

[0067] When the processing method is welding, the product defect type information is at least one of incomplete soldering or excessive solder. When the processing method is stamping, the product defect is at least one of surface cracks or deformation. When the processing method is grinding, the product defect is at least one of surface cracks or thermal deformation.

[0068] The second image training sample set includes both samples with product defects and samples without product defects.

[0069] Step S202: Input each sample from the second image training sample set into the first defect detection model to be trained for training, and obtain the product defect detection information corresponding to each sample.

[0070] Before step S202, for each sample in the second image training sample set, median filtering or Gaussian filtering is used to remove noise, and then the image pixel values ​​are normalized to the [0,1] interval.

[0071] Product defect detection information includes whether the product has a defect, and if so, the location and type of the defect.

[0072] Step S203: Determine the defect recognition accuracy value of the first defect detection model to be trained based on the first annotation information corresponding to each sample and the product defect detection information corresponding to each sample.

[0073] For each sample, the initial annotation information and the corresponding product defect detection information are compared to obtain the comparison results, which are either consistent or differ. Based on the comparison results for each sample, the defect identification accuracy value of the first defect detection model to be trained is determined. This is achieved by obtaining the number of samples with consistent comparison results and the total number of samples, and the ratio of the number of samples with consistent comparison results to the total number of samples is taken as the defect identification accuracy value of the first defect detection model to be trained. Alternatively, the cross-entropy loss function can be used to measure the difference between the product defect detection information corresponding to each sample and the initial annotation information corresponding to each sample, and this difference can be used as the defect identification accuracy value of the first defect detection model to be trained.

[0074] Step S204: If the defect recognition accuracy value of the first defect detection model to be trained is not greater than the first preset threshold, adjust the parameters of the first defect detection model to be trained until the defect recognition accuracy value of the first defect detection model to be trained is greater than the first preset threshold.

[0075] If the defect recognition accuracy of the first defect detection model to be trained is not greater than the first preset threshold, the parameters of the first defect detection model to be trained are continuously adjusted by optimization algorithms such as stochastic gradient descent until the defect recognition accuracy of the first defect detection model to be trained is greater than the first preset threshold.

[0076] In some embodiments, when the process parameter information is temperature, the method further includes training a pre-defined second defect detection model before step S110. The training process of the pre-defined second defect detection model includes the following steps S301 to S304.

[0077] Step S301: Obtain a second temperature training sample set, wherein each sample in the second temperature training sample set includes temperature and second annotation information collected based on the processing process. The second annotation information includes actual information on the first processing process status, which indicates whether the processing process is abnormal or not.

[0078] Temperature data collected during the processing can be collected for the same type of product or different types of products. The product type can be determined based on the product's size or processing method. Processing methods include welding, grinding, etc.

[0079] When the processing method is welding, the collected temperature is the molten pool temperature. When the processing method is grinding, the collected temperature is the temperature of the processed product.

[0080] The second temperature training sample set includes both samples where abnormalities occurred during processing and samples where no abnormalities occurred during processing.

[0081] Step S302: Input each sample from the second temperature training sample set into the second defect detection model to be trained for training, and obtain the first processing state detection information corresponding to each sample.

[0082] Before step S302, each sample in the second temperature training sample set is smoothed using a moving average filtering algorithm.

[0083] The first processing status detection information indicates whether the processing process is abnormal or not.

[0084] Step S303: Based on the actual information of the first processing state corresponding to each sample and the detection information of the first processing state corresponding to each sample, determine the detection accuracy value of the second defect detection model to be trained.

[0085] For each sample, the actual information of the first processing state and the corresponding detection information of the first processing state are compared to obtain the comparison result, which is either consistent or differs. Based on the comparison result for each sample, the detection accuracy value of the second defect detection model to be trained is determined. This involves obtaining the number of samples with consistent comparison results and the total number of samples, and using the ratio of the number of samples with consistent comparison results to the total number of samples as the detection accuracy value of the second defect detection model to be trained. Alternatively, the cross-entropy loss function can be used to measure the difference between the actual information of the first processing state corresponding to each sample and the detection information of the first processing state corresponding to each sample, and this difference can be used as the detection accuracy value of the second defect detection model to be trained.

[0086] Step S304: If the detection accuracy value of the second defect detection model to be trained is not greater than the second preset threshold, adjust the parameters of the second defect detection model to be trained until the detection accuracy value of the second defect detection model to be trained is greater than the second preset threshold.

[0087] If the detection accuracy of the second defect detection model to be trained is not greater than the second preset threshold, the parameters of the second defect detection model to be trained are continuously adjusted by optimization algorithms such as stochastic gradient descent until the detection accuracy of the first defect detection model to be trained is greater than the second preset threshold.

[0088] In some embodiments, when the process parameter information is sound, the method further includes training a pre-defined second defect detection model before step S110. The training process of the pre-defined second defect detection model includes the following steps S401 to S404.

[0089] Step S401: Obtain the second sound training sample set, wherein each sample in the second sound training sample set includes sound collected based on the processing process and third annotation information. The third annotation information includes the actual information of the second processing process status, which indicates whether the processing process is abnormal or not.

[0090] The sound collected during the processing can be generated during the processing of the same type of product or different types of products. The product type can be determined based on the product's size or processing method. Processing methods include stamping, grinding, etc.

[0091] The second audio training sample set includes both samples where abnormalities occurred during processing and samples where no abnormalities occurred during processing.

[0092] Step S402: Input each sample from the second sound training sample set into the second defect detection model to be trained for training, and obtain the second processing state detection information corresponding to each sample.

[0093] Before step S402, the time-domain signal of each sample in the second sound training sample set is converted into a frequency-domain signal by Fourier transform, and then noise reduction processing is performed.

[0094] The second processing status detection information indicates whether the processing process is abnormal or not.

[0095] Step S403: Determine the detection accuracy value of the second defect detection model to be trained based on the actual information of the second processing state corresponding to each sample and the detection information of the second processing state corresponding to each sample.

[0096] For each sample, the actual information of the second processing state is compared with the corresponding detection information of the second processing state to obtain the comparison result, which is either consistent or differs. Based on the comparison result for each sample, the detection accuracy value of the second defect detection model to be trained is determined. This involves obtaining the number of samples with consistent comparison results and the total number of samples, and the ratio of the number of samples with consistent comparison results to the total number of samples is determined as the detection accuracy value of the second defect detection model to be trained. Alternatively, the cross-entropy loss function can be used to measure the difference between the actual information of the second processing state corresponding to each sample and the detection information of the second processing state corresponding to each sample, and this difference can be used as the detection accuracy value of the second defect detection model to be trained.

[0097] Step S404: If the detection accuracy value of the second defect detection model to be trained is not greater than the second preset threshold, adjust the parameters of the second defect detection model to be trained until the detection accuracy value of the second defect detection model to be trained is greater than the second preset threshold.

[0098] If the detection accuracy of the second defect detection model to be trained is not greater than the second preset threshold, the parameters of the second defect detection model to be trained are continuously adjusted by optimization algorithms such as stochastic gradient descent until the detection accuracy of the first defect detection model to be trained is greater than the second preset threshold.

[0099] In some embodiments, the method further includes: when the product type corresponding to the product image is a newly introduced product type, retraining a set of first defect detection models using a first image training sample set to obtain a first defect detection model for identifying defects in products corresponding to the newly introduced product type; when the process parameter information is temperature and the process parameter information corresponds to a newly introduced product type, retraining a set of second defect detection models using a first temperature training sample set to obtain a second defect detection model for identifying whether an abnormality has occurred in the processing process based on the newly introduced product type; and when the process parameter information is sound and the process parameter information corresponds to a newly introduced product type, retraining a set of second defect detection models using a first sound training sample set to obtain a second defect detection model for identifying whether an abnormality has occurred in the processing process based on the newly introduced product type.

[0100] The product types corresponding to samples in the second image training sample set do not include the product types corresponding to samples in the first image training sample set. Similarly, the product types corresponding to samples in the second temperature training sample set do not include the product types corresponding to samples in the first temperature training sample set. Finally, the product types corresponding to samples in the second sound training sample set do not include the product types corresponding to samples in the first sound training sample set.

[0101] The retraining process of the first defect detection model using the first image training sample set can refer to the training process of the first defect detection model to be trained using the second image training sample set described above. Similarly, the retraining process of the second defect detection model using the first temperature training sample set can refer to the training process of the second defect detection model to be trained using the second temperature training sample set described above. Likewise, the retraining process of the second defect detection model using the first sound training sample set can refer to the training process of the second defect detection model to be trained using the second sound training sample set described above.

[0102] In this embodiment, the established first defect detection model and the established second defect detection model have good generalization ability and can adapt to changes in different types of products. When the factory introduces a new product type, it only needs to retrain the established first defect detection model and the established second defect detection model with a small amount of data to quickly adapt to the new detection requirements, without the need for large-scale modification of the entire detection system.

[0103] One embodiment of the present invention provides a product defect identification device. According to... Figure 2 As shown, the product defect identification device includes an acquisition module 210, a defect detection model detection result determination module 220, and a product defect identification result determination module 230.

[0104] The acquisition module 210 is used to acquire product images collected after processing and process parameter information collected based on the product processing process, wherein the process parameter information is temperature or sound.

[0105] The defect detection model detection result determination module 220 is used to input the product image into the set first defect detection model to obtain the first processing defect detection result, and to input the process parameter information into the set second defect detection model to obtain the second processing defect detection result.

[0106] The product defect identification result determination module 230 is used to determine the product defect identification result based on the first processing defect detection result and the second processing defect detection result.

[0107] In some embodiments, the process parameter information is time-series parameters collected during the product processing.

[0108] In some embodiments, the apparatus further includes a model training module.

[0109] The model training module is used to retrain a first defect detection model using a first image training sample set when the product type corresponding to the product image is a newly introduced product type, thereby obtaining a first defect detection model for identifying defects in products corresponding to the newly introduced product type; when the process parameter information is temperature and the process parameter information corresponds to a newly introduced product type, it is used to retrain a second defect detection model using a first temperature training sample set, thereby obtaining a second defect detection model for identifying whether abnormalities have occurred in the processing process based on the newly introduced product type; when the process parameter information is sound and the process parameter information corresponds to a newly introduced product type, it is used to retrain a second defect detection model using a first sound training sample set, thereby obtaining a second defect detection model for identifying whether abnormalities have occurred in the processing process based on the newly introduced product type.

[0110] In some embodiments, the model training module is used to acquire a second image training sample set, wherein each sample in the second image training sample set includes a product image corresponding to the processed product and first annotation information. The first annotation information includes whether the product has a defect and, if so, the location and type of the product defect. Each sample in the second image training sample set is input into a first defect detection model to be trained for training to obtain product defect detection information corresponding to each sample. Based on the first annotation information and the product defect detection information corresponding to each sample, the defect recognition accuracy value of the first defect detection model to be trained is determined. If the defect recognition accuracy value of the first defect detection model to be trained is not greater than a first preset threshold, the parameters of the first defect detection model to be trained are adjusted until the defect recognition accuracy value of the first defect detection model to be trained is greater than the first preset threshold.

[0111] In some embodiments, the model training module is used to acquire a second temperature training sample set, wherein each sample in the second temperature training sample set includes temperature collected based on the processing process and second annotation information, the second annotation information including first actual processing process status information, the first actual processing process status information being either an abnormality in the processing process or no abnormality in the processing process; each sample in the second temperature training sample set is input into the second defect detection model to be trained for training, to obtain the first processing process status detection information corresponding to each sample; based on the first actual processing process status information and the first processing process status detection information corresponding to each sample, the detection accuracy value of the second defect detection model to be trained is determined; if the detection accuracy value of the second defect detection model to be trained is not greater than a second preset threshold, the parameters of the second defect detection model to be trained are adjusted until the detection accuracy value of the second defect detection model to be trained is greater than the second preset threshold.

[0112] In some embodiments, the model training module is used to acquire a second sound training sample set, wherein each sample in the second sound training sample set includes sound collected based on the processing process and third annotation information, the third annotation information including actual information of the second processing process state, which indicates that the processing process is abnormal or not abnormal; input each sample in the second sound training sample set into the second defect detection model to be trained for training, to obtain the second processing process state detection information corresponding to each sample; determine the detection accuracy value of the second defect detection model to be trained based on the actual information of the second processing process state corresponding to each sample and the second processing process state detection information corresponding to each sample; if the detection accuracy value of the second defect detection model to be trained is not greater than a second preset threshold, adjust the parameters of the second defect detection model to be trained until the detection accuracy value of the second defect detection model to be trained is greater than the second preset threshold.

[0113] In some embodiments, the product defect identification result determination module 230 is used to determine that the product defect identification result is that the product has a defect when the first processing defect detection result is that the product has a defect and / or the first processing defect detection result is that the processing process is abnormal.

[0114] One embodiment of the present invention provides a product defect identification device. According to... Figure 3 As shown, the product defect identification device includes a memory 320 and a processor 310. The memory 320 stores a computer program that controls the processor 310 to operate and execute the product defect identification method provided according to any of the above embodiments.

[0115] One embodiment of the present invention provides an electronic device. The electronic device includes a product defect identification device as provided in any of the above embodiments, a camera for acquiring product images, and a temperature sensor for acquiring temperature, wherein the camera for acquiring product images and the temperature sensor for acquiring temperature are both connected to the product defect identification device. Alternatively, the electronic device includes a product defect identification device as provided in any of the above embodiments, a camera for acquiring product images, and a sound sensor for acquiring sound, wherein the camera for acquiring product images and the sound sensor for acquiring sound are both connected to the product defect identification device.

[0116] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0117] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0118] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0119] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages. Programming languages ​​include object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from computer-readable program instructions. These electronic circuits can execute computer-readable program instructions to implement various aspects of the present invention.

[0120] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0121] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0122] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0124] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A product defect identification method characterized by, The method comprises: obtaining product images collected after processing and process parameter information collected based on a product processing process, wherein the process parameter information is temperature or sound; inputting the product images into a set first defect detection model to obtain a first processing defect detection result, and inputting the process parameter information into a set second defect detection model to obtain a second processing defect detection result; determining a product defect identification result according to the first processing defect detection result and the second processing defect detection result.

2. The method of claim 1, wherein, The process parameter information is time sequence parameter information collected based on a product processing process.

3. The method of claim 1, wherein, The method further comprises: in a case where a product type corresponding to the product images is a newly introduced product type, retraining the set first defect detection model using a first image training sample set to obtain a first defect detection model for identifying defects of products of the newly introduced product type; in a case where the process parameter information is temperature and a product type corresponding to the process parameter information is a newly introduced product type, retraining the set second defect detection model using a first temperature training sample set to obtain a second defect detection model for identifying whether an abnormality occurs in a processing process based on the newly introduced product type; in a case where the process parameter information is sound and a product type corresponding to the process parameter information is a newly introduced product type, retraining the set second defect detection model using a first sound training sample set to obtain a second defect detection model for identifying whether an abnormality occurs in a processing process based on the newly introduced product type.

4. The method of claim 1, wherein, Before the obtaining of the product images collected after processing and the process parameter information collected based on the product processing process, the method further comprises: obtaining a second image training sample set, wherein each sample in the second image training sample set comprises a product image of a product after processing and first annotation information, and the first annotation information comprises whether the product has a defect and, in a case where the product has a defect, position information and type information of the defect of the product; inputting each sample in the second image training sample set into a first defect detection model to be trained to obtain product defect detection information corresponding to each sample; determining a defect identification accuracy value of the first defect detection model to be trained according to the first annotation information corresponding to each sample and the product defect detection information corresponding to each sample; in a case where the defect identification accuracy value of the first defect detection model to be trained is not greater than a first preset threshold, adjusting parameters of the first defect detection model to be trained until the defect identification accuracy value of the first defect detection model to be trained is greater than the first preset threshold.

5. The method of claim 1, wherein, Before the obtaining of the product images collected after processing and the process parameter information collected based on the product processing process, in a case where the process parameter information is temperature, the method further comprises: obtaining a second temperature training sample set, wherein each sample in the second temperature training sample set comprises temperature collected based on a processing process and second labeling information, the second labeling information comprises first processing process state actual information, and the first processing process state actual information indicates whether an abnormality occurs in the processing process; inputting each sample in the second temperature training sample set into a second defect detection model to be trained to perform training, to obtain first processing process state detection information corresponding to each sample; determining a detection accuracy value of the second defect detection model to be trained according to the first processing process state actual information corresponding to each sample and the first processing process state detection information corresponding to each sample; in a case where the detection accuracy value of the second defect detection model to be trained is not greater than a second preset threshold, adjusting parameters of the second defect detection model to be trained until the detection accuracy value of the second defect detection model to be trained is greater than the second preset threshold.

6. The method of claim 1, wherein, Before the obtaining of the product image collected after the completion of the processing and the process parameter information collected based on the product processing process, in a case where the process parameter information is sound, the method further comprises: obtaining a second sound training sample set, wherein each sample in the second sound training sample set comprises sound collected based on a processing process and third labeling information, the third labeling information comprises second processing process state actual information, and the second processing process state actual information indicates whether an abnormality occurs in the processing process; inputting each sample in the second sound training sample set into a second defect detection model to be trained to perform training, to obtain second processing process state detection information corresponding to each sample; determining a detection accuracy value of the second defect detection model to be trained according to the second processing process state actual information corresponding to each sample and the second processing process state detection information corresponding to each sample; in a case where the detection accuracy value of the second defect detection model to be trained is not greater than a second preset threshold, adjusting parameters of the second defect detection model to be trained until the detection accuracy value of the second defect detection model to be trained is greater than the second preset threshold.

7. The method of claim 1, wherein, the determining of a product defect identification result according to the first processing defect detection result and the second processing defect detection result comprises: in a case where the first processing defect detection result indicates that the product has a defect and / or the first processing defect detection result indicates that an abnormality occurs in the processing process, determining that the product defect identification result is that the product has a defect.

8. A product defect recognition apparatus characterized by comprising: comprises: an obtaining module, configured to obtain a product image collected after the completion of a processing and process parameter information collected based on a product processing process, wherein the process parameter information is temperature or sound; a defect detection model detection result determining module, configured to input the product image into a first defect detection model to obtain a first processing defect detection result, and input the process parameter information into a second defect detection model to obtain a second processing defect detection result; A product defect identification result determination module is configured to determine a product defect identification result based on the first processing defect detection result and the second processing defect detection result.

9. A product defect recognition apparatus characterized by comprising: The product defect identification device comprises a memory and a processor, the memory stores a computer program, and the computer program is used to control the processor to operate to execute the method according to any one of claims 1-7.

10. An electronic device, comprising: The product defect identification device comprises: The product defect identification device, the camera for collecting product images, and the temperature sensor for collecting temperature are connected with the product defect identification device. The product defect identification device, the camera for collecting product images, and the temperature sensor for collecting temperature are connected with the product defect identification device. The product defect identification device, the camera for collecting product images, and the sound sensor for collecting sound are connected with the product defect identification device. The product defect identification device, the camera for collecting product images, and the sound sensor for collecting sound are connected with the product defect identification device.

Images