Structured knowledge-driven industrial defect detection method and system

Abstract

The invention belongs to the technical field of industrial detection, and discloses a structured knowledge-driven industrial defect detection method and system, and the method comprises the steps: obtaining manufacturing data related to a product, and constructing a knowledge graph which reflects the relation between the defect characteristics of the industrial product and the machining process and environment through the manufacturing data; regarding the established industrial defect knowledge graph as a heterogeneous information network, and performing deep feature aggregation on various entities and relationships in the industrial defect knowledge graph by using a heterogeneous graph neural network technology to form a high-dimensional feature vector for guiding coupling of multiple environmental factors; establishing an industrial image illumination model by using a deep transfer learning technology, and simulating image features of industrial products under different acquisition conditions; an industrial product defect data enhancement framework based on a conditional diffusion model is constructed by training a conditional noise prediction network, a real and diversified industrial product image is generated according to the industrial product defect data enhancement framework, and industrial product defect detection is completed.

Description

Technical Field

[0001] The present invention relates to the field of industrial detection technology, and in particular to a structured knowledge-driven industrial defect detection method and system. Background Art

[0002] In recent years, the rapid implementation of knowledge graphs and generative artificial intelligence has led to technological changes in the field of industrial defect detection. Knowledge graphs, which originated from the early concepts of semantic networks and web ontologies, have now developed into advanced structures that can depict complex relationships between different entities in detail. These graphs play a vital role in encoding and parsing complex data networks in industrial environments, covering everything from product specifications to operating parameters. At the same time, generative artificial intelligence, especially generative adversarial networks (GANs) and diffusion models (DMs), have shown excellent performance in generating high-quality simulation data. Generative techniques were originally used for image and speech synthesis, and have now been successfully extended to data-intensive industries, becoming an indispensable tool in scenarios where limited data volumes or high data sensitivity are required. Knowledge graphs have evolved from early semantic networks to an indispensable element in industrial applications, which has had a transformative impact on the development of industrial intelligence and has become an important component of industrial information systems. In the industrial field, knowledge graph technology is a valuable tool for encoding detailed relationships between products, processes, and environmental factors. The rapid development of industrial knowledge graphs has a particularly profound impact on defect detection. By structured representation of the multifaceted nature of the manufacturing process, potential defect triggers are modeled, providing a visual prior knowledge framework for identifying and segmenting industrial defects. This feature is particularly important for industries where even minor defects can lead to significant economic or safety consequences. Knowledge graphs can put data in the right context in more complex manufacturing environments, making them an indispensable part of modern quality control systems. Summary of the invention

[0003] In order to solve the above technical problems, the present invention provides a structured knowledge-driven industrial defect detection method and system, which can more accurately identify and locate various defect types in industrial products and effectively improve the automation level of production quality control. By combining the advantages of different technical fields, the present invention provides strong technical support for improving industrial production efficiency and safety, and also lays a solid foundation for the further development of intelligent manufacturing.

[0004] The present invention provides a structured knowledge-driven industrial defect detection method, the method comprising:

[0005] S1: Obtain manufacturing data related to the product, and use the manufacturing data to construct a knowledge graph reflecting the relationship between the defect characteristics of industrial products and the processing process and environment, wherein the manufacturing data includes but is not limited to machine parameters, product design, production process and historical defect records;

[0006] S2: The established industrial defect knowledge graph is regarded as a heterogeneous information network, and the heterogeneous graph neural network technology is used to perform deep feature aggregation on various entities and relationships in the industrial defect knowledge graph to form a high-dimensional feature vector for guiding the coupling of multiple environmental factors;

[0007] S3: Based on the high-dimensional feature vector, an industrial image illumination model is established using deep transfer learning technology to simulate image features of industrial products under different acquisition conditions, where the acquisition conditions include factors such as light intensity, uniformity, and exposure time;

[0008] S4: According to the image features of industrial products under different simulated acquisition conditions, a conditional noise prediction network is trained to construct an industrial product defect data enhancement framework based on the conditional diffusion model. According to the industrial product defect data enhancement framework, realistic and diverse industrial product images are generated to complete industrial product defect detection.

[0009] Preferably, the method for constructing the industrial defect knowledge graph includes:

[0010] Acquire industrial product related data, wherein the industrial product related data includes the design, manufacturing process, quality inspection records, manufacturing environment parameter manufacturing process data of the industrial product, and pre-process the industrial product related data, specifically including: data cleaning and data standardization;

[0011] Identify knowledge entities from pre-processed industrial product related data, wherein the knowledge entities include but are not limited to: product type, defect type, detection equipment, and environmental parameters;

[0012] Define the attributes of knowledge entities, including but not limited to defect size, shape, illumination uniformity, and acquisition equipment noise transmission level;

[0013] Determine the relationship types between knowledge entities, including: <product-use-raw materials>, <equipment-production-product>, <operator-operation-equipment>, <defect-affect-product performance>, <process flow-includes-operation steps>, <detection method-applicable to-defect type>, <environmental conditions-influence-defect occurrence>, <repair method-solve-defect>, <quality standard-regulation-defect level>, <supplier-provides-raw materials>, <product-complies-with-safety standards>, <production batch-includes-product>;

[0014] The collected data and extracted triples are integrated into a unified Neo4j-based database to complete the construction of the industrial defect knowledge graph.

[0015] Preferably, the method of using heterogeneous graph neural network technology to perform deep feature aggregation on various entities and relationships in the industrial defect knowledge graph to form a high-dimensional feature vector for guiding the coupling of multiple environmental factors includes:

[0016] According to formula (1), the serialized entity features are initialized and embedded:

[0017]

[0018] Among them, E, O, and P represent the serialized representations of environmental characteristics, equipment characteristics, and product characteristics, respectively. Respectively represent the initial embedding representations corresponding to environmental features, device features, and product features, Λ α , Λ β , Λ γ They represent the initialization embedding parameter matrix, φ α ,φ β ,φ γ They represent the initialization embedding bias parameter vector, ReLU(·) represents the nonlinear activation function;

[0019] According to the feature extraction module defined in formula (2), the initialized embedded features are further aggregated:

[0020]

[0021] in, They represent the l-layer hidden variables corresponding to the environment features, equipment features, and product features, respectively. GCN represents the graph convolution mechanism, and GAT represents the graph attention mechanism. Represents the embedding parameter matrix of the relational attribute matrix at layer l, A attribute The parameter matrix representing the adjacency matrix;

[0022] After multiple rounds of knowledge graph feature aggregation, the defect feature guidance vector is obtained through the linear layer.

[0023] Preferably, the method of establishing an industrial image illumination model using deep transfer learning technology includes:

[0024] By pre-training the deep learning model from the existing large-scale natural scene image dataset, and then migrating the pre-trained deep learning model to the existing typical industrial defect dataset; specifically:

[0025] A deep U-Net network based on the ResNet architecture is trained on the ImageNet large-scale natural scene dataset, so that the U-Net network learns the key features that fully characterize natural scenes, including: light-shadow interaction, light field gradient, and over-exposed and under-exposed areas;

[0026] According to the characteristics of industrial product images, a multi-scale image feature enhancement network based on the pyramid mechanism is established to enhance the multi-scale industrial product feature images;

[0027] The U-Net network is initially fine-tuned using the enhanced multi-scale industrial product feature images, and the U-Net network is deepened or widened as required according to the resolution requirements of the industrial products;

[0028] Based on the freezing of some feature extraction layers in the network, the noise injection matrix and noise sampling step size are introduced to form a set of illumination models related to the degree of injected noise [U_Net1, U_Net2,..., U_Net n ].

[0029] Preferably, the method for constructing an industrial product defect data enhancement framework based on a conditional diffusion model includes:

[0030] According to formula (3), the industrial sample image sampled under the standard environment is processed by noise:

[0031]

[0032] in, represents the standard Wiener process, represents the Gaussian coefficient of the tth step, ρ t represents the variance of the added noise at step t, x0 represents the feature distribution of the original image, that is, the initial feature representation under noise-free conditions, and x t is the feature representation of the sampled image feature at the tth step after multiple steps of noise, and q0(x0) is the standard normal distribution representing the original image feature x0;

[0033] Using the illumination model [U_Net1, U_Net2, .., U_Net n ], as the initial network of the conditional noise prediction network, using the noise introduced in the tth step as the prediction label, and the aggregated As the conditional guidance vector, train the conditional noise prediction network ∈ θ (x t ,t,s t ), where s t is the conditional guidance vector representing the t-th step;

[0034] Using the conditional noise prediction network ∈ θ (x t ,t,s t ) decodes the randomly sampled Gaussian noise matrix to obtain a complete set of industrial defect data sets reflecting uncertain environmental characteristics, and completes the construction of an industrial product defect data enhancement framework based on the conditional diffusion model.

[0035] The present invention also provides a structured knowledge-driven industrial defect detection system, the system is used to implement any one of the methods described, the system comprising: a construction module, a fusion module, a simulation module and a detection module;

[0036] The construction module is used to obtain manufacturing data related to the product, and use the manufacturing data to construct a knowledge graph reflecting the relationship between the defect characteristics of industrial products and the processing process and environment, wherein the manufacturing data includes but is not limited to machine parameters, product design, production process and historical defect records;

[0037] The fusion module is used to regard the established industrial defect knowledge graph as a heterogeneous information network, and use the heterogeneous graph neural network technology to perform deep feature aggregation on various entities and relationships in the industrial defect knowledge graph to form a high-dimensional feature vector for guiding the coupling of multiple environmental factors;

[0038] The simulation module is used to establish an industrial image illumination model based on the high-dimensional feature vector using deep transfer learning technology to simulate image features of industrial products under different acquisition conditions, wherein the acquisition conditions include factors such as light intensity, uniformity, and exposure time;

[0039] The detection module is used to construct an industrial product defect data enhancement framework based on a conditional diffusion model by training a conditional noise prediction network according to the image features of industrial products under different simulated acquisition conditions. According to the industrial product defect data enhancement framework, real and diverse industrial product images are generated to complete industrial product defect detection.

[0040] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the methods described above when executing the program.

[0041] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, any one of the methods described above is implemented.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The core goal of the present invention is to improve the quality inspection accuracy of industrial products by accurately simulating the characteristic images of defective products in a multi-interference detection environment in the industrial production process, so as to achieve efficient, stable and safe operation of the quality inspection process. The present invention combines deep learning, knowledge graphs and image processing technology, and adopts heterogeneous graph neural networks and deep transfer learning to analyze and extract the key features of industrial defects. In addition, the conditional diffusion network technology is introduced to further improve the effect and adaptability of data enhancement. Through this method, various defect types in industrial products can be more accurately identified and located, and the automation level of production quality control can be effectively improved. By combining the advantages of different technical fields, the present invention provides strong technical support for improving industrial production efficiency and safety, and also lays a solid foundation for the further development of intelligent manufacturing. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0045] Figure 1 A schematic diagram of a flow chart of an industrial defect detection method driven by structured knowledge according to an embodiment of the present invention;

[0046] Figure 2 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention.

[0048] 1010, processor; 1020, memory; 1030, input / output interface; 1040, communication interface; 1050, bus. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0051] First, some technical terms used in the present invention are explained:

[0052] The structuring of knowledge refers to organizing the learned knowledge into a logical and hierarchical structure through induction and organization, so that it can be organized and programmatic, forming an organized knowledge system. Doing so can help people better understand the relationship between knowledge points, clarify the knowledge context, and thus master and apply knowledge more efficiently. The following are several key points of knowledge structuring: 1 Component division: Divide knowledge into several components or modules. 2 Relationship establishment: Establish connections between components to form an organic and orderly structure. 3 Intuitive operation: The structured knowledge framework should be easy to operate and understand intuitively. 4 Knowledge hierarchy: The knowledge structure is usually pyramid-shaped, from basic theory to professional knowledge, and then to the frontier knowledge of the discipline, layer by layer. 5 Application efficiency: Structured knowledge helps to quickly retrieve and apply, and improve learning efficiency. Knowledge structuring is of great significance to learning and personal development, because it not only helps memory and understanding, but also promotes the transfer and application of knowledge.

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] Embodiment 1

[0055] like Figure 1 As shown, an embodiment of the present invention provides a structured knowledge-driven industrial defect detection method, the method comprising:

[0056] S1: Acquire manufacturing data related to the product, and use the manufacturing data to construct a knowledge graph reflecting the relationship between industrial product defect characteristics and the processing process and environment for subsequent prompt condition feature extraction, wherein the manufacturing data includes but is not limited to machine parameters, product design, production process and historical defect records;

[0057] S2: The established industrial defect knowledge graph is regarded as a heterogeneous information network, and the heterogeneous graph neural network technology is used to perform deep feature aggregation on various entities and relationships in the industrial defect knowledge graph to form a high-dimensional feature vector for guiding the coupling of multiple environmental factors;

[0058] S3: In order to adapt to various complex image acquisition conditions in industrial defect detection, an industrial image illumination model is established based on the high-dimensional feature vector using deep transfer learning technology to simulate the image features of industrial products under different acquisition conditions, where the acquisition conditions include factors such as light intensity, uniformity, and exposure time;

[0059] S4: According to the image features of industrial products under different simulated acquisition conditions, a conditional noise prediction network is trained to construct an industrial product defect data enhancement framework based on the conditional diffusion model. According to the industrial product defect data enhancement framework, realistic and diverse industrial product images are generated to complete industrial product defect detection.

[0060] In this embodiment, the method for constructing the industrial defect knowledge graph includes:

[0061] Acquire industrial product related data, wherein the industrial product related data includes manufacturing process data such as the design, manufacturing process, quality inspection records, and manufacturing environment parameters of the industrial product, and pre-process the industrial product related data, specifically including: data cleaning and data standardization;

[0062] Identify knowledge entities from pre-processed industrial product related data, wherein the knowledge entities include but are not limited to: product type, defect type, detection equipment, and environmental parameters;

[0063] Define the attributes of knowledge entities, including but not limited to defect size, shape, illumination uniformity, and acquisition equipment noise transmission level;

[0064] Determine the relationship types between knowledge entities, including: <product-use-raw materials>, <equipment-production-product>, <operator-operation-equipment>, <defect-affect-product performance>, <process flow-includes-operation steps>, <detection method-applicable to-defect type>, <environmental conditions-influence-defect occurrence>, <repair method-solve-defect>, <quality standard-regulation-defect level>, <supplier-provides-raw materials>, <product-complies-with-safety standards>, <production batch-includes-product>;

[0065] The collected data and extracted triples are integrated into a unified Neo4j-based database to complete the construction of the industrial defect knowledge graph.

[0066] In this embodiment, the method of using heterogeneous graph neural network technology to perform deep feature aggregation on various entities and relationships in the industrial defect knowledge graph to form a high-dimensional feature vector for guiding the coupling of multiple environmental factors includes:

[0067] The heterogeneous graph includes three features: 1. Environmental characteristics, including light intensity, uniformity and light source type; 2. Equipment characteristics, including equipment model, resolution, sensitivity, operating parameters, etc.; 3. Product characteristics, including product type, size, material, parameters in the manufacturing process, etc.

[0068] According to formula (1), the serialized entity features are initialized and embedded:

[0069]

[0070] Among them, E, O, and P represent the serialized representations of environmental characteristics, equipment characteristics, and product characteristics, respectively. They represent the initial embedding representations corresponding to the environment features, equipment features, and product features, respectively, and v α ,Λ β ,Λ γ They represent the initialization embedding parameter matrix, φ α ,φ β ,φ γ They represent the initialization embedding bias parameter vector, ReLU(·) represents the nonlinear activation function;

[0071] According to the feature extraction module defined in formula (2), the initialized embedded features are further aggregated:

[0072]

[0073] in, They represent the l-layer hidden variables corresponding to the environment features, equipment features, and product features, respectively. GCN represents the graph convolution mechanism, and GAT represents the graph attention mechanism. Represents the embedding parameter matrix of the relational attribute matrix at layer l, A attribute The parameter matrix representing the adjacency matrix;

[0074] After multiple rounds of knowledge graph feature aggregation, the defect feature guidance vector is obtained through the linear layer.

[0075] In this embodiment, the method of establishing an industrial image illumination model using deep transfer learning technology includes:

[0076] By pre-training the deep learning model from the existing large-scale natural scene image dataset, and then migrating the pre-trained deep learning model to the existing typical industrial defect dataset; specifically:

[0077] A deep U-Net network based on the ResNet architecture is trained on the ImageNet large-scale natural scene dataset, so that the U-Net network can learn key features that fully characterize natural scenes, including: light-shadow interaction, light field gradient, and over-exposed and under-exposed areas;

[0078] According to the characteristics of industrial product images, a multi-scale image feature enhancement network based on the pyramid mechanism is established to enhance the multi-scale industrial product feature images;

[0079] The U-Net network is initially fine-tuned using the enhanced multi-scale industrial product feature images, and the U-Net network is deepened or widened as required according to the resolution requirements of the industrial products;

[0080] Based on the freezing of some feature extraction layers in the network, the noise injection matrix and noise sampling step size are introduced to form a set of illumination models related to the degree of injected noise [U_Net1, U_Net2,..., U_Net n ].

[0081] In this embodiment, a method for constructing an industrial product defect data enhancement framework based on a conditional diffusion model includes:

[0082] According to formula (3), the industrial sample image sampled under the standard environment is processed by noise:

[0083]

[0084] in, represents the standard Wiener process, represents the Gaussian coefficient of the tth step, ρ t represents the variance of the added noise at step t, x0 represents the feature distribution of the original image, that is, the initial feature representation under noise-free conditions, and x tis the feature representation of the sampled image features at the tth step after multiple steps of noise, and q0(x0) is the standard normal distribution representing the original image features x0.

[0085] Using the illumination model [U_Net1, U_Net2, ..., U_Net n ], as the initial network of the conditional noise prediction network, using the noise introduced in the tth step as the prediction label, and the aggregated As the conditional guidance vector, train the conditional noise prediction network ∈ θ (x t ,t,s t ), where s t is the conditional guidance vector representing the t-th step.

[0086] Using the conditional noise prediction network ∈ θ (x t ,t,s t ) decodes the randomly sampled Gaussian noise matrix to obtain a complete set of industrial defect data sets reflecting uncertain environmental characteristics, and completes the construction of an industrial product defect data enhancement framework based on the conditional diffusion model.

[0087] The technical solution of the present invention,

[0088] The structured knowledge-driven industrial defect data enhancement method proposed in this invention has broad application prospects, especially in the fields of industrial defect detection, quality control, and intelligent manufacturing. Several specific application scenarios are listed below:

[0089] (1) Defect detection in industrial production lines. In high-precision and large-scale industrial production processes, especially complex automated production lines, the detection system needs to cope with changing environmental conditions (such as changes in lighting, equipment failures, differences in workpiece materials, etc.). By introducing the data enhancement method of the present invention, it is possible to generate a variety of industrial defect images and simulate various interference conditions in real production environments. These enhanced data can be used to train more robust defect detection models, improve the accuracy and reliability of the detection system under various complex conditions, and especially in the detection of low-probability defects or special defect types, effectively reduce missed detections and misjudgments.

[0090] (2) Real-time quality monitoring during the production process. The present invention provides an industrial defect data enhancement framework based on a knowledge graph, which has important applications in real-time quality monitoring systems. By combining the real-time data collected during the production process (such as equipment status, operating parameters, lighting conditions, etc.) with historical defect records, a dynamically updated knowledge graph is constructed. The graph can provide timely defect identification prompts and fault predictions for the real-time monitoring system, helping the production line to adjust process parameters in a timely manner to avoid potential defects, thereby effectively improving the product qualification rate.

[0091] (3) Improve safety and efficiency in the industrial field. In some high-risk industrial fields, the data enhancement method proposed in this invention can simulate defect images under extreme working conditions and provide more representative training data. This will greatly improve the accuracy and robustness of the defect detection system and ensure that potential safety hazards are discovered before the product leaves the factory. At the same time, through efficient quality inspection and automated control, production efficiency can be improved, the cost and time of manual inspection can be reduced, and production safety can be guaranteed.

[0092] Through these specific applications, the present invention can not only effectively solve the quality control problems existing in the industrial production process, but also promote the intelligent upgrading of the manufacturing industry, improve product quality and production efficiency, and promote the further development of intelligent manufacturing.

[0093] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.

[0094] It should be noted that some embodiments of the present disclosure are described above. Other embodiments are within the scope of the attached claims. In some cases, it should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The actions or steps recorded in the claims can be performed in an order different from that in the above embodiment and still achieve the desired results. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0095] Embodiment 2

[0096] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present invention further provides a structured knowledge-driven industrial defect detection system, the system is used to implement any of the methods described, the system comprising: a construction module, a fusion module, a simulation module and a detection module;

[0097] The construction module is used to obtain manufacturing data related to the product, and use the manufacturing data to construct a knowledge graph reflecting the relationship between the defect characteristics of industrial products and the processing process and environment, wherein the manufacturing data includes but is not limited to machine parameters, product design, production process and historical defect records;

[0098] The fusion module is used to regard the established industrial defect knowledge graph as a heterogeneous information network, and use the heterogeneous graph neural network technology to perform deep feature aggregation on various entities and relationships in the industrial defect knowledge graph to form a high-dimensional feature vector for guiding the coupling of multiple environmental factors;

[0099] The simulation module is used to establish an industrial image illumination model based on the high-dimensional feature vector using deep transfer learning technology to simulate the image features of industrial products under different acquisition conditions, wherein the acquisition conditions include factors such as light intensity, uniformity, and exposure time;

[0100] The detection module is used to construct an industrial product defect data enhancement framework based on the conditional diffusion model by training a conditional noise prediction network according to the image features of industrial products under different simulated acquisition conditions. According to the industrial product defect data enhancement framework, real and diverse industrial product images are generated to complete industrial product defect detection.

[0101] The system of the above embodiment is used to implement a corresponding structured knowledge-driven industrial defect detection method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0102] It should be noted that the above-mentioned structured knowledge-driven industrial defect detection system is embodied in the form of functional units. The term "module" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0103] For example, a "module" may be a software program, a hardware circuit, or a combination of the two that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a combined logic circuit, and / or other suitable components that support the described functions.

[0104] Embodiment 3

[0105] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a structured knowledge-driven industrial defect detection method described in any of the above embodiments is implemented.

[0106] Figure 2A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0107] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0108] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0109] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0110] The communication interface 1040 is used to connect a communication module (not shown in the figure) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB (Universal Serial Bus), network cable, etc.), or through a wireless mode (such as mobile network, WIFI (Wireless Fidelity), Bluetooth, etc.).

[0111] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0112] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0113] The system of the above embodiment is used to implement a corresponding structured knowledge-driven industrial defect detection method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0114] Embodiment 4

[0115] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute a structured knowledge-driven industrial defect detection method as described in any of the above embodiments.

[0116] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0117] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute a structured knowledge-driven industrial defect detection method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0118] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0119] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present disclosure difficult to understand, the known power / ground connections to the integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it is apparent to those skilled in the art that the embodiments of the present disclosure can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0120] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0121] Therefore, the units of each example described in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0122] The embodiments of the present invention are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the present invention.

Claims

1. A structured knowledge driven industrial defect detection method, characterized in that: The method comprises: S1: Obtain manufacturing data related to the product, and use the manufacturing data to construct a knowledge graph reflecting the relationship between the defect characteristics of industrial products and the processing process and environment, wherein the manufacturing data includes but is not limited to machine parameters, product design, production process and historical defect records; S2: The established industrial defect knowledge graph is regarded as a heterogeneous information network, and the heterogeneous graph neural network technology is used to perform deep feature aggregation on various entities and relationships in the industrial defect knowledge graph to form a high-dimensional feature vector for guiding the coupling of multiple environmental factors; S3: Based on the high-dimensional feature vector, an industrial image illumination model is established using deep transfer learning technology to simulate image features of industrial products under different acquisition conditions, where the acquisition conditions include factors such as light intensity, uniformity, and exposure time; S4: According to the image features of industrial products under different simulated acquisition conditions, a conditional noise prediction network is trained to construct an industrial product defect data enhancement framework based on the conditional diffusion model. According to the industrial product defect data enhancement framework, realistic and diverse industrial product images are generated to complete industrial product defect detection.

2. The method according to claim 1, characterized in that The construction method of industrial defect knowledge graph includes: Acquire industrial product related data, wherein the industrial product related data includes the design, manufacturing process, quality inspection records, manufacturing environment parameter manufacturing process data of the industrial product, and pre-process the industrial product related data, specifically including: data cleaning and data standardization; Identify knowledge entities from pre-processed industrial product related data, wherein the knowledge entities include but are not limited to: product type, defect type, detection equipment, and environmental parameters; Define the attributes of knowledge entities, including but not limited to defect size, shape, illumination uniformity, and acquisition equipment noise transmission level; Determine the relationship types between knowledge entities, including: <product-use-raw materials>, <equipment-production-product>, <operator-operation-equipment>, <defect-affect-product performance>, <process flow-includes-operation steps>, <detection method-applicable to-defect type>, <environmental conditions-influence-defect occurrence>, <repair method-solve-defect>, <quality standard-regulation-defect level>, <supplier-provides-raw materials>, <product-complies-with-safety standards>, <production batch-includes-product>; The collected data and extracted triples are integrated into a unified Neo4j-based database to complete the construction of the industrial defect knowledge graph.

3. The method according to claim 1, characterized in that The method of using heterogeneous graph neural network technology to perform deep feature aggregation on various entities and relationships in the industrial defect knowledge graph to form a high-dimensional feature vector for guiding the coupling of multiple environmental factors includes: According to formula (1), the serialized entity features are initialized and embedded: Among them, E, O, and P represent the serialized representations of environmental characteristics, equipment characteristics, and product characteristics, respectively. Respectively represent the initial embedding representations corresponding to environmental features, device features, and product features, Λ α , Λ β , Λ γ They represent the initialization embedding parameter matrix, φ α ,φ β ,φ γ They represent the initialization embedding bias parameter vector, ReLU(·) represents the nonlinear activation function; According to the feature extraction module defined in formula (2), the initialized embedded features are further aggregated: in, They represent the l-layer hidden variables corresponding to the environment features, equipment features, and product features, respectively. GCN represents the graph convolution mechanism, and GAT represents the graph attention mechanism. Represents the embedding parameter matrix of the relational attribute matrix at layer l, A attribute The parameter matrix representing the adjacency matrix; After multiple rounds of knowledge graph feature aggregation, the defect feature guidance vector is obtained through the linear layer.

4. The method according to claim 1, characterized in that: Methods for building industrial image illumination models using deep transfer learning technology include: By pre-training the deep learning model from the existing large-scale natural scene image dataset, and then migrating the pre-trained deep learning model to the existing typical industrial defect dataset; specifically: A deep U-Net network based on the ResNet architecture is trained on the large-scale natural scene dataset of ImageNet, so that the U-Net network can learn the key features that fully characterize natural scenes, including: light-shadow interaction, light field gradient, and over- and under-exposed areas; According to the characteristics of industrial product images, a multi-scale image feature enhancement network based on the pyramid mechanism is established to enhance the multi-scale industrial product feature images; The U-Net network is initially fine-tuned using the enhanced multi-scale industrial product feature images, and the U-Net network is deepened or widened as required according to the resolution requirements of the industrial products; Based on the freezing of some feature extraction layers in the network, the noise injection matrix and noise sampling step size are introduced to form a set of illumination models related to the degree of injected noise [U_Net1, U_Net2, ..., U_Net n ].

5. The method according to claim 4, characterized in that The construction method of the industrial product defect data enhancement framework based on the conditional diffusion model includes: According to formula (3), the industrial sample image sampled under the standard environment is processed by noise: in, represents the standard Wiener process, represents the Gaussian coefficient of the tth step, ρ t represents the variance of the added noise at step t, x0 represents the feature distribution of the original image, that is, the initial feature representation under noise-free conditions, and x t is the feature representation of the sampled image feature at the tth step after multiple steps of noise, and q0(x0) is the standard normal distribution representing the original image feature x0; Using the illumination model [U_Net1, U_Net2, ..., U_Net n ], as the initial network of the conditional noise prediction network, using the noise introduced in the tth step as the prediction label, and the aggregated As the conditional guidance vector, train the conditional noise prediction network ∈ θ (x t ,t,s t ), where s t is the conditional guidance vector representing the t-th step; Using the conditional noise prediction network ∈ θ (x t ,t,s t ) decodes the randomly sampled Gaussian noise matrix to obtain a complete set of industrial defect data sets reflecting uncertain environmental characteristics, and completes the construction of an industrial product defect data enhancement framework based on the conditional diffusion model.

6. A structured knowledge-driven industrial defect detection system, the system being used to implement the method of any one of claims 1 to 5, characterized in that: The system comprises: a construction module, a fusion module, a simulation module and a detection module; The construction module is used to obtain manufacturing data related to the product, and use the manufacturing data to construct a knowledge graph reflecting the relationship between the defect characteristics of industrial products and the processing process and environment, wherein the manufacturing data includes but is not limited to machine parameters, product design, production process and historical defect records; The fusion module is used to regard the established industrial defect knowledge graph as a heterogeneous information network, and use the heterogeneous graph neural network technology to perform deep feature aggregation on various entities and relationships in the industrial defect knowledge graph to form a high-dimensional feature vector for guiding the coupling of multiple environmental factors; The simulation module is used to establish an industrial image illumination model based on the high-dimensional feature vector using deep transfer learning technology to simulate image features of industrial products under different acquisition conditions, wherein the acquisition conditions include factors such as light intensity, uniformity, and exposure time; The detection module is used to construct an industrial product defect data enhancement framework based on a conditional diffusion model by training a conditional noise prediction network according to the image features of industrial products under different simulated acquisition conditions. According to the industrial product defect data enhancement framework, real and diverse industrial product images are generated to complete industrial product defect detection.

7. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 5 is implemented.

Images