A structured knowledge driven industrial defect detection method and system

Abstract

The application belongs to the technical field of industrial detection, and discloses a structured knowledge-driven industrial defect detection method and system, manufacturing data related to a product is acquired, a knowledge graph reflecting the relationship between the defect characteristics of an industrial product and the processing process and environment is constructed by using the manufacturing data; the established industrial defect knowledge graph is regarded as a heterogeneous information network, and heterogeneous graph neural network technology is used to perform deep feature aggregation on various entities and relationships in the industrial defect knowledge graph, thereby forming a high-dimensional feature vector used to guide under the coupling of multiple environmental factors; a deep migration learning technology is used to establish an industrial image illumination model, and the image features of an industrial product under different collection conditions are simulated; a conditional noise prediction network is trained, an industrial product defect data enhancement framework based on a conditional diffusion model is constructed, and according to the industrial product defect data enhancement framework, real and diversified industrial product images are generated, thereby completing industrial product defect detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial detection, in particular to a structured knowledge-driven industrial defect detection method and system. BACKGROUND

[0002] In recent years, the rapid landing of knowledge graphs and generative artificial intelligence has caused a technological revolution in the field of industrial defect detection. Knowledge graphs, originating from the early concepts of semantic networks and network ontologies, have evolved into advanced structures that can detail complex relationships between different entities. These graphs play a crucial role in encoding and analyzing complex data networks in industrial environments, covering everything from product specifications to operational parameters. At the same time, generative artificial intelligence, particularly generative adversarial networks (GANs) and diffusion models (DMs), has shown exceptional performance in generating high-quality simulated data. Initially used for image and speech synthesis, generative techniques have now successfully expanded into data-intensive industries, becoming an indispensable tool in scenarios where data volume is limited or data sensitivity is high. Knowledge graphs have evolved from early semantic networks to become an indispensable element in industrial applications, having a transformative impact on industrial intelligence and becoming an important component of industrial information systems. In the industrial field, knowledge graph technology is a valuable tool for encoding detailed relationships between products, processes, and environmental factors. The rapid development of industrial knowledge graphs has had a profound impact on defect detection, modeling potential defect triggers by structurally representing the multi-faceted nature of manufacturing processes, providing a visual prior knowledge framework for identifying and segmenting industrial defects. This feature is particularly important for industries where even minor defects can lead to significant economic or safety consequences. Knowledge graphs can place data in the appropriate context in more complex manufacturing environments, making them an indispensable part of modern quality control systems. SUMMARY

[0003] To solve the above technical problems, the present application provides a structured knowledge-driven industrial defect detection method and system, which can more accurately identify and locate various defect types in industrial products, effectively improving the automation level of production quality control. By combining the advantages of different technical fields, the present application provides strong technical support for improving industrial production efficiency and safety, and also lays a solid foundation for the further development of intelligent manufacturing.

[0004] The present application provides a structured knowledge-driven industrial defect detection method, which comprises:

[0005] S1: Obtain manufacturing data related to the product, and use the manufacturing data to construct a knowledge graph reflecting the relationship between the defect characteristics of the industrial product and the processing process and the environment, wherein the manufacturing data includes but is not limited to machine parameters, product design, production process and historical defect records;

[0006] S2: The established industrial defect knowledge graph is regarded as a heterogeneous information network, and a heterogeneous graph neural network technology is used to aggregate the deep features of various entities and relationships in the industrial defect knowledge graph, forming a high-dimensional feature vector for guiding the coupling of multiple environmental factors;

[0007] S3: According to the high-dimensional feature vector, an industrial image illumination model is established by using deep transfer learning technology to simulate the image features of industrial products under different collection conditions, wherein the collection conditions include light intensity, uniformity, and exposure time factors;

[0008] S4: According to the simulated image features of industrial products under different collection conditions, a conditional noise prediction network is trained to construct an industrial product defect data enhancement framework based on a conditional diffusion model, and according to the industrial product defect data enhancement framework, real and diversified industrial product images are generated to complete the industrial product defect detection.

[0009] Preferably, the construction method of the industrial defect knowledge graph comprises:

[0010] Obtain industrial product related data, wherein the industrial product related data includes the design, manufacturing process, quality detection record, manufacturing environment parameter manufacturing process data of the industrial product, and pre-process the industrial product related data, specifically including data cleaning and data standardization;

[0011] Identify knowledge entities from the pre-processed industrial product related data, wherein the knowledge entities include but are not limited to product type, defect type, detection equipment, and environmental parameters;

[0012] Define the attributes of the knowledge entities, including but not limited to defect size, shape, light uniformity, and noise transmission level of the collection equipment;

[0013] Determine the relationship types between the knowledge entities, specifically including: <product-use-raw material>, <equipment-production-product>, <operator-operation-equipment>, <defect-impact-product performance>, <process flow-contains-operation step>, <detection method-applicable-defect type>, <environmental conditions-impact-defect occurrence>, <repair method-solve-defect>, <quality standard-regulate-defect level>, <supplier-provide-raw material>, <product-comply-safety standards>, <production batch-contains-product>;

[0014] The collected data and extracted triples are integrated into a unified Neo4j-based database to complete construction of the industrial defect knowledge graph.

[0015] Preferably, the heterogeneous graph neural network technology is used to aggregate deep features of various entities and relationships in the industrial defect knowledge graph to form a method for guiding high-dimensional feature vectors under the coupling of multiple environmental factors.

[0016] According to formula (1), the serialized entity features are initialized and embedded as follows:

[0017]

[0018] wherein E, O and P represent serialized representations of environmental features, equipment features and product features respectively, represent initial embedding representations of environmental features, equipment features and product features respectively, and α β γ represent initialization embedding parameter matrices respectively, and α β γ represent initialization embedding bias parameter vectors respectively, and ReLU(·) represents a nonlinear activation function.

[0019] According to the feature extraction module defined in formula (2), the initialized embedding features are further aggregated as follows:

[0020]

[0021] wherein represent l-layer hidden variables of environmental features, equipment features and product features respectively, GCN represents a graph convolution mechanism, and GAT represents a graph attention mechanism, represent embedding parameter matrices of relationship attribute matrices of the lth layer, and attribute represent parameter matrices of adjacency matrices.

[0022] After multiple rounds of knowledge graph feature aggregation, a defect feature guide vector is obtained through a linear layer as follows:

[0023] Preferably, the method for establishing an industrial image illumination model using deep transfer learning technology comprises:

[0024] The method comprises the following steps:

[0025] ​​​​The deep U-Net network based on the ResNet architecture is trained on the ImageNet large natural scene dataset, so that the U-Net network learns to learn the key features of the natural scene, wherein the key features include: light shadow interaction, light field gradient and overexposure and underexposure area.

[0026] According to the characteristics of the industrial product image, a multi-scale image feature enhancement network based on a pyramid mechanism is established to enhance the multi-scale industrial product feature image.

[0027] The enhanced multi-scale industrial product feature image is used to preliminarily fine-tune the U-Net network, and the U-Net network is deepened or widened according to the resolution requirements of the industrial product.

[0028] On the basis of freezing part of the feature extraction layer of the network, a noise injection matrix and a noise sampling step are introduced, thereby forming a set of illumination models [U_Net1, U_Net2,..., U_Net n ].

[0029] Preferably, the method for constructing the industrial product defect data enhancement framework based on the conditional diffusion model comprises:

[0030] According to formula (3), the industrial sample image sampled under the standard environment is noise processed:

[0031]

[0032] wherein, represents a standard Wiener process, represents the Gaussian coefficient of the t-th step, and ρ t represents the variance of the added noise of the t-th step, x0 represents the feature distribution of the original image, that is, the initial feature representation under the condition of no noise, x t is the feature representation of the sampled image at the t-th step, and the feature representation after multi-step noise processing, q0(x0) is the standard normal distribution of the original image feature x0;

[0033] The illumination model [U_Net1, U_Net2,.., U_Net n ] is used as the initial network of the conditional noise prediction network, the noise introduced at the t-th step is used as the prediction label, and the aggregated as a conditional guidance vector, and the conditional noise prediction network ∈ θ (x t , t, s t ) is trained, wherein s t represents the conditional guidance vector at the t-th step.

[0034] A conditional noise prediction network is utilized to predict θ (x t , t, s t ) is used to decode a randomly sampled Gaussian noise matrix to obtain a complete set of industrial defect data reflecting uncertain environmental characteristics, and an industrial product defect data enhancement framework based on a conditional diffusion model is constructed.

[0035] The application also provides a structured knowledge-driven industrial defect detection system, which is used to implement any of the methods, and the system comprises a construction module, a fusion module, a simulation module and a detection module.

[0036] The construction module is used to obtain manufacturing data related to a product, and a knowledge graph reflecting the relationship between industrial product defect characteristics and processing, environment is constructed using the manufacturing data, wherein the manufacturing data includes but is not limited to machine parameters, product design, production process and historical defect records.

[0037] The fusion module is used to regard the established industrial defect knowledge graph as a heterogeneous information network, and deep feature aggregation is performed on various entities and relationships in the industrial defect knowledge graph using a heterogeneous graph neural network technology to form a high-dimensional feature vector used to guide the coupling of multiple environmental factors.

[0038] The simulation module is used to establish an industrial image illumination model using deep transfer learning technology according to the high-dimensional feature vector to simulate the image features of industrial products under different acquisition conditions, wherein the acquisition conditions include light intensity, uniformity and exposure time factors.

[0039] The detection module is used to generate real and diversified industrial product images by training a conditional noise prediction network according to the simulated image features of industrial products under different acquisition conditions, construct an industrial product defect data enhancement framework based on a conditional diffusion model, and complete industrial product defect detection according to the industrial product defect data enhancement framework.

[0040] The application also provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the methods when executing the program.

[0041] The application also provides a computer-readable storage medium storing a computer program, wherein the computer program is executable to implement any of the methods.

[0042] Compared with the prior art, the application has the following beneficial effects:

[0043] The core objective of the present application is to improve the quality detection precision of industrial products by accurately simulating the feature images of defective products in a multi-interference detection environment during industrial production, and to realize efficient, stable and safe operation of the quality detection process. The present application combines deep learning, knowledge graph and image processing technology, and uses heterogeneous graph neural networks and deep transfer learning to analyze and extract key features of industrial defects. In addition, the conditional diffusion network technology is introduced to further improve the effect and adaptability of data enhancement. Through this method, various types of defects in industrial products can be more accurately identified and located, effectively improving the automation level of production quality control. By combining the advantages of different technical fields, the present application provides strong technical support for improving industrial production efficiency and safety, and also lays a solid foundation for the further development of intelligent manufacturing. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the present application, the following briefly introduces the drawings needed to be used in the embodiments. Obviously, the drawings described in the following only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0045] Figure 1 A structured knowledge-driven industrial defect detection method flowchart for an embodiment of the present application;

[0046] Figure 2 A structural diagram of an electronic device for an embodiment of the present application.

[0047] 1010, processor; 1020, memory; 1030, input / output interface; 1040, communication interface; 1050, bus. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0049] It should be noted that the technical terms or scientific terms used in the embodiments of the present disclosure should be understood as the general meaning understood by those skilled in the art to which the embodiments of the present disclosure belong, unless otherwise defined. The terms "first", "second", and similar terms used in the embodiments of the present disclosure do not represent any order, number, or importance, but are only used to distinguish different components. The terms "include", "contain", and similar terms mean that the elements or objects before the terms cover the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connect" or "connected" and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "upper", "lower", "left", "right", and the like are only used to represent relative positional relationships, and when the absolute positions of the described objects change, the relative positional relationships may also change accordingly.

[0050] First, some technical terms used in the present application are described:

[0051] Knowledge structuring refers to the process of organizing learned knowledge into a logical and hierarchical structure through induction and arrangement, making it systematic and programmatic, and forming an organized knowledge system. This can help people better understand the relationships between knowledge points and clarify the knowledge context, so as to more efficiently master and apply knowledge. The following are several key points of knowledge structuring: 1 Component division: divide knowledge into several components or modules. 2 Relationship establishment: establish connections between components to form an organic and orderly structure. 3 Intuitive operation: the structured knowledge framework should be easy to understand and operate. 4 Knowledge hierarchy: the knowledge structure usually takes the form of a pyramid, from basic theory to professional knowledge, and then to cutting-edge knowledge in the field, with each layer building on the previous one. 5 Application efficiency: structured knowledge helps quick retrieval and application, improving learning efficiency. Knowledge structuring is of great significance to learning and personal development, as it not only helps memory and understanding, but also promotes knowledge transfer and application.

[0052] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0053] Embodiment one

[0054] As shown in Figure 1 The present application provides a structured knowledge-driven industrial defect detection method, which comprises:

[0055] S1: Obtain manufacturing data related to the product, and use the manufacturing data to construct a knowledge graph reflecting the relationship between the defect characteristics of the industrial product and the processing process and environment, for subsequent prompt condition feature extraction, wherein the manufacturing data includes but is not limited to machine parameters, product design, production process and historical defect records;

[0056] S2: The established industrial defect knowledge graph is regarded as a heterogeneous information network, and a heterogeneous graph neural network technology is used to aggregate the deep features of various entities and relationships in the industrial defect knowledge graph, to form a high-dimensional feature vector for guiding the coupling of multiple environmental factors;

[0057] S3: In order to adapt to various complex image acquisition conditions in industrial defect detection, according to the high-dimensional feature vector, a deep transfer learning technology is used to establish an industrial image lighting model to simulate the image features of industrial products under different acquisition conditions, wherein the acquisition conditions include light intensity, uniformity, exposure time and other factors;

[0058] S4: According to the simulated image features of industrial products under different acquisition conditions, a conditional noise prediction network is trained to construct an industrial product defect data enhancement framework based on a conditional diffusion model, and according to the industrial product defect data enhancement framework, real and diversified industrial product images are generated to complete the industrial product defect detection.

[0059] In this embodiment, the construction method of the industrial defect knowledge graph comprises:

[0060] Obtain industrial product related data, wherein the industrial product related data includes design, manufacturing process, quality detection record, manufacturing environment parameter and other manufacturing process data of the industrial product, and the industrial product related data is preprocessed, specifically including data cleaning and data standardization;

[0061] Identify knowledge entities from the preprocessed industrial product related data, wherein the knowledge entities include but are not limited to product type, defect type, detection equipment and environmental parameters;

[0062] Define the attributes of the knowledge entities, including but not limited to defect size, shape, light uniformity, and acquisition device noise transmission level;

[0063] determine the relationship type between knowledge entities, including: <product-use-raw material>, <equipment-produce-product>, <operator-operate-equipment>, <defect-affect-product performance>, <process flow-contains-operation step>, <detection method-suitable-for-defect type>, <environmental conditions-affect-defect occurrence>, <repair method-solve-defect>, <quality standards-regulate-defect level>, <supplier-provide-raw materials>, <product-meet-safety standards>, <production batch-contains-product>;

[0064] Integrate the collected data and extracted triples into a unified Neo4j-based database, and complete the construction of the industrial defect knowledge graph.

[0065] In this embodiment, the heterogeneous graph neural network technology is used to aggregate the deep features of various entities and relationships in the industrial defect knowledge graph, forming a method for guiding high-dimensional feature vectors under the coupling of multiple environmental factors, which includes:

[0066] The heterogeneous graph includes three features: 1. Environmental features, including light intensity, uniformity, and light source type; 2. Device features, including device model, resolution, sensitivity, and operation parameters; 3. Product features, including product type, size, material, and manufacturing process parameters.

[0067] According to formula (1), the initialized embedding of the serialized entity features is initialized:

[0068]

[0069] Wherein, E, O, P represent the serialized representation of environmental features, device features and product features respectively, represent the initial embedding representation of environmental features, device features and product features respectively, v α ,Λ β ,Λ γ represent the initialized embedding parameter matrix respectively, φ α ,φ β ,φ γ represent the initialized embedding bias parameter vector respectively, and ReLU(·) represents a nonlinear activation function.

[0070] According to the feature extraction module defined by formula (2), the initialized embedding features are further aggregated:

[0071]

[0072] Wherein, represent the l-layer hidden variables of environmental features, device features and product features respectively, GCN represents a graph convolution mechanism, and GAT represents a graph attention mechanism, An embedding parameter matrix A representing a relational attribute matrix of the lth layer attribute A parameter matrix A representing an adjacency matrix

[0073] After multiple rounds of knowledge graph feature aggregation, a defect feature guide vector is obtained through a linear layer

[0074] In this embodiment, the method for establishing an industrial image illumination model by using deep transfer learning technology comprises the following steps:

[0075] A pre-trained deep learning model is obtained from an existing large natural scene image dataset, and then the pre-trained deep learning model is migrated to an existing typical industrial defect dataset; specifically:

[0076] A deep U-Net network based on a ResNet architecture is trained on an ImageNet large natural scene dataset, so that the U-Net network learns to fully represent key features of natural scenes, wherein the key features include: light and shadow interaction, light field gradient, and overexposed and underexposed regions, etc.

[0077] A multi-scale image feature enhancement network based on a pyramid mechanism is established to enhance multi-scale industrial product feature images;

[0078] The enhanced multi-scale industrial product feature images are used to preliminarily fine-tune the U-Net network, and the U-Net network is deepened or widened according to the resolution requirements of the industrial products;

[0079] On the basis of freezing part of the feature extraction layers of the network, a noise injection matrix and a noise sampling step are introduced, thereby forming a set of illumination models [U_Net1, U_Net2,..., U_Net n ].

[0080] In this embodiment, the method for constructing an industrial product defect data enhancement framework based on a conditional diffusion model comprises the following steps:

[0081] According to formula (3), the industrial sample images sampled under the standard environment are subjected to noise processing:

[0082]

[0083] Wherein, represents a standard Wiener process, represents the Gaussian coefficient of the tth step, and t represents the variance of the added noise of the tth step, x0 represents the feature distribution of the original image, i.e. the initial feature representation under the noise-free condition, and x tTo represent the sampled image features at step t, the multi-step denoised feature representation, q0(x0) is a standard normal distribution representing the original image features x0.

[0084] Using the illumination model [U_Net1, U_Net2,..., U_Net n ] as the initial network of the conditional noise prediction network, the noise introduced at step t as the prediction label, and the aggregated As a conditional guidance vector, the conditional noise prediction network ∈ θ (x t , t, s t ) is trained, where s t represents the conditional guidance vector at step t.

[0085] Using the conditional noise prediction network ∈ θ (x t , t, s t ) to decode the randomly sampled Gaussian noise matrix, a complete set of industrial defect data reflecting uncertain environmental features is obtained, and the construction of the industrial product defect data enhancement framework based on the conditional diffusion model is completed.

[0086] The technical scheme of the present application,

[0087] The structured knowledge-driven industrial defect data enhancement method proposed in the present application has wide application prospects, especially in the fields of industrial defect detection, quality control, intelligent manufacturing, etc. The following lists several specific application scenarios:

[0088] (1) Industrial production line defect detection. In high-precision and large-scale industrial production processes, especially in complex automated production lines, the detection system needs to cope with variable environmental conditions (such as changes in illumination, equipment failure, and differences in workpiece material). By introducing the data enhancement method in the present application, a variety of industrial defect images can be generated to simulate various interference conditions in the real production environment. These enhanced data can be used to train more robust defect detection models, improving the accuracy and reliability of the detection system under various complex conditions, especially in the detection of low-probability defects or special defect types, which can effectively reduce missed detection and misjudgment.

[0089] (2) Real-time quality monitoring in production process. The present application provides an industrial defect data enhancement framework based on knowledge graph, which has important applications in real-time quality monitoring systems. By combining real-time data collected during production (such as equipment status, operating parameters, and illumination conditions) with historical defect records, a dynamically updated knowledge graph is constructed. This graph can provide timely defect identification and fault prediction for real-time monitoring systems, helping production lines adjust process parameters in a timely manner to avoid potential defects, thereby effectively improving product pass rate.

[0090] (3) Improve the safety and efficiency of the industrial field. In some high-risk industrial fields, the data enhancement method proposed in this application can simulate defect images under extreme working conditions, providing more representative training data. This will greatly improve the accuracy and robustness of the defect detection system, ensuring that potential safety hazards are discovered before the product is shipped. At the same time, through efficient quality detection and automation control, production efficiency can be improved, reducing the cost and time of manual inspection, and ensuring production safety.

[0091] Through these specific applications, the application not only effectively solves the quality control problems existing in the industrial production process, but also promotes the intelligent upgrading of the manufacturing industry, improves product quality and production efficiency, and promotes the further development of intelligent manufacturing.

[0092] It should be noted that the method of the embodiments of the present disclosure can be executed by a single device, such as a computer or a server, etc. The method of the present embodiment can also be applied to a distributed scenario, and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only execute one or more steps in the method of the embodiments of the present disclosure, and the multiple devices will interact with each other to complete the method.

[0093] It should be noted that some embodiments of the present disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, it should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution. The execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The actions or steps recited in the claims can be executed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0094] Embodiment two

[0095] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a structured knowledge-driven industrial defect detection system, which is used to implement any of the methods, and the system comprises a construction module, a fusion module, a simulation module and a detection module.

[0096] The construction module is used to obtain manufacturing data related to a product, and a knowledge graph reflecting the relationship between the defect characteristics of the industrial product and the processing process and the environment is constructed using the manufacturing data, wherein the manufacturing data includes but is not limited to machine parameters, product design, production process and historical defect records.

[0097] The fusion module is configured to regard the established industrial defect knowledge graph as a heterogeneous information network, and perform deep feature aggregation on various entities and relationships in the industrial defect knowledge graph by using a heterogeneous graph neural network technology, to form a high-dimensional feature vector for guiding the detection of industrial defects under the coupling of multiple environmental factors.

[0098] The simulation module is configured to establish an industrial image illumination model by using a deep transfer learning technology according to the high-dimensional feature vector, to simulate image features of an industrial product under different acquisition conditions, wherein the acquisition conditions include light intensity, uniformity, and exposure time.

[0099] The detection module is configured to generate real and diversified images of the industrial product by training a conditional noise prediction network and constructing an industrial product defect data enhancement framework based on a conditional diffusion model according to the simulated image features of the industrial product under different acquisition conditions, to complete the detection of the industrial product defects.

[0100] The system of the above embodiment is configured to implement the structured knowledge driven industrial defect detection method of any one of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described herein.

[0101] It should be noted that the above structured knowledge driven industrial defect detection system is embodied in the form of functional units. The term "module" herein can be implemented by software and / or hardware, and is not specifically limited.

[0102] For example, the "module" can be a software program, a hardware circuit, or a combination of the two, which implements the above functions. The hardware circuit can include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor) and a memory for executing one or more software or firmware programs, a combination logic circuit, and / or other suitable components supporting the described functions.

[0103] Embodiment Three

[0104] Based on the same inventive concept, the disclosure also provides an electronic device corresponding to the method of any of the above embodiments, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the structured knowledge driven industrial defect detection method of any one of the above embodiments.

[0105] Figure 2A more specific electronic device hardware structure diagram provided by the embodiment is shown, which can include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected through the bus 1050 for communication between each other within the device.

[0106] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present specification.

[0107] The memory 1020 can be implemented by a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 1020 and called and executed by the processor 1010.

[0108] The input / output interface 1030 is used to connect input / output modules to realize information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0109] The communication interface 1040 is used to connect a communication module (not shown in the figure) to realize the communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as USB (Universal Serial Bus), a network cable, etc.) or a wireless manner (such as a mobile network, WIFI (Wireless Fidelity), Bluetooth, etc.).

[0110] The bus 1050 includes a channel to transmit information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0111] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain components necessary to implement the embodiments of the present specification, and does not necessarily contain all the components shown in the figure.

[0112] The system of the above embodiment is used to implement the structured knowledge driven industrial defect detection method of any one of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not repeated here.

[0113] Embodiment four

[0114] Based on the same inventive concept, the disclosure also provides a non-transitory computer readable storage medium corresponding to the method of any of the above embodiments, the non-transitory computer readable storage medium stores computer instructions for causing the computer to execute a structured knowledge driven industrial defect detection method according to any one of the above embodiments.

[0115] The computer readable medium of the present embodiment includes permanent and non-permanent, removable and non-removable media, which can be realized by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tape, magnetic tape magnetic disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0116] The computer instructions stored in the storage medium of the above embodiment are used to cause the computer to execute a structured knowledge driven industrial defect detection method according to any one of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which are not repeated here.

[0117] Those skilled in the art should understand that the above discussion of any embodiment is only exemplary, and is not intended to mean that the scope (including claims) of the present disclosure is limited to these examples; the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of the embodiments of the present disclosure as described above, which are not provided in details for the sake of brevity. The embodiments of the present disclosure are not limited in scope by the sum of the whole descriptions herein. Many modifications and variations will occur to those skilled in the art upon reading this disclosure. Also, it is intended that the scope of the present disclosure extend to all such modifications and variations and alternate embodiments. Accordingly, the terms "comprises", "comprising", "includes" and "including" are used herein not to limit the scope of the embodiments of the present disclosure to the listed components but to encompass the possibility that other components can be added and still be within the scope of the present disclosure. Likewise, the term "sub-comprises" is used herein to mean "consists of.

[0118] In addition, to simplify the illustration and discussion, and so as not to make the embodiments of the present disclosure difficult to understand, the well-known power / ground connections of integrated circuit (IC) chips and other components can or can not be shown in the provided drawings. Furthermore, the apparatus can be shown in the form of block diagrams in order to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform to be implemented (i.e., these details should be fully within the understanding of those skilled in the art). Where specific details (e.g., circuitry) are set forth in order to describe an illustrative embodiment of the present disclosure, it will be apparent to those skilled in the art that the embodiments of the present disclosure can be practiced without these specific details or with an equivalent arrangement. Therefore, these descriptions should be considered as illustrative and not restrictive.

[0119] Although the present disclosure has been described in conjunction with the specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0120] Therefore, the units of each example described in the embodiments of the present application can be realized in electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solutions. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0121] The embodiments of the present application are intended to cover all such alternatives, modifications and variations as falling within the broad scope of the appended claims. Accordingly, any and all departures from the embodiments of the present application that are within the spirit and scope of the present application are intended to be included.

Claims

1. A structured knowledge-driven industrial defect detection method, characterized in that: The method comprises: S1: Obtain manufacturing data related to the product and use the manufacturing data to construct a knowledge graph that reflects the relationship between industrial product defect characteristics and the processing process and environment, where the manufacturing data includes machine parameters, product design, production process, and historical defect records; S2: Treat the established industrial defect knowledge graph as a heterogeneous information network, and use heterogeneous graph neural network technology to perform deep feature aggregation on various entities and relationships in the industrial defect knowledge graph to form a high-dimensional feature vector for guiding the coupling of multiple environmental factors; S3: Based on the high-dimensional feature vector, deep transfer learning technology is used to establish an industrial image illumination model to simulate the image characteristics of industrial products under different acquisition conditions, where the acquisition conditions include light intensity, uniformity, and exposure time. S4: Based on the image features of industrial products under different simulated acquisition conditions, a conditional noise prediction network is trained to build an industrial product defect data enhancement framework based on the conditional diffusion model. Based on this industrial product defect data enhancement framework, realistic and diverse industrial product images are generated to complete industrial product defect detection; The method of using heterogeneous graph neural network technology to perform deep feature aggregation on various entities and relationships in the industrial defect knowledge graph to form a high-dimensional feature vector for guiding the coupling of multiple environmental factors includes: According to formula (1), the serialized entity features are initialized and embedded: ； in, Represents the serialized representation of environmental characteristics, equipment characteristics and product characteristics respectively, Represent the initial embedding representations corresponding to environment features, device features, and product features, respectively. They represent the initialization embedding parameter matrix, Respectively represent the initialization embedding bias parameter vector, represents a nonlinear activation function; According to the feature extraction module defined by formula (2), the initialized embedded features are further aggregated: (2)； in, Respectively represent the corresponding environmental characteristics, equipment characteristics and product characteristics Layer hidden variables, GCN represents the graph convolution mechanism, GAT represents the graph attention mechanism, Indicates the The embedding parameter matrix of the relational attribute matrix of the layer, The parameter matrix representing the adjacency matrix; After multiple rounds of knowledge graph feature aggregation, the defect feature guidance vector is obtained through the linear layer ; Methods for building industrial image illumination models using deep transfer learning technology include: By pre-training a deep learning model from an existing large-scale natural scene image dataset, the pre-trained deep learning model is then transferred to an existing typical industrial defect dataset; specifically: A deep U-Net network based on the ResNet architecture is trained on the large-scale ImageNet natural scene dataset, enabling the U-Net network to learn key features that fully characterize natural scenes, including light-shadow interaction, light field gradients, and over- and under-exposed areas. According to the characteristics of industrial product images, a multi-scale image feature enhancement network based on the pyramid mechanism is established to enhance multi-scale industrial product feature images; The U-Net network is initially fine-tuned using the enhanced multi-scale industrial product feature images, and the U-Net network is deepened or widened as required based on the resolution requirements of the industrial products. Based on the freezing of some feature extraction layers in the network, the noise injection matrix and noise sampling step size are introduced to form a set of illumination models related to the degree of injected noise. .

2. The method according to claim 1, characterized in that The construction method of industrial defect knowledge graph includes: Acquire industrial product-related data, wherein the industrial product-related data includes industrial product design, manufacturing process, quality inspection records, manufacturing environment parameter manufacturing process data, and preprocess the industrial product-related data, specifically including: data cleaning and data standardization; Identifying knowledge entities from pre-processed industrial product-related data, wherein the knowledge entities include: product type, defect type, detection equipment, and environmental parameters; Define the attributes of knowledge entities, including defect size, shape, illumination uniformity, and noise transmission level of acquisition equipment; Determine the relationship types between knowledge entities, including: <product-use-raw materials>, <equipment-production-product>, <operator-operation-equipment>, <defect-affect-product performance>, <process flow-includes-operation steps>, <detection method-applicable to-defect type>, <environmental conditions-influence-defect occurrence>, <repair method-solve-defect>, <quality standard-regulation-defect level>, <supplier-provides-raw materials>, <product-complies with-safety standards>, <production batch-includes-product>; The collected data and extracted triples are integrated into a unified Neo4j-based database to complete the construction of the industrial defect knowledge graph.

3. The method according to claim 1, characterized in that The construction method of the industrial product defect data enhancement framework based on the conditional diffusion model includes: According to formula (3), the industrial sample image sampled under the standard environment is subjected to noise processing: ； in, represents the standard Wiener process, represents the Gaussian coefficient of step t, represents the variance of the added noise at step t, To represent the feature distribution of the original image, that is, the initial feature representation under noise-free conditions, To represent the sampled image features at step t, the feature representation after multi-step noise generation, To represent the original image features The standard normal distribution of Using lighting models , as the initial network of the conditional noise prediction network, the noise introduced in the tth step is used as the prediction label, and the aggregated As a conditional guidance vector, train the conditional noise prediction network ,in, is the conditional guidance vector representing the t-th step; Using Conditional Noise Prediction Network The randomly sampled Gaussian noise matrix is ​​decoded to obtain a complete set of industrial defect data sets reflecting uncertain environmental characteristics, and the construction of an industrial product defect data enhancement framework based on the conditional diffusion model is completed.

4. A structured knowledge-driven industrial defect detection system, the system being used to implement the method according to any one of claims 1 to 3, characterized in that: The system includes: a construction module, a fusion module, a simulation module and a detection module; The construction module is used to obtain manufacturing data related to the product and use the manufacturing data to construct a knowledge graph that reflects the relationship between industrial product defect characteristics and the processing process and environment, wherein the manufacturing data includes machine parameters, product design, production process and historical defect records; The fusion module is used to treat the established industrial defect knowledge graph as a heterogeneous information network, and use heterogeneous graph neural network technology to perform deep feature aggregation on various entities and relationships in the industrial defect knowledge graph to form a high-dimensional feature vector for guiding the coupling of multiple environmental factors; The simulation module is used to establish an industrial image illumination model based on the high-dimensional feature vector using deep transfer learning technology to simulate the image characteristics of industrial products under different acquisition conditions, wherein the acquisition conditions include light intensity, uniformity, and exposure time factors; The detection module is used to construct an industrial product defect data enhancement framework based on the conditional diffusion model by training a conditional noise prediction network based on the image features of industrial products under different simulated acquisition conditions. Based on the industrial product defect data enhancement framework, realistic and diverse industrial product images are generated to complete industrial product defect detection.

5. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 3 is implemented.

Images