Semantic communication method for industrial anomaly detection task

Abstract

The invention relates to a semantic communication method for an industrial anomaly detection task, and belongs to the field of industrial anomaly detection. The method comprises an edge-end collaborative industrial image semantic transmission architecture, a high-precision industrial image anomaly detection network and an anomaly detection-oriented semantic feature extraction model. In the training stage, the edge server trains an anomaly detection model of a certain industrial product type, and after training is completed, an anomaly detection network is issued to the terminal equipment. In the test stage, when a certain type of industrial product image is input, the terminal equipment performs semantic coding and channel coding on the input image to obtain semantic features, and semantic information is directly transmitted to the corresponding edge server through a wireless channel. And when the edge server receives the semantic features, channel decoding and semantic decoding are carried out, and an industrial anomaly detection task result is directly output. According to the invention, the data transmission pressure and the edge calculation load in the industrial Internet of Things can be reduced, and the real-time surface anomaly detection of the industrial product is realized.

Description

Technical Field

[0001] The present invention belongs to the field of industrial anomaly detection and relates to a semantic communication method for industrial anomaly detection tasks. Background Art

[0002] With the development of industrial automation and intelligence, industrial anomaly detection technology plays an increasingly important role in ensuring product quality, improving production efficiency and maintaining production stability. Industrial anomaly detection aims to identify and locate various defects in the production process of industrial products, so as to take timely measures to repair them and avoid greater losses.

[0003] “Specified types of industrial products” refer to:

[0004] Industrial products produced on the same production line or production link: for example, auto parts produced on the same automobile production line, electronic components produced on the same electronic product production line, etc.

[0005] Industrial products with similar appearance features and manufacturing processes: for example, different models of mobile phones, different brands of cars, etc.

[0006] Industrial products for specific industries or application scenarios: for example, parts for the aerospace field, parts for medical equipment, etc.

[0007] With the development of deep learning technology, anomaly detection methods in industrial images can be divided into two categories: reconstruction-based methods and embedding-based methods. In current research, most reconstruction-based methods train networks based on encoder-decoder architectures to reconstruct input images and reconstruct abnormal images with large errors. Vitjan Zavrtanik et al. proposed an abnormal sample generation method to obtain anomalies that are just out of distribution, combining reconstruction subnetworks and identification subnetworks to discriminate images in an end-to-end manner. However, in practical applications, the learning ability of neural networks is too strong, so it is possible to reconstruct abnormal areas in images well. Embedding-based methods usually use pre-trained networks on general datasets to extract high-level features of the original image, and calculate the anomaly score by the distance between the features of the test sample and the normal sample to obtain the abnormal area. However, the existing differences between the industrial anomaly detection dataset and the pre-training dataset are ignored, and the direct use of the pre-trained network will cause domain mismatch problems. Niv Cohen et al. designed a nearest neighbor method of deep pre-trained features to detect anomalies in images by pixel-level correspondence between the test image and the normal image. Embedding-based models have been widely used due to their simplicity and effectiveness. However, complex feature matching processes are often required in the reasoning stage, which greatly limits the reasoning speed of the model. Semantic communication is a communication paradigm that extracts and transmits semantic information based on a common and understandable knowledge base. It directly transmits the meaning of the information itself to complete intelligent tasks. Huiqiang Xie et al. proposed a semantic communication system for text transmission, using a convolutional neural network based on a transformer and encoder-decoder architecture to maximize system capacity and minimize semantic errors.

[0008] However, existing industrial anomaly detection methods still face the following challenges:

[0009] (1) In actual industrial production, the vast majority of samples are normal, while the number of abnormal samples of specific types of industrial products is scarce, which makes it difficult for anomaly detection methods based on supervised learning to train effective models.

[0010] (2) There are often distribution differences between industrial anomaly detection datasets and datasets used by pre-trained models. Directly using pre-trained models will lead to domain mismatch problems, affecting the performance of anomaly detection models for specific types of industrial products.

[0011] (3) Industrial production scenes are complex and changeable, image acquisition conditions are poor, and there are many interference factors such as lighting and noise, which poses great challenges to the transmission and recognition of images of specific types of industrial products.

[0012] (4) In the industrial Internet of Things environment, a large number of sensors and devices generate massive amounts of data in real time, resulting in a sharp increase in bandwidth demand, while existing communication networks often cannot meet the needs of real-time data transmission.

[0013] In order to solve the above problems, the present invention proposes a semantic communication method for anomaly detection tasks of specific types of industrial products. Summary of the invention

[0014] In view of this, the purpose of the present invention is to provide a semantic communication method for industrial anomaly detection tasks. In view of the problem that the traditional architecture does not consider the insufficient computing power of the local server, the present invention introduces an edge-end collaborative architecture, fully considers the characteristics of edge-end collaborative technology, and uses semantic communication in combination with edge computing resources to perform collaborative industrial anomaly detection. Secondly, in view of the problem that the traditional model ignores the distribution difference between the data set used by the pre-training model and the industrial anomaly detection data set, a high-precision industrial image anomaly detection network is proposed. The proposed network first designs an anomaly simulation method, then designs a feature adapter to reduce the difference in data set distribution, and finally makes full use of features at different levels for industrial anomaly detection tasks. In addition, since the traditional communication method uses a classical compression algorithm to compress images, it does not take into account that the information at different positions of the image has different importance. The present invention proposes a semantic feature extraction model for anomaly detection. The proposed model combines a high-precision industrial anomaly detection network to design a semantic encoding method to extract key information from the image, which can achieve accurate and efficient transmission of images under low signal-to-noise ratio conditions to improve the robustness of industrial anomaly detection tasks.

[0015] In order to achieve the above object, the present invention provides the following technical solutions:

[0016] A semantic communication method for industrial anomaly detection tasks, comprising:

[0017] The edge server trains anomaly detection models for specific types of industrial products and deploys the trained anomaly detection network to terminal devices;

[0018] The terminal device performs semantic coding and channel coding on the input image of a specific type of industrial product to extract key semantic features;

[0019] The extracted semantic features are transmitted to the corresponding edge server through a wireless channel;

[0020] The edge server performs channel decoding and semantic decoding on the received semantic features, and uses its powerful computing resources to output the results of the industrial anomaly detection task.

[0021] The end-edge collaborative industrial anomaly detection semantic transmission architecture is mainly composed of terminal devices and edge servers, where the terminal devices are mainly responsible for collecting industrial product images, extracting the semantic features of the images and sending them to the edge servers. The edge servers receive the semantic features of the industrial images and use the corresponding models to perform industrial anomaly detection. The method proposed in the present invention uses the feature extractor of the high-precision anomaly detection network as a semantic encoder to map the input image to the feature space, which can greatly reduce the amount of transmitted data, and then transmit it through a wireless link to obtain the anomaly detection results using the powerful computing resources of the edge server.

[0022] The domain adaptive high-precision industrial image anomaly detection network consists of five parts: feature extractor, anomaly generator, feature adapter, feature fusion module and discriminator. In the training phase, first, the training set in the image dataset is divided into two parts, one of which is extracted by the feature extractor to extract image features, combined with the noise generator to generate negative samples, and then sent to the feature adapter, and the other part directly enters the feature adapter. Secondly, the feature adapter combines the feature fusion module to process the input feature map to obtain feature maps with different receptive fields. Finally, the discriminator performs end-to-end positioning of the abnormal area in the image to obtain the anomaly score and anomaly segmentation map. In the testing phase, the noise generator will be removed.

[0023] The semantic feature extraction model mainly consists of three parts: the transmitter, the wireless channel, and the receiver. At the transmitter, first, semantic information is extracted from the input image, that is, the key information related to the task in the image. The extracted information is further compressed after semantic encoding. Finally, the information is transmitted through the wireless channel after channel encoding and modulation. The knowledge base is composed of the parameter set of the trained anomaly detection network, which contains the knowledge of the anomaly detection task. At the receiver, after the signal is demodulated and channel decoded, the knowledge base is used for semantic decoding, and the task result is finally output. The channel coding and channel decoding are composed of neural network layers.

[0024] The beneficial effects of the present invention are:

[0025] (1) Through the domain adaptive high-precision industrial image anomaly detection network, the problem of data set distribution differences can be effectively solved and the performance of anomaly detection models for specific types of industrial products can be improved.

[0026] (2) Through semantic coding and channel coding, the amount of information to be transmitted can be effectively compressed, reducing the pressure of data transmission in the industrial Internet of Things.

[0027] (3) By migrating some computing tasks to terminal devices, the computing load of the edge server can be reduced and the overall operating efficiency of the system can be improved.

[0028] (4) Through semantic communication, it can effectively resist noise interference in the wireless channel, improve the robustness of industrial anomaly detection tasks, and maintain high detection accuracy even in complex industrial scenarios.

[0029] (5) The end-edge collaborative architecture enables real-time anomaly detection on the surface of industrial products, timely discovery and processing of problems, and avoids causing greater losses.

[0030] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0032] Figure 1 is a flow chart of the present invention;

[0033] Figure 2 This is a diagram of the industrial image semantic transmission architecture of the end-edge collaboration described in the present invention;

[0034] Figure 3 A high-precision industrial image anomaly detection network diagram designed for the present invention;

[0035] Figure 4 A diagram of the architecture of a semantic feature extraction model for anomaly detection provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0037] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0038] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0039] 1. Industrial Image Semantic Transmission Architecture with Device-Edge Collaboration

[0040] Figure 1 Flow chart of the present invention. Figure 2 As shown, the end-edge collaborative industrial image semantic transmission architecture proposed in the present invention is mainly composed of terminal devices and edge servers.

[0041] Terminal device: The terminal device is responsible for collecting images of specific types of industrial products, performing semantic encoding and channel encoding on them, extracting key semantic features, and then transmitting the encoded semantic features to the edge server through a wireless channel.

[0042] Edge server: The edge server receives the semantic features transmitted by the terminal device, performs channel decoding and semantic decoding on them, restores the feature map, and then uses the trained domain adaptive high-precision industrial image anomaly detection network to perform anomaly detection and output the detection results.

[0043] 2. High-precision Industrial Image Anomaly Detection Network

[0044] Figure 3 A high-precision industrial image anomaly detection network structure provided in a specific embodiment of the present invention specifically includes the following steps:

[0045] (1) Image feature extraction

[0046] For any image in the dataset, the pre-trained feature extraction network extracts features from different hierarchies. Define φ l is the feature extraction network, l is the subset containing the hierarchical parameters. The feature map of the i-th layer (i∈l) of the backbone network is expressed as Among them, H l , W l , C lare the height, width, and number of channels of the feature map, respectively. The first layer of the feature extraction network tends to capture low-level features of the image, such as texture and color information, while the last layer usually tends to capture higher-level, more abstract features. In order to combine features from different layers All feature maps are upsampled to the same size (h, w), which is the size of the largest feature map. Then the feature maps are connected by channel to obtain the feature map

[0047] (2) Abnormal image generation

[0048] In order to train the neural network model to identify the possibility of normal samples, supervised deep learning methods put positive and negative samples into the model for training. However, the number of negative samples in anomaly detection tasks is usually small, so training is impossible. Existing research usually relies on additional data to synthesize real defect images. However, the defect images synthesized by such methods are often very different from the defect images that appear in reality. Therefore, semantic-level anomalies are added to the feature space of normal samples, that is, noise is added. The anomaly generator generates a noise signal in the normal feature map. Gaussian noise is added to generate abnormal features. Specifically, the Gaussian noise vector is sampled, where each item ò∈N(μ,σ 2 ) follow independent and identical distribution. Abnormal characteristics Obtained by the fusion of both.

[0049] (3) Feature Fusion Module

[0050] Since industrial image datasets usually have inherent target domain differences with the datasets used in pre-trained networks, directly using the features extracted by the pre-trained network will cause domain mismatch problems. θ To process the input feature map, it is composed of fully connected layers, making the pre-trained network more suitable for industrial anomaly detection tasks. Feature transformer T θ Transform the input feature map into the adapted feature map Existing anomaly recognition methods are often incapable of identifying small-area anomalies. The main reason is that these models are difficult to effectively capture subtle features within a small range. In the multi-layer feature extraction process of the network, the features of small-area anomalies may be diluted or ignored, thereby reducing the accuracy of detection. In order to solve this problem, the present invention introduces a multi-scale feature fusion method to enhance the detailed information captured under different receptive fields.

[0051] For the top-down path, feature maps {P1, P2, P3} of different scales are obtained by downsampling the input feature map P1 of the module layer by layer. Then, a 1*1 convolution operation is performed on P3 to obtain the feature map C1, followed by layer-by-layer upsampling and lateral connection operations to obtain feature maps {C1, C2, C3} of different scales.

[0052] (4) Anomaly Identifier

[0053] A two-layer multilayer perceptron structure is used as the discriminator D. As an anomaly scorer, it directly estimates the anomaly score of each position (h, w) in the feature map. The features of negative samples and normal samples are fed into the classifier D during training, expecting the output of normal samples to be positive and the output of abnormal features to be negative.

[0054] 3. Semantic Feature Extraction Model for Anomaly Detection

[0055] Figure 4 A semantic feature extraction model application architecture for anomaly detection is provided for a specific embodiment of the present invention. The feature processing and anomaly discriminator are composed of the trained anomaly detection model designed in Section 4, and its parameters will be frozen at this stage. The dense layer and reshaping layer at the transmitting end play the role of channel coding, and the dense layer and reshaping layer at the receiving end play the role of channel decoding. In general, its purpose is to further reduce the amount of transmitted information while reducing the impact of wireless channel noise on model performance. Therefore, the operation of the transmitting end on the image can be expressed as b i =f fe (a i ,θ),x i =f cc (b i ), where a i ∈{a 1 ,a 2 ,...,a n} is the original input image set; is the feature processing process of the image; θ is the parameter set of the industrial anomaly detection model. i ∈{b 1 ,b 2 ,...,b n} is the set of feature maps obtained by the image through the feature processing module; is the channel coding process of the feature map; x i ∈{x 1 ,x 2 ,...,x n} is a set of feature graphs containing important semantics obtained by the transmitter through channel coding. i, reaches the receiving end through the physical channel. The signal at the receiving end affected by the channel noise can be expressed as y i =hx i +n, where y i ∈{y 1 ,y 2 ,...,y n} is the semantic feature map received by the receiver; h is the channel gain of the Rayleigh fading channel; n is Gaussian white noise (AWGN). For end-to-end training of the encoder and decoder, the physical channel must allow back propagation, so the physical channel can be modeled using a neural network. First, the semantic feature map y is decoded through the channel to obtain the restored feature map d i , and then input the feature map into the anomaly detector, that is, the anomaly classifier of the anomaly detection model designed in Section 4, to obtain the final task result. The process can be expressed as d i =f cd (y i ), Where d i is the feature map restored after channel decoding; is the channel decoding process; r i The final output of the task is the anomaly classification result and anomaly segmentation map; It is the semantic recovery process; is the parameter set of the semantic restoration process. The goal is to keep the performance of the model as stable as possible under harsh communication conditions, that is, to make the information received by the receiver as consistent as possible with the information sent by the transmitter. Therefore, the mean square error (MSE) can be used as the objective function of the image semantic model, and the loss function is The purpose is to minimize the distance between the feature maps of the transmitter and the receiver, that is, to minimize the impact of noise in the wireless channel on the performance of the industrial anomaly detection network.

[0056] 4. Training Process

[0057] The training process of the domain adaptive high-precision industrial image anomaly detection network and the semantic feature extraction model for anomaly detection proposed in the present invention is as follows:

[0058] Data preparation: Collect image datasets of specific types of industrial products and annotate them to mark normal and abnormal areas in the images.

[0059] Model training: Divide the dataset into training set and test set. Use the training set to train the domain adaptive high-precision industrial image anomaly detection network and the semantic feature extraction model for anomaly detection.

[0060] Model evaluation: Use the test set to evaluate the trained model. Evaluation indicators include anomaly score, accuracy of anomaly segmentation map, etc.

[0061] 5. Application Scenarios

[0062] The semantic communication method for industrial anomaly detection tasks proposed in this invention can be applied to various industrial production scenarios, such as:

[0063] Automobile manufacturing: used to detect surface defects of automobile parts, such as scratches, cracks, deformation, etc.

[0064] Electronic product manufacturing: used to detect defects on the surface of electronic components, such as poor solder joints, device damage, etc.

[0065] Aerospace Manufacturing: Used to detect surface defects of aerospace parts, such as fatigue cracks, corrosion, etc.

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A semantic communication method for industrial anomaly detection tasks, characterized by: include: The edge server trains anomaly detection models for specific types of industrial products and deploys the trained anomaly detection network to terminal devices; The terminal device performs semantic coding and channel coding on the input image of a specific type of industrial product to extract key semantic features; The extracted semantic features are transmitted to the corresponding edge server through a wireless channel; The edge server performs channel decoding and semantic decoding on the received semantic features, and uses its powerful computing resources to output the results of the industrial anomaly detection task.

2. The semantic communication method for industrial anomaly detection tasks according to claim 1 is characterized in that: The anomaly detection model trained by the edge server is a domain adaptive high-precision industrial image anomaly detection network, including: A feature extractor for extracting image features; An anomaly generator for generating anomaly samples; Feature adapters to reduce the distribution differences of datasets; Fusion module for fusing features of different receptive fields; Discriminator for locating anomalous regions in images.

3. The semantic communication method for industrial anomaly detection tasks according to claim 1 is characterized in that: The terminal device performs semantic coding and channel coding on the input image, specifically including: Using the feature extractor of the domain adaptive high-precision industrial image anomaly detection network to semantically encode the input image and extract key semantic features; The extracted key semantic features are channel coded to compress the amount of information that needs to be transmitted.

4. The semantic communication method for industrial anomaly detection tasks according to claim 1 is characterized in that: The edge server performs channel decoding and semantic decoding on the received semantic features, specifically including: Channel-decoding the received semantic features using a channel decoder corresponding to the domain adaptive high-precision industrial image anomaly detection network to restore the feature map; The restored feature map is input into the discriminator of the domain adaptive high-precision industrial image anomaly detection network for semantic decoding, and the industrial anomaly detection task result is output.

5. The semantic communication method for industrial anomaly detection tasks according to claim 1 is characterized in that: The training process of the domain adaptive high-precision industrial image anomaly detection network includes: The training set of the image dataset is divided into two parts; Extract features from the first part of the image and combine it with the noise generated by the noise generator to generate abnormal samples, which are then input into the feature adapter; The second part of the image is directly input into the feature adapter; The input feature map is processed by the feature adapter and feature fusion module to obtain feature maps with different receptive fields; The discriminator is used to locate abnormal areas in the image end-to-end, and anomaly scores and anomaly segmentation maps are obtained.

6. The semantic communication method for industrial anomaly detection tasks according to claim 1 is characterized in that: The channel encoder and decoder are composed of neural network layers.

7. The semantic communication method for industrial anomaly detection tasks according to claim 1 is characterized in that: The objective function of the semantic feature extraction model is the mean square error, which minimizes the distance between the feature maps of the transmitter and the receiver, thereby minimizing the impact of noise in the wireless channel on the performance of the industrial anomaly detection network.

8. The semantic communication method for industrial anomaly detection tasks according to claim 1 is characterized by: The terminal device comprises: An image acquisition module for acquiring images of industrial products; A coding module for performing semantic coding and channel coding on the acquired images; A transmission module for transmitting the encoded semantic features to an edge server.

9. The semantic communication method for industrial anomaly detection tasks according to claim 1, characterized in that: The edge server comprises: A receiving module for receiving semantic features transmitted by a terminal device; A decoding module for performing channel decoding and semantic decoding on received semantic features; Anomaly detection module for performing industrial anomaly detection tasks.

Images