Abnormality detection method, device and electronic equipment

Abstract

The application discloses an abnormality detection method, device and electronic equipment. The method comprises the following steps: receiving semantic transmission signals of multiple physical channel transmissions, wherein the semantic transmission signals are obtained by sequentially compressing and encoding multiple types of industrial data collected by multiple unmanned aerial vehicles by using a first encoder and a second encoder; performing channel compression decoding on the semantic transmission signals according to at least channel state information of the physical channels to obtain compressed semantic signals; performing semantic decoding on the compressed semantic signals according to a first decoder to obtain first semantic signals; extracting feature data in the first semantic signals by using a multi-modal semantic receiver, determining an abnormality score of the multiple types of industrial data according to the feature data; and determining that there is an abnormality in the multiple types of industrial data when the abnormality score is greater than a preset threshold. The application solves the technical problems of a current unmanned aerial vehicle group-based industrial abnormality detection system, such as short endurance time, high communication cost and low communication efficiency.

Description

Technical Field

[0001] This application relates to the field of data encoding, and more specifically, to an anomaly detection method, apparatus, and electronic device. Background Technology

[0002] In recent years, edge intelligence has attracted significant attention from both academia and industry. Specifically, edge intelligence is defined as advanced edge computing that possesses machine learning, distributed computing, and advanced networking capabilities, processing and storing data at network edge nodes. Edge intelligence efficiently utilizes all computing and storage resources along the terminal-edge-cloud path through collaboration, comprehensively enhancing data processing capabilities and promoting the implementation and widespread adoption of industrial intelligent applications. Edge intelligence will effectively address many challenges currently facing the development of the Industrial Internet, such as resource allocation, high-performance computing, and industrial anomaly detection. In the field of industrial anomaly detection, particularly in scenarios based on drone swarms, current industrial anomaly detection in this scenario suffers from problems such as reliance on single-modal data, short battery life, high communication costs, and inefficient communication content.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides an anomaly detection method, apparatus, and electronic device to at least solve the technical problems of short battery life, high communication cost, and inefficient communication content in current industrial anomaly detection systems based on UAV swarms.

[0005] According to one aspect of the embodiments of this application, an anomaly detection method is provided, comprising: receiving semantic transmission signals transmitted through multiple physical channels, wherein the semantic transmission signals are obtained by sequentially compressing and encoding multiple types of industrial data collected by multiple drones using a first encoder and a second encoder, wherein the first encoder is used to perform semantic compression on the semantic information of the multiple types of industrial data, and the second encoder is used to perform channel compression on the compressed semantic information; performing channel compression decoding on the semantic transmission signals at least based on the channel state information of the physical channels to obtain a compressed semantic signal; performing semantic decoding on the compressed semantic signal based on a first decoder to obtain a first semantic signal; extracting feature data from the first semantic signal using a multimodal semantic receiver, determining anomaly scores of the multiple types of industrial data based on the feature data; and determining that anomalies exist in the multiple types of industrial data if the anomaly score is greater than a preset threshold.

[0006] Optionally, receiving semantic transmission signals transmitted through multiple physical channels includes: receiving multiple types of industrial data collected by multiple drones, wherein the multiple types of industrial data include image data, text data, and voice data; performing semantic compression on the semantic information of the multiple types of industrial data according to a first encoder to obtain compressed semantic information, wherein the first encoder is a semantic encoder corresponding to the industrial data collected by each drone, and the first learning parameter in the first encoder has a corresponding relationship with each drone; inputting the compressed semantic information into a second encoder for channel compression to obtain a semantic transmission signal, and transmitting the semantic transmission signal through multiple physical channels, wherein the second encoder is a channel encoder corresponding to the industrial data collected by each drone, and the second learning parameter in the second encoder has a corresponding relationship with each drone.

[0007] Optionally, the semantic transmission signal is compressed and decoded at least based on the channel state information of the physical channel to obtain a compressed semantic signal, including: determining the received signal based at least on the semantic transmission signal and the channel matrix, wherein the channel matrix is ​​the channel matrix between the roadside unit and the UAV, and the roadside unit is a computing unit fixed at a specific location; determining the estimated value of the channel state information based on the channel state information and the error estimate of the channel state information; and determining the compressed semantic signal based on the received signal and the estimated value of the channel state information.

[0008] Optionally, semantic decoding of the compressed semantic signal based on the first decoder to obtain the first semantic signal includes: determining a third learning parameter in the first decoder, wherein the first decoder is a decoder corresponding to the second encoder, the third learning parameter has a corresponding relationship with each UAV, and the first decoder is used to decompress semantic information; and semantic decoding of the compressed semantic signal based on the third learning parameter to obtain the first semantic signal.

[0009] Optionally, the feature data in the first semantic signal is extracted using a multimodal semantic receiver, including: determining a fourth learning parameter in a second decoder, wherein the second decoder is a decoder corresponding to the first encoder, and the fourth learning parameter has a corresponding relationship with each UAV; decoding the first semantic signal according to the fourth learning parameter to obtain a target semantic signal; and extracting feature data in the target semantic signal using a multimodal semantic receiver, wherein the feature data includes: first feature data corresponding to image data, second feature data corresponding to text data, and third feature data corresponding to speech data.

[0010] Optionally, after extracting feature data from the first semantic signal using a multimodal semantic receiver, the method further includes: fusing the first feature data, the second feature data, and the third feature data to obtain fused feature data; performing feature reconstruction on the fused feature data to obtain target feature data; and determining the anomaly scores of multiple types of industrial data based on the target feature data.

[0011] Optionally, determining the anomaly score of multiple types of industrial data based on feature data includes: obtaining the true feature value and corresponding output feature value of each type of industrial data in the target feature data, wherein the output feature value is the feature value of the feature data extracted by the multimodal semantic receiver; and determining the anomaly score of each type of industrial data based on the true feature value, the output feature value and the weight value of each type of industrial data, wherein the weight value is determined by the feature dimension of each type of industrial data in the target feature data.

[0012] According to another aspect of the embodiments of this application, an anomaly detection device is also provided, comprising: a receiving module, configured to receive semantic transmission signals transmitted through multiple physical channels, wherein the semantic transmission signals are obtained by sequentially compressing and encoding multiple types of industrial data collected by multiple drones using a first encoder and a second encoder, the first encoder being used to perform semantic compression on the semantic information of the multiple types of industrial data, and the second encoder being used to perform channel compression on the compressed semantic information; a first decoding module, configured to perform channel compression decoding on the semantic transmission signals at least based on the channel state information of the physical channels to obtain a compressed semantic signal; a second decoding module, configured to perform semantic decoding on the compressed semantic signal based on the first decoder to obtain a first semantic signal; a first determining module, configured to extract feature data from the first semantic signal using a multimodal semantic receiver, and determine anomaly scores for the multiple types of industrial data based on the feature data; and a second determining module, configured to determine that anomalies exist in the multiple types of industrial data when the anomaly score is greater than a preset threshold.

[0013] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory for storing program instructions; and a processor connected to the memory for executing program instructions to perform the following functions: receiving semantic transmission signals transmitted through multiple physical channels, wherein the semantic transmission signals are obtained by sequentially compressing and encoding multiple types of industrial data collected by multiple drones using a first encoder and a second encoder, the first encoder being used to perform semantic compression on the semantic information of the multiple types of industrial data, and the second encoder being used to perform channel compression on the compressed semantic information; performing channel compression decoding on the semantic transmission signals at least based on the channel state information of the physical channels to obtain a compressed semantic signal; performing semantic decoding on the compressed semantic signal based on a first decoder to obtain a first semantic signal; extracting feature data from the first semantic signal using a multimodal semantic receiver, determining anomaly scores for the multiple types of industrial data based on the feature data; and determining that anomalies exist in the multiple types of industrial data when the anomaly score is greater than a preset threshold.

[0014] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned anomaly detection method by running the computer program.

[0015] In this embodiment, semantic transmission signals transmitted through multiple physical channels are received. These signals are obtained by sequentially compressing and encoding various types of industrial data collected by multiple drones using a first encoder and a second encoder. The first encoder performs semantic compression on the semantic information of the various types of industrial data, and the second encoder performs channel compression on the compressed semantic information. The semantic transmission signals are then channel-compressed and decoded based on the channel state information of the physical channels to obtain a compressed semantic signal. The compressed semantic signal is then semantically decoded using a first decoder to obtain a first semantic signal. Feature data is extracted from the first semantic signal using a multimodal semantic receiver, and anomaly scores for the various types of industrial data are determined based on the feature data. If the anomaly score exceeds a preset threshold, anomalies are determined to exist in the various types of industrial data. This achieves the goal of detecting abnormal data in industrial data using a multimodal semantic receiver, thereby improving the technical performance of anomaly detection and solving the technical problems of short battery life, high communication costs, and inefficient communication content in current drone-based industrial anomaly detection systems. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1This is a hardware structure block diagram of a computer terminal (or electronic device) for implementing an anomaly detection method according to an embodiment of this application;

[0018] Figure 2 This is a flowchart of an anomaly detection method according to an embodiment of this application;

[0019] Figure 3 This is a structural diagram of an anomaly detection device according to an embodiment of this application. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] First, some nouns or terms that appear in the explanation of the embodiments of this application shall be interpreted as follows:

[0023] Semantic information: The concept of semantic information has been continuously developed and improved since its inception. Early work on semantic communication was mainly based on Shannon's probabilistic information theory, using information entropy as a foundation, supplemented by logical inference and fuzzy transformation.

[0024] Semantic communication: Semantic communication is content-aware, task-oriented, and semantically relevant, where only information that is important, relevant, and useful to the user / application is extracted from large amounts of data and delivered to the destination. Existing semantic communication work can be divided into two parts: data reconstruction-based and task execution-based.

[0025] Multimodal learning: Single-modal representation learning is responsible for representing information as numerical vectors that computers can process or further abstracting it into higher-level feature vectors. Multimodal representation learning refers to learning better feature representations by utilizing the complementarity between multiple modalities and eliminating redundancy between modalities.

[0026] Anomaly detection: In data mining, anomaly detection is the identification of items, events, or observations that do not conform to expected patterns or other items in the dataset.

[0027] Current industrial anomaly detection systems based on drone swarms suffer from problems such as reliance on single-modal data, short flight time, high communication costs, and inefficient communication content. Therefore, drones, as edge devices, are limited by their own battery power, data storage, and computing capabilities, resulting in low efficiency for the entire anomaly detection system. To address these issues, this application's embodiments no longer use traditional communication systems. Instead, they introduce a multi-user (i.e., distributed) semantic communication system as the communication framework for the entire anomaly detection system to reduce communication costs and improve semantic communication efficiency. Furthermore, a multimodal learning algorithm is designed to further enhance the performance of industrial anomaly detection. The following is a detailed description.

[0028] The anomaly detection method embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware block diagram of a computer terminal (or electronic device) for implementing an anomaly detection method is shown. Figure 1 As shown, the computer terminal 10 (or electronic device 10) may include one or more processors (shown as 102a, 102b, ..., 102n in the figure) (the processor may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0029] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be wholly or partially embodied in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuit may be a single, independent processing module, or may be wholly or partially integrated into any other element within the computer terminal 10 (or electronic device). As involved in the embodiments of this application, the data processing circuit serves as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0030] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the anomaly detection method in this embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned anomaly detection method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0031] The transmission module 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission module 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission module 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0032] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or electronic device).

[0033] It should be noted here that, in some optional embodiments, the above... Figure 1 The computer device (or electronic device) shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 1 This is only one instance of a particular specific instance, and is intended to illustrate the types of components that may exist in the aforementioned computer equipment (or electronic equipment).

[0034] In the above operating environment, this application provides an embodiment of an anomaly detection method. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] Figure 2 This is a flowchart of an anomaly detection method according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:

[0036] Step S202: Receive semantic transmission signals transmitted through multiple physical channels. The semantic transmission signals are obtained by sequentially compressing and encoding multiple types of industrial data collected by multiple UAVs using a first encoder and a second encoder. The first encoder is used to perform semantic compression on the semantic information of the multiple types of industrial data, and the second encoder is used to perform channel compression on the compressed semantic information.

[0037] Step S204: At least based on the channel state information of the physical channel, perform channel compression decoding on the semantic transmission signal to obtain a compressed semantic signal;

[0038] Step S206: Semantically decode the compressed semantic signal according to the first decoder to obtain the first semantic signal;

[0039] Step S208: Use a multimodal semantic receiver to extract feature data from the first semantic signal, and determine the anomaly scores of multiple types of industrial data based on the feature data;

[0040] Step S210: If the anomaly score is greater than a preset threshold, anomalies are determined to exist in multiple types of industrial data.

[0041] In step S202 of the above anomaly detection method, receiving semantic transmission signals transmitted through multiple physical channels specifically includes the following steps: receiving multiple types of industrial data collected by multiple drones, wherein the multiple types of industrial data include image data, text data, and voice data; performing semantic compression on the semantic information of the multiple types of industrial data according to a first encoder to obtain compressed semantic information, wherein the first encoder is a semantic encoder corresponding to the industrial data collected by each drone, and the first learning parameter in the first encoder has a corresponding relationship with each drone; inputting the compressed semantic information into a second encoder for channel compression to obtain a semantic transmission signal, and transmitting the semantic transmission signal through multiple physical channels, wherein the second encoder is a channel encoder corresponding to the industrial data collected by each drone, and the second learning parameter in the second encoder has a corresponding relationship with each drone.

[0042] In this embodiment of the application, it is assumed that a drone swarm consisting of k drones needs to complete anomaly detection tasks, where the data collected by each drone k is represented by Di ( I represents image data, T represents text data, and A represents voice data. Specifically, each data source contains anomalous semantic information. First, the formal expression for extracting semantic information is as follows:

[0043] S k =φ(D k ;α k ),

[0044] Where S k It is semantic information (i.e., the compressed semantic information mentioned above), with a length of LS and φ(D). k ;α k ) is a function with a learnable parameter α k The modality D of the kth user (i.e., the first learning parameter mentioned above) k The semantic encoder (i.e., the first encoder mentioned above). Due to the limited communication resources and complex communication environment of wireless communication, the semantic information of the k-th user is compressed into:

[0045]

[0046] Where, x k It is a complex signal being transmitted (i.e., the semantically transmitted signal mentioned above), with a length of LC (LC <LS), It has a learnable parameter β k (i.e., the second learning parameter mentioned above) mode D k The k-th user joint source channel (JSC) encoder (i.e., the second encoder mentioned above). The neural JSC encoder in semantic communication compresses semantic information to reduce the number of transmitted symbols and improve robustness to channel variations.

[0047] In step S204 of the above anomaly detection method, the semantic transmission signal is compressed and decoded at least based on the channel state information of the physical channel to obtain a compressed semantic signal. Specifically, this includes the following steps: determining the received signal based at least on the semantic transmission signal and the channel matrix, wherein the channel matrix is ​​the channel matrix between the roadside unit and the UAV, and the roadside unit is a computing unit fixed at a specific location; determining the estimated value of the channel state information based on the channel state information and the error estimate of the channel state information; and determining the compressed semantic signal based on the received signal and the estimated value of the channel state information.

[0048] In step S206 of the above anomaly detection method, the compressed semantic signal is semantically decoded according to the first decoder to obtain the first semantic signal. Specifically, the steps include: determining the third learning parameter in the first decoder, wherein the first decoder is a decoder corresponding to the second encoder, the third learning parameter has a corresponding relationship with each UAV, and the first decoder is used to decompress semantic information; and the compressed semantic signal is semantically decoded according to the third learning parameter to obtain the first semantic signal.

[0049] In this embodiment of the application, when the UAV transmits a signal and passes through a Multiple-Input Multiple-Output (MIMO) physical channel, the received signal Y at the receiving end can be expressed as:

[0050] Y = HX + N

[0051] Where X T =[x1,x2,...,x k ] represents the transmission symbols (i.e., the semantic transmission signals mentioned above) from all k UAVs, H T =[h1,h2,...,h k [ ] is the channel matrix between the roadside unit and the UAV. For Rayleigh fading channels, the channel coefficients follow CN(0,1); for Rician fading channels, it follows CN(μ,σ) 2 )and Where r is the Rician coefficient. The elements of N are variables with zero mean and zero variance σ. n 2 For independent and identically distributed data, SNR is defined as follows:

[0052] The transmitted signal is recovered using a linear minimum mean square error (L-MMSE) detector and estimated channel state information (CSI):

[0053]

[0054] in, This is the estimated CSI, and ΔH is the error estimate. That is, the compressed semantic signal mentioned above.

[0055] The semantic information from the k-th user is recovered by the JSC decoder (i.e., the first decoder mentioned above) as follows:

[0056]

[0057] in It has modal D k and learning parameter γ k The JSC decoder for the k-th user (i.e., the third learning parameter mentioned above). This refers to the first semantic signal mentioned above. The JSC decoder aims to decompress semantic information while mitigating the effects of channel distortion and inter-user interference.

[0058] In step S208 of the above anomaly detection method, feature data in the first semantic signal is extracted using a multimodal semantic receiver. Specifically, this includes the following steps: determining the fourth learning parameter in the second decoder, wherein the second decoder is a decoder corresponding to the first encoder, and the fourth learning parameter has a corresponding relationship with each UAV; decoding the first semantic signal according to the fourth learning parameter to obtain the target semantic signal; and extracting feature data in the target semantic signal using a multimodal semantic receiver, wherein the feature data includes: first feature data corresponding to image data, second feature data corresponding to text data, and third feature data corresponding to speech data.

[0059] In the embodiments of this application, based on the independence of transmitting semantic information, there are single-modal semantic receivers and multi-modal semantic receivers.

[0060] (1) Single-modal semantic receiver: For single-modal semantic transmission, the semantic information of each user is used to independently perform different tasks. The recovered semantic information is used for the task of the k-th user:

[0061]

[0062] Where R k It is the result of semantic communication decoding. The modality D of the k-th user k The semantic decoder (i.e., the second decoder mentioned above) learns parameters ω. k (i.e., the fourth learning parameter mentioned above).

[0063] (2) Multimodal semantic receiver: With multimodal semantic information, the final task is to directly fuse semantic information from different users.

[0064]

[0065] (3) Training algorithm:

[0066] Algorithm 1: Training Algorithm for Semantic Encoders

[0067] Input: Training dataset Dk, mini-training batch B

[0068] Output: The trained semantic encoder φ(D) k ;α k )

[0069] Training process:

[0070] Step 1. Randomly extract mini-training batches of data from dataset Dk. Conduct training;

[0071] Step 2. Perform semantic encoding:

[0072] Step 3. Calculate the semantic encoding loss:

[0073] Step 4. Utilize the loss Training parameters α using gradient descent algorithm k,j ;

[0074] Step 5. Continue until the loss converges to obtain φ(D) k ;α k ).

[0075] Algorithm 2: JSC Encoder Training Algorithm

[0076] Input: Semantic information

[0077] Output: Trained

[0078] Training process:

[0079] Step 1. Input semantic information

[0080] Step 2. for j = n -> n + B do;

[0081] Step 3. JSC encoding:

[0082] Step 4. Transmit JSC-encoded information;

[0083] Step 5. The receiver receives JSC encoded information Y;

[0084] Step 6. The JSC decoder performs decoding.

[0085] Step 7. End for

[0086] Step 8. Calculate x k,j and The mean square error;

[0087] Step 9. Train the parameter β using the error and gradient descent algorithm described above. k,j and γ k,j ;

[0088] Step 10. Return

[0089] In step S208 of the above anomaly detection method, after extracting feature data from the first semantic signal using a multimodal semantic receiver, the method further includes the following steps: fusing the first feature data, the second feature data, and the third feature data to obtain fused feature data; performing feature reconstruction on the fused feature data to obtain target feature data; and determining the anomaly scores of multiple types of industrial data based on the target feature data.

[0090] In step S210 of the above anomaly detection method, the anomaly score of multiple types of industrial data is determined based on the feature data, which specifically includes the following steps: obtaining the true feature value and the corresponding output feature value of each type of industrial data in the target feature data, wherein the output feature value is the feature value of the feature data extracted by the multimodal semantic receiver; determining the anomaly score of each type of industrial data based on the true feature value, the output feature value and the weight value of each type of industrial data, wherein the weight value is determined by the feature dimension of each type of industrial data in the target feature data.

[0091] In this embodiment of the application, the multimodal data D collected by the UAV k The semantic information has been transmitted to the receiver for anomaly detection via an efficient semantic communication system. Further feature extraction is required in this anomaly detection module. Specifically, this patent uses the currently popular Transformer as the encoder for feature extraction, denoted as G. T The formal process of feature extraction can be represented as follows:

[0092] (1) Transformer performs low-dimensional embedding encoding:

[0093]

[0094] (2) Multimodal feature fusion (i.e., the fused feature data obtained above):

[0095]

[0096] in, The symbol represents the operation of concatting features according to a specified dimension.

[0097] (3) Feature reconstruction (i.e., G) T D Decoding is then performed, which is to obtain the target feature data as described above:

[0098]

[0099] (4) Loss function (training):

[0100]

[0101] Anomaly detection can be performed based on the obtained reconstructed features. In this embodiment, anomaly scores are used for anomaly detection, and their formal definition is as follows:

[0102]

[0103] in:

[0104]

[0105]

[0106]

[0107] Where, x i x represents the true feature value. i ' indicates the output feature value, and size() is the modal feature dimension. The threshold for anomaly scores can be determined based on the training data, thereby detecting anomalies.

[0108] The anomaly detection method provided in this application has the following advantages: 1. It introduces a task-driven multi-user semantic communication system to empower the UAV swarm-based industrial anomaly detection system, reducing the overall system communication cost, improving semantic-level communication efficiency, and significantly increasing the UAV swarm's endurance. 2. This application designs a feature reconstruction-based multimodal learning algorithm to further improve the performance of anomaly detection, further expand the UAV's cruising range, and fully utilize the diverse data collected.

[0109] Figure 3 This is a structural diagram of an anomaly detection device according to an embodiment of this application, such as... Figure 3 As shown, the device includes:

[0110] The receiving module 302 is used to receive semantic transmission signals transmitted through multiple physical channels. The semantic transmission signals are obtained by sequentially compressing and encoding multiple types of industrial data collected by multiple UAVs using a first encoder and a second encoder. The first encoder is used to perform semantic compression on the semantic information of the multiple types of industrial data, and the second encoder is used to perform channel compression on the compressed semantic information.

[0111] The first decoding module 304 is used to perform channel compression decoding on the semantic transmission signal based at least on the channel state information of the physical channel to obtain a compressed semantic signal;

[0112] The second decoding module 306 is used to perform semantic decoding on the compressed semantic signal according to the first decoder to obtain the first semantic signal;

[0113] The first determining module 308 is used to extract feature data from the first semantic signal using a multimodal semantic receiver, and to determine the anomaly score of multiple types of industrial data based on the feature data.

[0114] The second determining module 310 is used to determine that there are anomalies in multiple types of industrial data when the anomaly score is greater than a preset threshold.

[0115] In the receiving module of the aforementioned anomaly detection device, the semantic transmission signal transmitted through multiple physical channels is received, specifically including the following process: receiving multiple types of industrial data collected by multiple drones, wherein the multiple types of industrial data include image data, text data, and voice data; performing semantic compression on the semantic information of the multiple types of industrial data according to a first encoder to obtain compressed semantic information, wherein the first encoder is a semantic encoder corresponding to the industrial data collected by each drone, and the first learning parameter in the first encoder has a corresponding relationship with each drone; inputting the compressed semantic information into a second encoder for channel compression to obtain a semantic transmission signal, and transmitting the semantic transmission signal through multiple physical channels, wherein the second encoder is a channel encoder corresponding to the industrial data collected by each drone, and the second learning parameter in the second encoder has a corresponding relationship with each drone.

[0116] In the first decoding module of the above-mentioned anomaly detection device, the semantic transmission signal is compressed and decoded at least based on the channel state information of the physical channel to obtain a compressed semantic signal. Specifically, the process includes the following steps: determining the received signal based at least on the semantic transmission signal and the channel matrix, wherein the channel matrix is ​​the channel matrix between the roadside unit and the UAV, and the roadside unit is a computing unit fixed at a specific location; determining the estimated value of the channel state information based on the channel state information and the error estimate of the channel state information; and determining the compressed semantic signal based on the received signal and the estimated value of the channel state information.

[0117] In the second decoding module of the above-mentioned anomaly detection device, the compressed semantic signal is semantically decoded according to the first decoder to obtain the first semantic signal. Specifically, the process includes the following steps: determining the third learning parameter in the first decoder, wherein the first decoder is a decoder corresponding to the second encoder, the third learning parameter has a corresponding relationship with each UAV, and the first decoder is used to decompress semantic information; and performing semantic decoding on the compressed semantic signal according to the third learning parameter to obtain the first semantic signal.

[0118] In the first determining module of the above-mentioned anomaly detection device, feature data in the first semantic signal is extracted using a multimodal semantic receiver. Specifically, the process includes: determining the fourth learning parameter in the second decoder, wherein the second decoder is a decoder corresponding to the first encoder, and the fourth learning parameter has a corresponding relationship with each UAV; decoding the first semantic signal according to the fourth learning parameter to obtain the target semantic signal; and extracting feature data in the target semantic signal using a multimodal semantic receiver, wherein the feature data includes: first feature data corresponding to image data, second feature data corresponding to text data, and third feature data corresponding to speech data.

[0119] In the first determining module of the above-mentioned anomaly detection device, after extracting feature data from the first semantic signal using a multimodal semantic receiver, the first determining module is also used to fuse the first feature data, the second feature data, and the third feature data to obtain fused feature data; perform feature reconstruction on the fused feature data to obtain target feature data, and determine the anomaly scores of multiple types of industrial data based on the target feature data.

[0120] In the first determining module of the above-mentioned anomaly detection device, the anomaly score of multiple types of industrial data is determined based on feature data, specifically including the following process: obtaining the true feature value and the corresponding output feature value of each type of industrial data in the target feature data, wherein the output feature value is the feature value of the feature data extracted by the multimodal semantic receiver; determining the anomaly score of each type of industrial data based on the true feature value, the output feature value and the weight value of each type of industrial data, wherein the weight value is determined by the feature dimension of each type of industrial data in the target feature data.

[0121] It should be noted that, Figure 3 The anomaly detection device shown is used to perform Figure 2 The anomaly detection method shown above is also applicable to this anomaly detection device, and will not be repeated here.

[0122] This application embodiment also provides a non-volatile storage medium, which includes a stored computer program. The device containing the non-volatile storage medium executes the following anomaly detection method by running the computer program: receiving semantic transmission signals transmitted through multiple physical channels, wherein the semantic transmission signals are obtained by sequentially compressing and encoding multiple types of industrial data collected by multiple drones using a first encoder and a second encoder. The first encoder is used to perform semantic compression on the semantic information of the multiple types of industrial data, and the second encoder is used to perform channel compression on the compressed semantic information; performing channel compression decoding on the semantic transmission signals based at least on the channel state information of the physical channels to obtain a compressed semantic signal; performing semantic decoding on the compressed semantic signal based on a first decoder to obtain a first semantic signal; extracting feature data from the first semantic signal using a multimodal semantic receiver, determining anomaly scores for the multiple types of industrial data based on the feature data; and determining that anomalies exist in the multiple types of industrial data if the anomaly score is greater than a preset threshold.

[0123] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0124] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0126] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0127] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0128] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0129] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An anomaly detection method, characterized in that, include: The system receives semantic transmission signals transmitted through multiple physical channels. The semantic transmission signals are obtained by sequentially compressing and encoding multiple types of industrial data collected by multiple drones using a first encoder and a second encoder. The first encoder is used to perform semantic compression on the semantic information of the multiple types of industrial data, and the second encoder is used to perform channel compression on the compressed semantic information. The semantic transmission signal is compressed and decoded at least based on the channel state information of the physical channel to obtain a compressed semantic signal; The compressed semantic signal is semantically decoded according to the first decoder to obtain the first semantic signal; The feature data in the first semantic signal is extracted using a multimodal semantic receiver, and the anomaly score of the multi-type industrial data is determined based on the feature data. If the anomaly score is greater than a preset threshold, it is determined that there is an anomaly in the multiple types of industrial data; Receiving semantic transmission signals from multiple physical channels includes: receiving various types of industrial data collected by multiple drones, wherein the various types of industrial data include image data, text data, and voice data; performing semantic compression on the semantic information of the various types of industrial data according to a first encoder to obtain compressed semantic information, wherein the first encoder is a semantic encoder corresponding to the industrial data collected by each drone, and the first learning parameter in the first encoder has a corresponding relationship with each drone; inputting the compressed semantic information into a second encoder for channel compression to obtain the semantic transmission signal, and transmitting the semantic transmission signal through the multiple physical channels, wherein the second encoder is a channel encoder corresponding to the industrial data collected by each drone, and the second learning parameter in the second encoder has a corresponding relationship with each drone.

2. The method according to claim 1, characterized in that, At least based on the channel state information of the physical channel, the semantic transmission signal is subjected to channel compression decoding to obtain a compressed semantic signal, including: The received signal is determined based at least on the semantic transmission signal and the channel matrix, wherein the channel matrix is ​​the channel matrix between the roadside unit and the UAV, and the roadside unit is a computing unit fixed at a preset position; Based on the channel state information and the error estimate of the channel state information, the estimated value of the channel state information is determined; The compressed semantic signal is determined based on the received signal and the estimated value of the channel state information.

3. The method according to claim 1, characterized in that, The compressed semantic signal is semantically decoded according to the first decoder to obtain the first semantic signal, including: A third learning parameter is determined in the first decoder, wherein the first decoder is a decoder corresponding to the second encoder, the third learning parameter has a corresponding relationship with each UAV, and the first decoder is used to decompress semantic information; Based on the third learning parameter, the compressed semantic signal is semantically decoded to obtain the first semantic signal.

4. The method according to claim 1, characterized in that, The feature data in the first semantic signal is extracted using a multimodal semantic receiver, including: A fourth learning parameter is determined in the second decoder, wherein the second decoder is a decoder corresponding to the first encoder, and the fourth learning parameter has a corresponding relationship with each UAV; Based on the fourth learning parameter, the first semantic signal is decoded to obtain the target semantic signal; The multimodal semantic receiver is used to extract feature data from the target semantic signal, wherein the feature data includes: first feature data corresponding to the image data, second feature data corresponding to the text data, and third feature data corresponding to the speech data.

5. The method according to claim 4, characterized in that, After extracting feature data from the first semantic signal using a multimodal semantic receiver, the method further includes: The first feature data, the second feature data, and the third feature data are fused to obtain fused feature data; The fused feature data is reconstructed to obtain target feature data, and the anomaly scores of the multiple types of industrial data are determined based on the target feature data.

6. The method according to claim 5, characterized in that, Determining the anomaly scores of the multiple types of industrial data based on the aforementioned feature data includes: Obtain the true feature value and corresponding output feature value of each type of industrial data in the target feature data, wherein the output feature value is the feature value of the feature data extracted by the multimodal semantic receiver; Based on the true feature value, the output feature value, and the weight value of each type of industrial data, an anomaly score for each type of industrial data is determined, wherein the weight value is determined by the feature dimension of each type of industrial data in the target feature data.

7. An anomaly detection device, characterized in that, include: A receiving module is configured to receive semantic transmission signals transmitted through multiple physical channels. The semantic transmission signals are obtained by sequentially compressing and encoding various types of industrial data collected by multiple drones using a first encoder and a second encoder. The first encoder performs semantic compression on the semantic information of the various types of industrial data, and the second encoder performs channel compression on the compressed semantic information. Receiving the semantic transmission signals through multiple physical channels includes: receiving various types of industrial data collected by the multiple drones, including image data, text data, and voice data; performing semantic compression on the semantic information of the various types of industrial data according to the first encoder to obtain compressed semantic information, wherein the first encoder is a semantic encoder corresponding to the industrial data collected by each drone, and a first learning parameter in the first encoder corresponds to each drone; inputting the compressed semantic information into the second encoder for channel compression to obtain the semantic transmission signals, and transmitting the semantic transmission signals through the multiple physical channels, wherein the second encoder is a channel encoder corresponding to the industrial data collected by each drone, and a second learning parameter in the second encoder corresponds to each drone. The first decoding module is used to perform channel compression decoding on the semantic transmission signal based at least on the channel state information of the physical channel to obtain a compressed semantic signal; The second decoding module is used to perform semantic decoding on the compressed semantic signal according to the first decoder to obtain the first semantic signal; The first determining module is used to extract feature data from the first semantic signal using a multimodal semantic receiver, and determine the anomaly score of the multi-type industrial data based on the feature data. The second determining module is used to determine that there is an anomaly in the multi-type industrial data when the anomaly score is greater than a preset threshold.

8. An electronic device, characterized in that, include: Memory, used to store program instructions; A processor, connected to the memory, is configured to execute program instructions to perform the following functions: receiving semantic transmission signals transmitted through multiple physical channels, wherein the semantic transmission signals are obtained by sequentially compressing and encoding multiple types of industrial data collected by multiple drones using a first encoder and a second encoder, wherein the first encoder is used to perform semantic compression on the semantic information of the multiple types of industrial data, and the second encoder is used to perform channel compression on the compressed semantic information; performing channel compression decoding on the semantic transmission signals based at least on the channel state information of the physical channels to obtain a compressed semantic signal; performing semantic decoding on the compressed semantic signal based on a first decoder to obtain a first semantic signal; extracting feature data from the first semantic signal using a multimodal semantic receiver, determining anomaly scores for the multiple types of industrial data based on the feature data; and determining the anomaly score if the anomaly score is greater than a preset threshold. Anomalies exist in multiple types of industrial data; receiving semantic transmission signals transmitted through multiple physical channels includes: receiving multiple types of industrial data collected by multiple drones, wherein the multiple types of industrial data include image data, text data, and voice data; semantically compressing the semantic information of the multiple types of industrial data according to the first encoder to obtain compressed semantic information, wherein the first encoder is a semantic encoder corresponding to the industrial data collected by each drone, and the first learning parameter in the first encoder has a corresponding relationship with each drone; inputting the compressed semantic information into the second encoder for channel compression to obtain the semantic transmission signal, and transmitting the semantic transmission signal through the multiple physical channels, wherein the second encoder is a channel encoder corresponding to the industrial data collected by each drone, and the second learning parameter in the second encoder has a corresponding relationship with each drone.

9. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored computer program, wherein the device containing the non-volatile storage medium executes the anomaly detection method according to any one of claims 1 to 6 by running the computer program.

Images