Transform-based large-scale Internet of Things equipment activity detection method and system

Abstract

The invention belongs to the technical field of ground communication, and discloses a Transform-based large-scale Internet of Things equipment activity detection method, which can dynamically predict the equipment activation probability by introducing an activity estimation module and a double-sparse Transform module, and optimizes a detection model by utilizing sparse processing, thereby improving the detection precision, reducing the calculation complexity and improving the detection efficiency. And the generalization ability of the model is enhanced. The device activation probability is dynamically predicted, and the activation probability does not need to be assumed in advance; the DST module reduces calculation redundancy through sparse processing, dynamically adjusts the attention weight by using equipment activation probability information, and enhances the attention of the model on important features; a deep cooperative detection framework is provided, an activeness estimation module and a double-sparse Transform module are organically combined, and efficient mapping from a received signal to an equipment activation state is realized.

Description

Technical Field

[0001] The present invention belongs to the field of terrestrial communication technology, and in particular relates to a Transformer-based large-scale Internet of Things device activity detection method and system. Background Art

[0002] In recent years, with the rapid development of the Internet of Things (IoT) technology, the massive number of smart devices has placed unprecedented connectivity demands on wireless communication networks. In cellular communication systems, 4G / 5G networks, through high-density base station deployment and technologies such as narrowband IoT (NB-IoT), provide a wide-area connectivity foundation for scenarios such as smart cities, the Industrial Internet of Things, and the Internet of Vehicles. However, faced with the explosive growth of device scale and diverse service demands (such as massive machine-type communications (mMTC) and ultra-reliable low-latency communications (URLLC), traditional random access schemes struggle to meet the requirements for efficient and low-latency communication. To address this challenge, the Grant-Free Random Access (GF-RA) scheme has been proposed, allowing devices to send data directly without prior authorization. However, the core challenge of the GF-RA scheme lies in how to efficiently detect active devices, namely the device activity detection (DAD) problem.

[0003] To achieve device activity detection, existing technologies mainly rely on compressed sensing (CS) algorithms and deep learning (DL) technologies. For example, compressed sensing algorithms are used to solve the problem of sparse signal recovery, or deep learning methods such as convolutional neural networks (CNN) and long short-term memory networks (LSTM) are used to estimate device activity status. However, these methods have obvious limitations: on the one hand, the computational complexity of compressed sensing algorithms increases significantly when a large number of devices are connected, and the error propagation during the iteration process will reduce the detection accuracy; on the other hand, most deep learning methods assume that the device activation probability is known and fixed, which is difficult to meet in practical applications, especially in scenarios with diverse device access requirements. In addition, the detection performance of existing methods drops sharply when dealing with complex channel conditions (such as high noise environments or low signal-to-noise ratio conditions), making it difficult to adapt to the complex environment of terrestrial communication networks.

[0004] Therefore, existing technologies have shortcomings in terms of dynamic device access, detection accuracy under complex channel conditions, computational complexity of large-scale device access, and model generalization capabilities, and need to be improved to achieve accurate device activity detection in large-scale IoT scenarios. Summary of the Invention

[0005] In response to the problems existing in the prior art, the present invention provides a large-scale IoT device activity detection method based on Transformer.

[0006] The present invention is implemented as follows: a Transformer-based large-scale IoT device activity detection method includes:

[0007] Step 1: For each available device, construct a received signal dataset under different activation probabilities;

[0008] Step 2: Based on the impact of device activation probability on received signals, an activity estimation module is designed to predict device activation probability from received signals.

[0009] Step 3: For each device, combining the sparsity and regularity of the received signal matrix, using the output of the activity estimation module as prior information, a double-sparse Transformer module is proposed. Feature extraction and optimization are performed on the entire dataset to obtain a device activity detection model based on the double-sparse Transformer.

[0010] Step 4: Use the dual-sparse Transformer-based device activity detection model to detect device activity.

[0011] Furthermore, the received signal dataset is generated by simulating channel conditions and device activation probability, wherein the channel condition is a Gaussian white noise channel (AWGN), and the constructed received signal dataset is:

[0012]

[0013] Furthermore, the activity estimation module uses a multi-layer perceptron structure and is trained by minimizing the mean square error (MSE) loss function to improve the prediction accuracy of the device activation probability.

[0014] Furthermore, the dual-sparse Transformer module includes an initial embedding layer, an attention mechanism layer, and an output layer. The initial embedding layer is used to reshape the received signal into a format suitable for neural network processing; the attention mechanism layer uses device activation probability information to dynamically adjust the attention weight to enhance the model's focus on important features; the output layer outputs the device activation state through a Sigmoid function; and the constructed received signal dataset is then used to train a dual-sparse Transformer-based device activity detection model.

[0015] Furthermore, the dual sparse Transformer architecture uses activation probability information to dynamically adjust attention weights, enhancing the model's focus on important features and improving detection performance.

[0016] Furthermore, the device activity detection model can achieve a stable high detection success rate under different activation probabilities and signal-to-noise ratio (SNR) conditions, and the detection success rate is higher than that of the existing technology under different SNR conditions.

[0017] Another object of the present invention is to provide a Transformer-based large-scale IoT device activity detection system comprising:

[0018] A data set construction module is used to construct a received signal data set under different activation probabilities for each available device;

[0019] The prediction module is used to design an activity estimation module based on the impact of device activation probability on received signals, and is used to predict the device activation probability from the received signals;

[0020] The feature extraction module is used to analyze the sparsity and regularity of the received signal matrix for each device. It uses the output of the activity estimation module as prior information and proposes a dual-sparse Transformer module to extract and optimize features from the entire dataset, resulting in a dual-sparse Transformer-based device activity detection model.

[0021] The detection module is used to detect device activity using a dual-sparse Transformer-based device activity detection model.

[0022] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the Transformer-based large-scale Internet of Things device activity detection method.

[0023] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the Transformer-based large-scale Internet of Things device activity detection method.

[0024] Another object of the present invention is to provide an information data processing terminal, which is used to implement the Transformer-based large-scale Internet of Things device activity detection system.

[0025] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0026] First, the purpose of the present invention is to provide a large-scale IoT device activity detection method based on the Transformer architecture, which is used to solve the problem of device activity detection when large-scale devices are connected to the IoT. In the prior art, device activity detection methods usually rely on fixed activation probability assumptions, and there are performance bottlenecks when dealing with complex channel conditions and dynamic device access. By introducing an activity estimation module and a dual-sparse Transformer module, the present invention can dynamically predict the device activation probability and optimize the detection model using sparsification processing, thereby improving detection accuracy, reducing computational complexity, and enhancing the generalization ability of the model.

[0027] (1) Dynamically predict device activation probability without pre-assuming activation probability;

[0028] (2) The DST module reduces computational redundancy through sparse processing and dynamically adjusts attention weights using device activation probability information to enhance the model's focus on important features;

[0029] (3) A deep collaborative detection framework is proposed, which organically combines the activity estimation module and the dual sparse Transformer module to achieve efficient mapping from the received signal to the device activation state.

[0030] Second, this invention achieves efficient and real-time device activity detection in large-scale IoT environments through dynamic prediction of device activation probabilities and the use of dual-sparse Transformer modules. This solution has significant potential for improving communication efficiency, enhancing user experience, expanding market applications, and reducing device costs. It can provide operators and device manufacturers with higher spectrum utilization and lower operating costs, meeting the needs of multiple scenarios such as emergency communications, environmental monitoring, and logistics tracking, thus possessing broad application prospects in the IoT industry.

[0031] This paper proposes a series of innovative solutions in the field of large-scale Internet of Things. In particular, it uses a sparse estimation module to dynamically predict device activation probabilities, breaking through the previous over-reliance on fixed activation probabilities. The design of a dual-sparse Transformer module, combined with sparsification processing and an attention mechanism, significantly improves detection accuracy and model generalization. By generating a dataset for training under complex channel conditions, this paper maintains stable performance in complex environments and also achieves significant progress in real-time performance.

[0032] This paper presents a systematic solution to address four major technical bottlenecks: the massive number of IoT devices and their dynamically changing access requirements, high local environmental noise, and stringent real-time requirements. By dynamically predicting device activation probabilities, system adaptability is significantly improved. The dual-sparse Transformer module and sparsification processing enhance detection accuracy under complex channel conditions. By optimizing algorithm design, the computational complexity of large-scale device access scenarios is reduced, meeting the requirements of high efficiency and real-time performance.

[0033] This invention utilizes a sparse estimation module, overcoming the traditional method's fixed activation probability assumption and enabling dynamic changes in device activation probability based on the actual environment and service needs. Addressing the shortcomings of existing methods, which typically rely on specific channel models, this invention simulates real data under different channel conditions and trains the model. This allows the detection model to maintain stable performance in scenarios with massive numbers of devices and complex channels, significantly improving the flexibility and applicability of large-scale IoT systems.

[0034] To address the high computational complexity and low real-time performance associated with large-scale device access, this paper utilizes dual-sparse Transformers and sparsification techniques, significantly reducing the computational overhead of the inference process and enabling highly efficient device activity detection in multiple scenarios. Furthermore, this paper fully considers multiple activation probabilities during model design and dataset construction. By continuously optimizing algorithms and training data, it significantly improves the model's generalization capabilities, overcoming the limitations of existing technologies, which often suffer from unstable performance under different environments.

[0035] By enabling efficient device activity detection in the massive Internet of Things (IoT), this invention provides a feasible and forward-looking technical path for the industry. Its innovations in dynamic prediction of activation probability, adaptation to complex channel conditions, and real-time detection not only directly improve network performance and resource utilization for operators, but also open new application directions for deep learning and distributed optimization technologies in related fields. By systematically addressing key technical difficulties, this invention has significant demonstration significance and promotional value in the large-scale application of the IoT. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of a Transformer-based large-scale IoT device activity detection method provided by an embodiment of the present invention.

[0037] Figure 2 This is a structural block diagram of a large-scale IoT device activity detection system based on Transformer provided by an embodiment of the present invention.

[0038] Figure 3This is a diagram of a Transformer-based device activity detection solution provided by an embodiment of the present invention.

[0039] Figure 4 This is a flow chart of online device activity detection provided by an embodiment of the present invention.

[0040] Figure 5 3 is a mean square error (MSE) diagram of predictions with different sparsities provided by an embodiment of the present invention.

[0041] Figure 6 is a successful detection probability diagram of different signal-to-noise ratio data set models provided by an embodiment of the present invention.

[0042] Figure 7 1 is a graph of successful detection probabilities of different methods under the conditions of K=0.2 and SNR=-20dB provided by an embodiment of the present invention.

[0043] Figure 8 1 is a diagram of successful detection probabilities of different methods under the condition of SNR=-20dB provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0045] like Figure 1 As shown, the embodiment of the present invention provides a Transformer-based large-scale IoT device activity detection method comprising the following steps:

[0046] S101, for each available device, construct a received signal dataset under different activation probabilities;

[0047] S102, based on the impact of device activation probability on received signals, design an activity estimation module to predict device activation probability from received signals;

[0048] S103: For each device, a dual-sparse Transformer module is proposed, combining the sparsity and regularity of the received signal matrix and using the output of the activity estimation module as prior information. Feature extraction and optimization are performed on the entire dataset to obtain a dual-sparse Transformer-based device activity detection model.

[0049] S104: Detect device activity using a dual-sparse Transformer-based device activity detection model.

[0050] In terrestrial communication systems, the active state of a device affects the statistical characteristics of its received signal. Therefore, the first step (S101) is to collect received signal data for each available device at different activation probabilities and construct a multidimensional signal dataset. For each device, the system records its historical communication data, including time series signal strength, noise interference level, and possible inter-signal interference patterns. During the dataset construction process, a signal matrix is used to represent the spatiotemporal characteristics of the device, and standardized preprocessing is performed to reduce the impact of signal non-stationarity, making it more suitable for subsequent modeling.

[0051] In S102, an activity estimation module is designed based on the device's historical activation probability and the characteristics of its received signal. The core of this module is a multilayer perceptron, which accurately fits the underlying relationship between received signals and device activation probabilities, thereby accurately predicting device activation probabilities. The activity estimation module takes the constructed dataset as input and outputs the predicted device activation probability, providing prior information for the subsequent Transformer model, enabling it to learn the inherent relationship between device activation probability and signals.

[0052] In S103, a dual-sparse Transformer module is proposed to model the complex relationship between device activity and received signals. First, based on the sparsity of the received signal matrix, the attention mechanism is applied to screen out key features and reduce the impact of redundant information. Secondly, considering the regularity of device activity data, global and local attention mechanisms are introduced to perform multi-level modeling of the data. The Transformer module learns the importance of signal features through the multi-head self-attention mechanism (MHSA), and uses the prior information provided by the device activity estimation module to optimize parameters. Ultimately, the module can effectively extract device activity features in a sparse data environment and generate a stable detection model.

[0053] In S104, the trained dual-sparse Transformer-based device activity detection model is used to perform real-time detection of devices in the large-scale IoT. This model analyzes received signal data and combines historical activation probability estimates to classify and predict the current activity of each device. The detection results can be used to optimize communication resource allocation, improve communication efficiency, and monitor device status. If a device is detected to be in a low activity state for a long period of time, the system can further trigger intelligent management strategies, such as adjusting signal coverage, optimizing power allocation, or sending instructions to wake up or perform maintenance on the device.

[0054] The received signal dataset provided in the embodiment of the present invention is generated by simulating channel conditions and device activation probabilities. The channel condition is a Gaussian white noise channel (AWGN). The constructed received signal dataset is:

[0055]

[0056] The activity estimation module provided in the embodiment of the present invention uses a multi-layer perceptron structure and is trained by minimizing the mean square error (MSE) loss function to improve the prediction accuracy of the device activation probability.

[0057] The dual-sparse Transformer module provided in an embodiment of the present invention includes an initial embedding layer, an attention mechanism layer and an output layer, wherein the initial embedding layer is used to reshape the received signal into a format suitable for neural network processing; the attention mechanism layer uses device activation probability information to dynamically adjust the attention weight to enhance the model's attention to important features; the output layer outputs the activation state of the device through a Sigmoid function; and then the constructed received signal dataset is used to train a device activity detection model based on the dual-sparse Transformer.

[0058] The dual sparse Transformer architecture provided by the embodiment of the present invention uses activation probability information to dynamically adjust attention weights, enhance the model's attention to important features, and improve detection performance.

[0059] The device activity detection model provided by the embodiment of the present invention can achieve a stable high detection success rate under different activation probabilities and signal-to-noise ratio (SNR) conditions. The detection success rate is higher than that of the prior art under different SNR conditions.

[0060] like Figure 2 As shown, an embodiment of the present invention provides a Transformer-based large-scale IoT device activity detection system comprising:

[0061] A data set construction module is used to construct a received signal data set under different activation probabilities for each available device;

[0062] The prediction module is used to design an activity estimation module based on the impact of device activation probability on received signals, and is used to predict the device activation probability from the received signals;

[0063] The feature extraction module is used to analyze the sparsity and regularity of the received signal matrix for each device. It uses the output of the activity estimation module as prior information and proposes a dual-sparse Transformer module to extract and optimize features from the entire dataset, resulting in a dual-sparse Transformer-based device activity detection model.

[0064] The detection module is used to detect the device activity by using the device activity detection model based on the dual-sparse Transformer.

[0065] Another object of the present invention is to provide a computer device, which includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor is caused to execute the steps of the method for detecting the activity of large-scale Internet of Things devices based on the Transformer.

[0066] Another object of the present invention is to provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is caused to execute the steps of the method for detecting the activity of large-scale Internet of Things devices based on the Transformer.

[0067] Another object of the present invention is to provide an information data processing terminal, which is used to implement the system for detecting the activity of large-scale Internet of Things devices based on the Transformer.

[0068] Specific implementation of the present invention:

[0069] Figure 3 It is a diagram of the device activity detection scheme based on the Transformer provided by an embodiment of the present invention.

[0070] Figure 4 Flow chart of online device activity detection.

[0071] The first step: Construct a received signal data set

[0072] Consider the GF-RA system in the cellular Internet of Things, where a single-antenna base station serves N single-antenna devices. Within the coverage of a single base station, the probability that K devices are activated is ε, where ε = K / N and K < N. In a single GF-RA time slot, a ZC sequence with good correlation characteristics is selected as the unique preamble sequence for each active device. Specifically as follows:

[0073]

[0074] where L is the sequence length, and r ∈ {1, L, N , , n ,

[0073] , n ,

[0072] ,

[0076] , ,

[0074] , n , ZC , , , , ,

[0075] } is the root index. The channel between the base station and the nth device can be defined as:

[0075]

[0076] where λ n and g n respectively represent the large-scale coefficient and the small-scale coefficient. Use a nTo indicate whether the nth IoT device is active, it is defined as follows:

[0077]

[0078] Among them, when device n is in active state a n =1, otherwise a n = 0. The nth device uses power p n Send preamble sequence s r,n , at this time, the received signal at the base station can be modeled as

[0079]

[0080] in, is the perception matrix, is the transmit power matrix, is the leading matrix. The state matrix is the additive white Gaussian noise at the base station.

[0081] At this time, according to the generation formula of the received signal, the activation probability range and signal-to-noise ratio conditions of the device are set to obtain the corresponding received signal to generate G non-repeated data samples, and the data set is represented by D.

[0082]

[0083] It should be noted that, represents the recombination of the real and imaginary parts of the received signal, that is, Moreover, when training different modules, the corresponding training labels must also change accordingly.

[0084] Step 2: Build an activity estimation module

[0085] The main task of the activity estimation module (SE module) is to find the optimal mapping function between the received signal and the output target under different device activation probabilities. Specifically, the SE module mainly uses a multi-layer perceptron to accurately fit the optimal architecture of the mapping function. The output of the SE module can be expressed as:

[0086]

[0087] in, is the predicted device activation probability, W SE and b SE are the weights and biases of the MLP.

[0088] In addition, in order to minimize the error, the mean squared error loss function is used as the regression layer of the SE module, which is defined as follows:

[0089]

[0090] Usually, the device activation probability ε in the received signal is an unknown parameter. The SE module can accurately predict the device activation probability information by learning the distribution characteristics of the received signal under different device activation probabilities, providing important prior parameters for subsequent modules. To adjust the internal parameters and enhance the generalization of the framework.

[0091] Step 3: Design and Optimization of Double Sparse Transformer Module

[0092] First, the received signals in the dataset Reshape it to make it more suitable for neural network processing, that is, Where W in and b in are the weights and biases of the initial embedding layer.

[0093] Then the output of the initial embedding layer Splice to Then the dual-headed attention mechanism is used to extract the relevance of the input data, i.e. in is the weight matrix of the attention mechanism.

[0094] It is worth noting that the prior information of activity obtained by the activity estimation module First, the first step of sparsification is performed on K and V: the K and V matrices are projected into a more concentrated group matrix through group convolution while retaining the activation information.

[0095]

[0096] Next, a dynamic mask matrix H is designed to adjust the attention score, which is the second step of sparsification.

[0097]

[0098] Among them, e represents the element-by-element product operation, f softmax Represents the softmax activation function.

[0099] Finally, the sigmoid activation function is used to output the activation state of the device.

[0100]

[0101] At this point, the Transformer-based device activity detection model consisting of the dual-sparse Transformer module and the activity estimation module is formed.

[0102] Step 4: Detect active devices based on the Transformer-based device activity detection model.

[0103] In a cellular IoT system, signals received at the base station are first fed into the activity estimation module. This module analyzes the characteristics of the received signal and dynamically predicts the device activation probability, providing critical prior information for subsequent processing. The predicted device activation probability is then combined with the original received signal and fed into a dual-sparse Transformer module. This module leverages its unique sparsification and attention mechanisms to efficiently extract signal features and accurately identify active devices. This entire process not only improves detection efficiency but also significantly enhances the model's adaptability and robustness in complex channel environments, providing reliable technical support for large-scale device access in the cellular IoT.

[0104] Figure 5 The proposed framework demonstrates the MSE of regression prediction for different unknown activity information. It can be seen that the proposed framework can ensure that the predicted activity information has negligible error under different noise conditions and accurately provides prior parameters.

[0105] Figure 6 The successful detection probabilities of the trained models under different signal-to-noise ratios are shown. It can be seen that the network trained with the dataset at -20dB SNR significantly outperforms the other two training scenarios. This is because the model trained in a high-noise environment is able to learn more generalizable features, leading to better performance in high-noise testing. Meanwhile, models trained in a low-noise environment may overfit and struggle to adapt to high-noise data.

[0106] Figure 7 The proposed model is described in terms of the successful detection probability under different signal-to-noise ratio conditions, when the user device activation probability e = 0.2 and the signal-to-noise ratio SNR = -20dB. It can be seen that the results of the proposed model are always better than those of the other methods. This result is due to the coordinated interaction between the internal modules of the proposed framework, which enables the dual sparse Transformer performing detection to fully utilize the prior activity information. In addition, the dynamic perception operation further reduces the risk of network overfitting.

[0107] Figure 8 The proposed model demonstrates the successful detection probability under different device activation probabilities. It can be seen that the proposed model performs significantly better than the other three methods for both low and high device activation probabilities. This result demonstrates that the proposed framework can fully leverage the estimated precise prior activity information, significantly improving the model's generalization capabilities.

[0108] An embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the Transformer-based large-scale Internet of Things device activity detection algorithm.

[0109] An embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the Transformer-based large-scale Internet of Things device activity detection algorithm.

[0110] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art will appreciate that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0111] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A large-scale IoT device activity detection method based on Transformer, characterized in that: The following steps are involved: Step 1: For each available device, construct a received signal dataset under different activation probabilities; Step 2: Based on the impact of device activation probability on received signals, an activity estimation module is designed to predict device activation probability from received signals. Step 3: For each device, combining the sparsity and regularity of the received signal matrix, using the output of the activity estimation module as prior information, a double-sparse Transformer module is proposed. Feature extraction and optimization are performed on the entire dataset to obtain a device activity detection model based on the double-sparse Transformer. Step 4: Use the dual-sparse Transformer-based device activity detection model to detect device activity.

2. The Transformer-based large-scale IoT device activity detection method according to claim 1, characterized in that: The received signal dataset is generated by simulating channel conditions and device activation probabilities. The channel condition is a Gaussian white noise channel. The constructed received signal dataset is:

3. The Transformer-based large-scale IoT device activity detection method according to claim 1, characterized in that: The activity estimation module uses a multi-layer perceptron structure and is trained by minimizing the mean square error loss function.

4. The Transformer-based large-scale IoT device activity detection method according to claim 1, characterized in that: The dual sparse Transformer module includes an initial embedding layer, an attention mechanism layer, and an output layer, wherein the initial embedding layer is used to reshape the received signal into a format suitable for neural network processing; The attention mechanism layer uses device activation probability information to dynamically adjust attention weights to enhance the model's focus on important features. The output layer outputs the device's activation status through a Sigmoid function. Next, the constructed received signal dataset is used to train a dual-sparse Transformer-based device activity detection model.

5. The Transformer-based large-scale IoT device activity detection method according to claim 1, characterized in that: The dual sparse Transformer architecture uses activation probability information to dynamically adjust attention weights.

6. The Transformer-based large-scale IoT device activity detection method according to claim 1, characterized in that: The device activity detection model can achieve a stable high detection success rate under different activation probabilities and signal-to-noise ratio (SNR) conditions, and the detection success rate is higher than that of the existing technology under different SNR conditions.

7. A Transformer-based large-scale Internet of Things device activity detection system implementing the Transformer-based large-scale Internet of Things device activity detection method according to any one of claims 1 to 6, characterized in that: The Transformer-based large-scale IoT device activity detection system includes: A data set construction module is used to construct a received signal data set under different activation probabilities for each available device; The prediction module is used to design an activity estimation module based on the impact of device activation probability on received signals, and is used to predict the device activation probability from the received signals; The feature extraction module is used to analyze the sparsity and regularity of the received signal matrix for each device. It uses the output of the activity estimation module as prior information and proposes a dual-sparse Transformer module to extract and optimize features from the entire dataset, resulting in a dual-sparse Transformer-based device activity detection model. The detection module is used to detect device activity using a dual-sparse Transformer-based device activity detection model.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the Transformer-based large-scale Internet of Things device activity detection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor performs the steps of the Transformer-based large-scale Internet of Things device activity detection method according to any one of claims 1 to 6.

10. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the Transformer-based large-scale Internet of Things device activity detection system as described in claim 7.

Images