An incremental learning method and system for dynamic pulsar data analysis
By employing incremental learning and Bayesian neural network techniques, the problems of pulsar recognition accuracy and adaptability caused by data drift and equipment differences have been solved, achieving efficient pulsar recognition in dynamic astronomical data environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HARBIN INST OF TECH
- Filing Date
- 2024-08-02
- Publication Date
- 2026-05-08
AI Technical Summary
Existing technologies struggle to adapt to data drift and equipment differences when processing large-scale astronomical data, especially when data distribution changes significantly, resulting in low accuracy and efficiency in pulsar identification.
By employing incremental learning and Bayesian neural network techniques, and through weight adjustment, Bayesian neural network replacement, multimodal structure, and adaptive replay mechanism, an incremental learning model is designed to dynamically adjust the learning model, adapt to new datasets, and utilize knowledge from previous tasks, thereby enhancing recognition accuracy and stability.
It significantly improves the accuracy of pulsar identification and the adaptability of the system, enabling it to continuously learn and optimize in environments with gradually changing data characteristics, reducing the need for retraining the model.
Smart Images
Figure CN118916679B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of pulsar candidate identification technology, and in particular to an incremental learning model for handling data drift problems, applicable to intelligent processing and analysis of astronomical observation data, and especially to an incremental learning method and system for dynamic pulsar data analysis. Background Technology
[0002] With the rapid development of astronomical observation technology and the significant improvement in data acquisition capabilities, scientists are now able to acquire vast amounts of astronomical data, especially in the field of pulsar observation. Pulsars are highly compressed neutron stars with extreme physical properties and regular pulse emission characteristics, making them important objects of astrophysical research. Traditional pulsar identification methods mainly rely on predefined thresholds and manually selected features. These methods are often inefficient in large-scale data environments and are easily affected by changes in data quality and volume. Specifically, over time, due to the replacement and upgrading of observation equipment, changes in the observation environment, and adjustments to observation strategies, the properties and distribution of collected data may change significantly, a phenomenon known as "data drift." Data drift poses a significant challenge to automatic pulsar identification systems because traditional machine learning models typically assume that training data and future data maintain the same distribution, an assumption that often does not hold true for astronomical observation data. Furthermore, differences in technology and design between different astronomical telescopes and observation equipment can lead to systematic errors in the data. These differences, if not properly addressed, can affect the model's generalization ability and identification accuracy. Simultaneously, the inherent non-uniformity and sparsity of pulsar data itself also increase the complexity of the identification process.
[0003] Existing technologies attempt to address these problems using various statistical methods and machine learning techniques, such as traditional algorithms like Support Vector Machines and Random Forests, and deep learning algorithms like ResNet and CNNs for classification and recognition. However, these methods still struggle to handle large-scale datasets in practical applications, particularly regarding adaptability when data distribution changes significantly. Therefore, there is an urgent need to develop new technical solutions to improve the accuracy and efficiency of pulsar recognition technology in dynamic and big data environments, especially to adaptively handle data drift and equipment differences, thereby achieving breakthroughs in the field of astronomical data analysis. This invention is proposed against this backdrop, aiming to solve the aforementioned problems and improve the recognition performance of pulsar candidates by introducing incremental learning and Bayesian neural network techniques. Summary of the Invention
[0004] This invention proposes an incremental learning method and system for dynamic pulsar data analysis. It aims to dynamically adjust the learning model to adapt to new data distributions, effectively address data drift issues, and improve the accuracy and efficiency of pulsar candidate identification.
[0005] This invention is achieved through the following technical solution: This invention proposes an incremental learning method for dynamic pulsar data analysis, the method comprising:
[0006] S1: Analyze the pulsar dataset to determine the existence of a data drift problem;
[0007] S2: Employ weighting techniques to address potential data imbalance caused by an excessive amount of non-pulsar candidate data;
[0008] S3: Replace the neural network in the model with a Bayesian neural network to capture the uncertainty of the data and enhance the robustness of the model;
[0009] S4: Design an incremental learning model that allows the incremental learning model to dynamically adjust when receiving new data, adapting to the new dataset through posterior distribution updates, without having to train from scratch.
[0010] S5: The incremental learning model adopts a multimodal structure, based on a Bayesian neural network, and incorporates an adaptive replay mechanism into the training of the incremental learning model for the current task. This enables the incremental learning model to utilize knowledge from previous tasks and enhance the accuracy and stability when processing new tasks.
[0011] Furthermore, in S1, the changing trends of the optimal period, signal-to-noise ratio, and optimal DM value of the pulsar candidate data over time are observed to determine if there is a data drift problem. In addition, it is also necessary to observe changes in the observation environment, adjustments to the observation strategy, and the data drift problem caused by dynamic changes in the interstellar medium.
[0012] Furthermore, in S2, the Focal loss function is introduced to dynamically adjust the loss weights of easily classified samples, thereby mitigating the negative impact of imbalanced data on the training of the incremental learning model.
[0013] Furthermore, in S3, the incremental learning models all employ Bayesian neural networks, and KL divergence is calculated for each layer to capture data uncertainty and enhance model robustness; for Bayesian inference, an evidence lower bound approximation is used as the objective function:
[0014] F(D,θ)=D KL (q(w|θ)||p(w))-E q(w|θ) [log P(D|w)] (1).
[0015] Furthermore, in S4, by iteratively updating the posterior distribution, the network gradually absorbs new data without forgetting old data; when new data arrives, the current posterior distribution is used as the new prior distribution, and the posterior distribution of the parameters is updated through Bayesian inference, realizing the dynamic update of the incremental learning model; the dynamic update formula is expressed as:
[0016] P(w|D new D old )∝P(D new |w)P(w|D old (2).
[0017] Furthermore, in S5, the multimodal incremental learning model is based on a Bayesian neural network, and is modeled by a combination of a Bayesian one-dimensional convolutional neural network and a Bayesian two-dimensional convolutional neural network. It is further processed by a Bayesian linear network to generate the final output.
[0018] Furthermore, the Bayesian two-dimensional convolutional neural network is used to extract local features, and multiple feature enhancements and extractions are achieved through the SEBlock residual module, which includes batch normalization layers, activation functions, adaptive average pooling layers, linear layers, and sigmoid activation. The attention mechanism helps the incremental learning model to focus more effectively on important feature regions, thereby improving the quality of data representation.
[0019] This invention proposes an incremental learning system for dynamic pulsar data analysis, the system comprising:
[0020] Analysis module: Analyzes pulsar datasets to determine the existence of data drift issues;
[0021] Adjustment module: Employs weight adjustment techniques to address potential data imbalance caused by an excessive amount of non-pulsar candidate data;
[0022] Replacement module: Replace the neural network in the model with a Bayesian neural network to capture the uncertainty of the data and enhance the robustness of the model;
[0023] Design module: Design an incremental learning model that allows the incremental learning model to dynamically adjust when receiving new data, adapting to the new dataset through posterior distribution updates, without having to train from scratch;
[0024] Processing module: The incremental learning model adopts a multimodal structure, based on a Bayesian neural network, and incorporates an adaptive replay mechanism into the training of the incremental learning model for the current task. This enables the incremental learning model to utilize knowledge from previous tasks and enhance the accuracy and stability when processing new tasks.
[0025] The present invention also proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the incremental learning method for dynamic pulsar data analysis.
[0026] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the incremental learning method for dynamic pulsar data analysis.
[0027] The beneficial effects of this invention are:
[0028] This invention, by introducing incremental learning and Bayesian neural network techniques, effectively adapts to data drift and systematic errors between different observation devices, thereby significantly improving the accuracy of pulsar identification. This adaptive learning mode ensures that the model can continuously learn and optimize in environments where data characteristics are gradually changing. Simultaneously, due to the introduction of the incremental learning framework, this invention can continuously learn from new data without needing to retrain the model from scratch. This allows the model to better adapt to rapidly changing astronomical observation data, which is particularly beneficial in long-term astronomical research projects. Attached Figure Description
[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0030] Figure 1 This is an overall structural diagram of the incremental learning method for dynamic pulsar data analysis proposed in this invention.
[0031] Figure 2 This is a schematic diagram of the incremental learning model framework. Detailed Implementation
[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0033] This invention proposes an incremental learning method and system for dynamic pulsar data analysis, aiming to address the problems of low accuracy and poor adaptability in pulsar identification caused by data drift and equipment differences in existing technologies. The implementation includes the system architecture, implementation details of key algorithms, model training and update procedures, and how to utilize this invention to process data generated by different observation devices. Through these embodiments, this invention can improve the accuracy and adaptability of pulsar identification, while optimizing resource utilization efficiency and the reliability of decision support.
[0034] Specifically, this invention proposes an incremental learning method for dynamic pulsar data analysis, the method comprising:
[0035] S1: Analyze the pulsar dataset to determine the existence of a data drift problem;
[0036] S2: Employ weighting techniques to address potential data imbalance caused by an excessive amount of non-pulsar candidate data;
[0037] S3: Replace the neural network in the model with a Bayesian neural network to capture the uncertainty of the data and enhance the robustness of the model;
[0038] S4: Design an incremental learning model that allows the incremental learning model to dynamically adjust when receiving new data, adapting to the new dataset through posterior distribution updates, without having to train from scratch.
[0039] S5: The incremental learning model adopts a multimodal structure, based on a Bayesian neural network, and incorporates an adaptive replay mechanism into the training of the incremental learning model for the current task. This enables the incremental learning model to utilize knowledge from previous tasks and enhance the accuracy and stability when processing new tasks.
[0040] In S1, the changing trends of the optimal period, signal-to-noise ratio, and optimal DM value of the pulsar candidate data over time are observed to determine if there is a data drift problem. In addition, it is also necessary to observe changes in the observation environment, adjustments to the observation strategy, and the data drift problem caused by dynamic changes in the interstellar medium.
[0041] In step S2, the Focal loss function is introduced to dynamically adjust the loss weights of easily classified samples, mitigating the negative impact of imbalanced data on the training of the incremental learning model. This invention experimentally compares the performance of traditional loss functions and the Focal loss function in handling highly imbalanced datasets. The Focal loss function optimizes the model's ability to identify minority classes by reducing the weights of easily classified samples and increasing the influence of difficult-to-classify samples. Experimental results show that the model using the Focal loss function significantly outperforms the model using the traditional loss function in terms of recall and F1 score.
[0042] In S3, the incremental learning models all employ Bayesian neural networks, and KL divergence is calculated for each layer to capture data uncertainty and enhance model robustness. To implement Bayesian inference, network parameters are optimized using an evidence lower bound approximation as the objective function.
[0043] F(D,θ)=D KL (q(w|θ)||p(w))-E q(w|θ) [logP(D|w)] (1).
[0044] In step S4, the network gradually absorbs new data without forgetting old data by iteratively updating the posterior distribution. When new data arrives, the current posterior distribution is used as the new prior distribution, and the posterior distribution of the parameters is updated through Bayesian inference, thus achieving dynamic updating of the incremental learning model. The dynamic update formula is expressed as:
[0045] P(w|D new D old )∝P(D new |w)P(w|D old (2).
[0046] In S5, the multimodal incremental learning model is based on a Bayesian neural network. It is modeled by combining a Bayesian one-dimensional convolutional neural network and a Bayesian two-dimensional convolutional neural network, and further processed by a Bayesian linear network to generate the final output.
[0047] The Bayesian 2D convolutional neural network is used to extract local features, and multiple feature enhancements and extractions are achieved through the SEBlock residual module, which includes batch normalization layers, activation functions, adaptive average pooling layers, linear layers, and sigmoid activation. The attention mechanism helps the incremental learning model to focus more effectively on important feature regions, thereby improving the quality of data representation.
[0048] This invention proposes an incremental learning system for dynamic pulsar data analysis, the system comprising:
[0049] Analysis module: Analyzes pulsar datasets to determine the existence of data drift issues;
[0050] Adjustment module: Employs weight adjustment techniques to address potential data imbalance caused by an excessive amount of non-pulsar candidate data;
[0051] Replacement module: Replace the neural network in the model with a Bayesian neural network to capture the uncertainty of the data and enhance the robustness of the model;
[0052] Design module: Design an incremental learning model that allows the incremental learning model to dynamically adjust when receiving new data, adapting to the new dataset through posterior distribution updates, without having to train from scratch;
[0053] Processing module: The incremental learning model adopts a multimodal structure, based on a Bayesian neural network, and incorporates an adaptive replay mechanism into the training of the incremental learning model for the current task. This enables the incremental learning model to utilize knowledge from previous tasks and enhance the accuracy and stability when processing new tasks.
[0054] The experimental results of this invention on a pulsar candidate dataset are described in detail below.
[0055] To verify the effectiveness of the incremental learning method proposed in this invention for pulsar identification, we constructed a candidate pulsar dataset containing data from multiple telescopes. The datasets mainly come from the Parkes and FAST telescopes. Each dataset underwent preprocessing, including signal denoising, standardization, and feature extraction, to ensure data quality and consistency.
[0056] We used the FAST dataset as Task 1 and the HTRU dataset as Task 2. For each dataset, we randomly selected 500 pulsar samples and 10,000 non-pulsar samples as the training set. The test set consisted of 500 pulsar samples and 500 non-pulsar samples randomly selected from the remaining samples. We used cross-validation to test the model 10 times, and the average of the results was taken. During model training, we used the Focal Loss function to mitigate the impact of imbalanced samples in the training set. Furthermore, we combined Focal Loss with KL loss to form the overall loss, with a 1:1 ratio between Focal Loss and KL loss. To enhance the performance of the incremental learning model, during the training of Task 2, we randomly selected 200 pulsar samples and 200 non-pulsar samples from the training set of Task 1 as replay data. The final test results are shown in Table 1, demonstrating that the model achieved the expected results in both Task 1 and Task 2, and its performance is comparable to existing models.
[0057] Table 1
[0058]
[0059] The present invention also proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the incremental learning method for dynamic pulsar data analysis.
[0060] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the incremental learning method for dynamic pulsar data analysis.
[0061] The memory in this application embodiment can be volatile memory or non-volatile memory, or it can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DRRAM). It should be noted that the memory used in the methods described in this invention is intended to include, but is not limited to, these and any other suitable types of memory.
[0062] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).
[0063] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules in the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here.
[0064] It should be noted that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuitry in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above methods.
[0065] The above provides a detailed description of the incremental learning method and system for dynamic pulsar data analysis proposed in this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.
Claims
1. An incremental learning method for dynamic pulsar data analysis, characterized in that, The method includes: S1: Analyze the pulsar dataset to determine the existence of a data drift problem; S2: Employ weighting techniques to address potential data imbalance caused by an excessive amount of non-pulsar candidate data; S3: Replace the neural network in the model with a Bayesian neural network to capture the uncertainty of the data and enhance the robustness of the model; S4: Design an incremental learning model that allows the incremental learning model to dynamically adjust when receiving new data, adapting to the new dataset through posterior distribution updates, without having to train from scratch. S5: The incremental learning model adopts a multimodal structure, based on a Bayesian neural network, and incorporates an adaptive replay mechanism into the training of the incremental learning model for the current task, enabling the incremental learning model to utilize knowledge from previous tasks and enhance the accuracy and stability when processing new tasks. In S5, the multimodal incremental learning model is based on a Bayesian neural network, and is modeled by a combination of a Bayesian one-dimensional convolutional neural network and a Bayesian two-dimensional convolutional neural network. It is further processed by a Bayesian linear network to generate the final output. The Bayesian 2D convolutional neural network is used to extract local features, and multiple feature enhancements and extractions are achieved through the SEBlock residual module, which includes batch normalization layers, activation functions, adaptive average pooling layers, linear layers, and sigmoid activation. The attention mechanism helps the incremental learning model to focus more effectively on important feature regions, thereby improving the quality of data representation.
2. The method according to claim 1, characterized in that, In S1, the changing trends of the optimal period, signal-to-noise ratio, and optimal DM value of the pulsar candidate data over time are observed to determine if there is a data drift problem. In addition, it is also necessary to observe changes in the observation environment, adjustments to the observation strategy, and the data drift problem caused by dynamic changes in the interstellar medium.
3. The method according to claim 1, characterized in that, In S2, the Focal loss function is introduced to dynamically adjust the loss weight of easily classified samples, thereby mitigating the negative impact of imbalanced data on the training of the incremental learning model.
4. The method according to claim 1, characterized in that, In S3, the incremental learning models all employ Bayesian neural networks, and KL divergence is calculated for each layer to capture data uncertainty and enhance model robustness; for Bayesian inference, an evidence lower bound approximation is used as the objective function. （1）。 5. The method according to claim 1, characterized in that, In step S4, the network gradually absorbs new data without forgetting old data by iteratively updating the posterior distribution. When new data arrives, the current posterior distribution is used as the new prior distribution, and the posterior distribution of the parameters is updated through Bayesian inference, thus achieving dynamic updating of the incremental learning model. The dynamic update formula is expressed as: （2）。 6. An incremental learning system for dynamic pulsar data analysis, characterized in that, The system includes: Analysis module: Analyzes pulsar datasets to determine the existence of data drift issues; Adjustment module: Employs weight adjustment techniques to address potential data imbalance caused by an excessive amount of non-pulsar candidate data; Replacement module: Replace the neural network in the model with a Bayesian neural network to capture the uncertainty of the data and enhance the robustness of the model; Design module: Design an incremental learning model that allows the incremental learning model to dynamically adjust when receiving new data, adapting to the new dataset through posterior distribution updates, without having to train from scratch; Processing module: The incremental learning model adopts a multimodal structure, based on a Bayesian neural network, and incorporates an adaptive replay mechanism into the training of the incremental learning model for the current task, enabling the incremental learning model to utilize knowledge from previous tasks and enhance the accuracy and stability when processing new tasks. In the processing module, the multimodal incremental learning model is based on a Bayesian neural network, and is modeled by a combination of a Bayesian one-dimensional convolutional neural network and a Bayesian two-dimensional convolutional neural network. It is further processed by a Bayesian linear network to generate the final output. The Bayesian 2D convolutional neural network is used to extract local features, and multiple feature enhancements and extractions are achieved through the SEBlock residual module, which includes batch normalization layers, activation functions, adaptive average pooling layers, linear layers, and sigmoid activation. The attention mechanism helps the incremental learning model to focus more effectively on important feature regions, thereby improving the quality of data representation.
7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-5.
8. A computer-readable storage medium for storing computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-5.