Incremental learning method and system for dynamic pulsar data analysis

Through incremental learning model and Bayesian neural network, the pulsar recognition model is dynamically adjusted, which solves the problems of data drift and equipment differences, improves the accuracy and efficiency of pulsar recognition, and is suitable for rapidly changing astronomical observation data.

CN118916679AActive Publication Date: 2024-11-08HARBIN INST OF TECH
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411056094.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-11-08
Estimated Expiration
2044-08-02

AI Technical Summary

Technical Problem

Traditional pulsar recognition methods are inefficient in big data environments, are susceptible to data drift and equipment differences, and are difficult to adapt to data distribution changes, resulting in low recognition accuracy and efficiency.

Method used

The incremental learning model is used to combine Bayesian neural networks, and the model is dynamically adjusted to adapt to new data through weight adjustment, posterior distribution update and multimodal structure, and the model is enhanced by using Bayesian inference and adaptive playback mechanisms. and adaptability.

Benefits of technology

It significantly improves the accuracy and efficiency of pulsar recognition, can adapt to data drift and device differences, continuously learn and optimize models, and is suitable for rapidly changing astronomical observation data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916679A_ABST
    Figure CN118916679A_ABST
Patent Text Reader

Abstract

The invention provides an incremental learning method and system for dynamic pulsar data analysis. The pulsar recognition method and device are used for improving pulsar recognition accuracy and efficiency and solving the problem of data drift in pulsar candidate data. Firstly, pulsar data are preprocessed through a multi-source data integration and feature engineering technology so as to extract key features and enhance the input quality of a model; thirdly, basic model training is carried out through the initial Bayesian neural network, and the adaptability and stability of the model in various data environments are ensured; in order to cope with the challenge of dynamic change of data, an incremental learning strategy is adopted, so that the model can be updated in real time when receiving new data, the whole network does not need to be retrained, and the learning efficiency and the real-time performance are remarkably improved. In addition, by introducing model uncertainty evaluation, model parameters are further optimized and adjusted, and the accuracy and reliability of the recognition process are ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of pulsar candidate identification, and in particular to an incremental learning model for processing data drift problems, which is suitable for intelligent processing and analysis of astronomical observation data, and in particular to an incremental learning method and system for dynamic pulsar data analysis. Background Art

[0002] With the rapid development of astronomical observation technology and the significant improvement of data collection capabilities, scientists have been able to obtain a large amount of astronomical data, especially in the field of pulsar observation. Pulsars are highly compressed neutron stars with extreme physical properties and regular pulse emission characteristics, and are important objects in astrophysical research. Traditional pulsar identification methods mainly rely on pre-defined thresholds and manually selected features. These methods are often inefficient in big data environments and are easily affected by changes in data quality and data volume. Specifically, over time, due to the replacement and upgrading of observation equipment, changes in the observation environment, and adjustments to observation strategies, the nature and distribution of the collected data may change significantly, which is the so-called "data drift." The data drift problem poses a great challenge to the automatic identification system of pulsars, because traditional machine learning models usually assume that training data and future data maintain the same distribution, which is often not true in astronomical observation data. In addition, different astronomical telescopes and observation equipment may cause systematic errors between data due to differences in technology and design. These differences will affect the generalization ability and recognition accuracy of the model if they are not properly processed. At the same time, the inhomogeneity and sparsity of pulsar data themselves also increase the complexity of identification.

[0003] The prior art attempts to solve these problems through various statistical methods and machine learning techniques, such as using traditional algorithms such as support vector machines and random forests, and deep learning algorithms such as ResNet and CNN for classification and identification. However, these methods are still difficult to handle large-scale data sets in practical applications, especially the adaptability problem when the data distribution changes significantly. Therefore, it is urgent to develop new technical solutions to improve the accuracy and efficiency of pulsar identification technology in dynamic and big data environments, especially to be able to adaptively handle data drift and equipment differences, so as to achieve breakthrough progress in the field of astronomical data analysis. The present invention is proposed in this context, and aims to solve the above problems and improve the recognition performance of pulsar candidates by introducing incremental learning and Bayesian neural network technology. Summary of the invention

[0004] The present invention proposes an incremental learning method and system for dynamic pulsar data analysis, aiming to dynamically adjust the learning model to adapt to new data distribution, effectively deal with data drift problems, and improve the accuracy and efficiency of pulsar candidate identification.

[0005] The present invention is implemented by the following technical solution. The present invention proposes an incremental learning method for dynamic pulsar data analysis, the method comprising:

[0006] S1: Analyze the pulsar data set and determine the existence of data drift issues;

[0007] S2: Use weight adjustment techniques to deal with potential data imbalance caused by excessive non-pulsar candidate data;

[0008] S3: Replace the neural network in the model with a Bayesian neural network to capture the uncertainty of the data and enhance the robustness of the model;

[0009] S4: Design an incremental learning model that allows the incremental learning model to dynamically adjust when receiving new data and adapt to the new data set by updating the posterior distribution without training from scratch;

[0010] S5: The incremental learning model adopts a multimodal structure, based on a Bayesian neural network, and combines an adaptive replay mechanism into the incremental learning model training of the current task, so that the incremental learning model can utilize the knowledge of previous tasks and enhance the accuracy and stability when processing new tasks.

[0011] Furthermore, in S1, the optimal period, signal-to-noise ratio and optimal DM value of the pulsar candidate data are observed to show the changing trends over time, and it is determined that the data has a drift problem. In addition, it is necessary to observe the changes in the environment, the adjustment of the observation strategy, and the dynamic changes of the interstellar medium to cause the data drift problem.

[0012] Furthermore, in S2, the Focal loss function is introduced to dynamically adjust the loss weight of easy-to-classify samples, thereby reducing the negative impact of unbalanced data on the training of the incremental learning model.

[0013] Furthermore, in S3, the incremental learning model adopts a Bayesian neural network, and performs KL divergence calculation on each layer of the network to capture the uncertainty of the data and enhance the robustness of the model; in order to perform Bayesian inference, the evidence lower bound approximation is used as the objective function:

[0014] F(D,θ)=D KL (q(w|θ)||p(w))-E q(w|θ) [log P(D|w)] (1).

[0015] Furthermore, in S4, by iteratively updating the posterior distribution, the network gradually absorbs new data without forgetting old data; when new data arrives, the current posterior distribution is used as the new prior distribution, and the posterior distribution of the parameters is updated through Bayesian inference to achieve dynamic update of the incremental learning model; the dynamic update formula is expressed as:

[0016] P(w|D new ,D old )∝P(D new |w)P(w|D old ) (2).

[0017] Furthermore, in S5, the multimodal incremental learning model is based on a Bayesian neural network, and is modeled by combining a Bayesian one-dimensional convolutional neural network and a Bayesian two-dimensional convolutional neural network, and is further processed by a Bayesian linear network to generate a final output.

[0018] Furthermore, the Bayesian two-dimensional convolutional neural network is used to extract local features, and multiple feature enhancements and extractions are achieved through a SEBlock residual module including a batch normalization layer, an activation function, an adaptive average pooling layer, and a linear layer and sigmoid activation; the attention mechanism helps the incremental learning model to focus on important feature areas more effectively, thereby improving the quality of data representation.

[0019] The present invention proposes an incremental learning system for dynamic pulsar data analysis, the system comprising:

[0020] Analysis module: Analyze the pulsar data set to determine if there is a data drift problem;

[0021] Adjustment module: uses weight adjustment technology to deal with potential data imbalance caused by excessive non-pulsar candidate data;

[0022] Replacement module: Replace the neural network in the model with a Bayesian neural network to capture the uncertainty of the data and enhance the robustness of the model;

[0023] Design module: Design an incremental learning model that allows the incremental learning model to dynamically adjust when receiving new data and adapt to the new data set by updating the posterior distribution without training from scratch;

[0024] Processing module: The incremental learning model adopts a multimodal structure based on a Bayesian neural network, and combines an adaptive replay mechanism into the incremental learning model training of the current task, so that the incremental learning model can utilize the knowledge of previous tasks and enhance the accuracy and stability when processing new tasks.

[0025] The present invention also proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the incremental learning method for dynamic pulsar data analysis when executing the computer program.

[0026] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the incremental learning method for dynamic pulsar data analysis.

[0027] Beneficial effects of the present invention:

[0028] By introducing incremental learning and Bayesian neural network technology, the present invention can effectively adapt to data drift and systematic errors between different observation devices, thereby significantly improving the accuracy of pulsar identification. This adaptive learning mode ensures that the model can continue to learn and optimize in an environment where data characteristics gradually change. At the same time, due to the introduction of the incremental learning framework, the present invention can continuously learn from new data without retraining the model from scratch. This allows the model to better adapt to rapidly changing astronomical observation data, which is very beneficial in long-term astronomical research projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0030] Figure 1 It is an overall structural diagram of the incremental learning method used for dynamic pulsar data analysis in the present invention.

[0031] Figure 2 It is a schematic diagram of the incremental learning model framework. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] The present invention proposes an incremental learning method and system for dynamic pulsar data analysis, aiming to solve the problems of low pulsar identification accuracy and poor adaptability caused by data drift and equipment differences in the prior art. The implementation methods include the system architecture of the invention, the implementation details of key algorithms, the model training and update process, and how to use the present invention to process data generated by different observation equipment. Through these embodiments, the present invention can improve the accuracy of pulsar identification and the adaptability of the system, while optimizing resource utilization efficiency and the reliability of decision support.

[0034] Specifically, the present invention proposes an incremental learning method for dynamic pulsar data analysis, the method comprising:

[0035] S1: Analyze the pulsar data set and determine the existence of data drift issues;

[0036] S2: Use weight adjustment techniques to deal with potential data imbalance caused by excessive non-pulsar candidate data;

[0037] S3: Replace the neural network in the model with a Bayesian neural network to capture the uncertainty of the data and enhance the robustness of the model;

[0038] S4: Design an incremental learning model that allows the incremental learning model to dynamically adjust when receiving new data and adapt to the new data set by updating the posterior distribution without training from scratch;

[0039] S5: The incremental learning model adopts a multimodal structure, based on a Bayesian neural network, and combines an adaptive replay mechanism into the incremental learning model training of the current task, so that the incremental learning model can utilize the knowledge of previous tasks and enhance the accuracy and stability when processing new tasks.

[0040] In S1, the optimal period, signal-to-noise ratio and optimal DM value of the pulsar candidate data are observed to show the changing trends over time, and the data drift problem is determined. In addition, the change of the observation environment, the adjustment of the observation strategy, and the dynamic change of the interstellar medium that cause the data drift problem need to be observed.

[0041] In S2, the Focal loss function is introduced to dynamically adjust the loss weight of easy-to-classify samples, thereby reducing the negative impact of unbalanced data on the training of incremental learning models. The present invention compares the performance of traditional loss functions and Focal loss functions in processing highly unbalanced data sets through experiments. The Focal loss function optimizes the model's recognition ability for minority classes by reducing the weight of easy-to-classify samples and increasing the influence of difficult-to-classify samples. Experimental results show that the model using the Focal loss function is significantly better than the model using the traditional loss function in terms of recall rate and F1 score.

[0042] In S3, the incremental learning model adopts Bayesian neural network, and KL divergence calculation is performed on each layer of the network to capture the uncertainty of the data and enhance the robustness of the model; in order to implement Bayesian inference to optimize the network parameters, the evidence lower bound approximation is used as the objective function:

[0043] F(D,θ)=D KL (q(w|θ)||p(w))-E q(w|θ) [logP(D|w)] (1).

[0044] In S4, by iteratively updating the posterior distribution, the network gradually absorbs new data without forgetting old data; when new data arrives, the current posterior distribution is used as the new prior distribution, and the posterior distribution of the parameters is updated through Bayesian inference to achieve dynamic update of the incremental learning model; the dynamic update formula is expressed as:

[0045] P(w|D new ,D old )∝P(D new |w)P(w|D old ) (2).

[0046] In S5, the multimodal incremental learning model is based on a Bayesian neural network, and is modeled by combining a Bayesian one-dimensional convolutional neural network and a Bayesian two-dimensional convolutional neural network, and is further processed by a Bayesian linear network to generate the final output.

[0047] The Bayesian two-dimensional convolutional neural network is used to extract local features, and multiple feature enhancements and extractions are achieved through a SEBlock residual module including a batch normalization layer, an activation function, an adaptive average pooling layer, and a linear layer and sigmoid activation; the attention mechanism helps the incremental learning model to focus on important feature areas more effectively, thereby improving the quality of data representation.

[0048] The present invention proposes an incremental learning system for dynamic pulsar data analysis, the system comprising:

[0049] Analysis module: Analyze the pulsar data set to determine if there is a data drift problem;

[0050] Adjustment module: uses weight adjustment technology to deal with potential data imbalance caused by excessive non-pulsar candidate data;

[0051] Replacement module: Replace the neural network in the model with a Bayesian neural network to capture the uncertainty of the data and enhance the robustness of the model;

[0052] Design module: Design an incremental learning model that allows the incremental learning model to dynamically adjust when receiving new data and adapt to the new data set by updating the posterior distribution without training from scratch;

[0053] Processing module: The incremental learning model adopts a multimodal structure based on a Bayesian neural network, and combines an adaptive replay mechanism into the incremental learning model training of the current task, so that the incremental learning model can utilize the knowledge of previous tasks and enhance the accuracy and stability when processing new tasks.

[0054] The experimental results of the present invention on the pulsar candidate dataset are described in detail below.

[0055] In order to verify the effectiveness of the incremental learning method proposed in this paper in pulsar identification, we constructed a pulsar candidate dataset containing data from multiple telescopes. The dataset mainly comes from the Parkes telescope and the FAST telescope. The datasets from each data source have been preprocessed, including signal denoising, standardization, and feature extraction, to ensure data quality and consistency.

[0056] We use the FAST dataset as Task 1 and the HTRU dataset as Task 2. We randomly select 500 pulsar samples and 10,000 non-pulsar samples from each dataset as the training set. The test set randomly selects 500 pulsar samples and 500 non-pulsar samples from the remaining samples. We use cross-validation to test 10 times and finally take the average of the test results. In the model training stage, we use the Focal Loss loss function to mitigate the impact of sample imbalance in the training set. In addition, we combine the KL loss to form the overall loss, and the ratio of Focal Loss to KL loss is 1:1. In order to enhance the performance of the incremental learning model, when training Task 2, we randomly select 200 pulsar samples and 200 non-pulsar samples from the training set of Task 1 as playback data. The final test results are shown in Table 1, which shows that the model has achieved the expected results in both Task 1 and Task 2, and its performance matches the existing model.

[0057] Table 1

[0058]

[0059] The present invention also proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the incremental learning method for dynamic pulsar data analysis when executing the computer program.

[0060] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the incremental learning method for dynamic pulsar data analysis.

[0061] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DRRAM). It should be noted that the memory of the method described in the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0062] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disc (SSD)).

[0063] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0064] It should be noted that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor can be combined and performed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0065] The above is a detailed introduction to the incremental learning method and system for dynamic pulsar data analysis proposed in the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An incremental learning method for dynamic pulsar data analysis, characterized in that: The method comprises: S1: Analyze the pulsar data set and determine the existence of data drift issues; S2: Use weight adjustment techniques to deal with potential data imbalance caused by excessive non-pulsar candidate data; S3: Replace the neural network in the model with a Bayesian neural network to capture the uncertainty of the data and enhance the robustness of the model; S4: Design an incremental learning model that allows the incremental learning model to dynamically adjust when receiving new data and adapt to the new data set by updating the posterior distribution without training from scratch; S5: The incremental learning model adopts a multimodal structure, based on a Bayesian neural network, and combines an adaptive replay mechanism into the incremental learning model training of the current task, so that the incremental learning model can utilize the knowledge of previous tasks and enhance the accuracy and stability when processing new tasks.

2. The method according to claim 1, characterized in that: In S1, the optimal period, signal-to-noise ratio and optimal DM value of the pulsar candidate data are observed to show the changing trends over time, and the data drift problem is determined. In addition, the change of the observation environment, the adjustment of the observation strategy, and the dynamic change of the interstellar medium that cause the data drift problem need to be observed.

3. The method according to claim 1, characterized in that In S2, the Focal loss function is introduced to dynamically adjust the loss weight of easy-to-classify samples, thereby reducing the negative impact of unbalanced data on the training of the incremental learning model.

4. The method according to claim 1, characterized in that: In S3, the incremental learning model adopts Bayesian neural network, and KL divergence calculation is performed on each layer of the network to capture the uncertainty of the data and enhance the robustness of the model; in order to perform Bayesian inference, the evidence lower bound approximation is used as the objective function: F(D,θ)=D KL (q(w|θ)||p(w))-E q(w|θ) [logP(D|w)] (1).

5. The method according to claim 1, characterized in that In S4, by iteratively updating the posterior distribution, the network gradually absorbs new data without forgetting old data; when new data arrives, the current posterior distribution is used as the new prior distribution, and the posterior distribution of the parameters is updated through Bayesian inference to achieve dynamic update of the incremental learning model; the dynamic update formula is expressed as: P(w|D new ,D old )∝P(D new |w)P(w|D old ) (2)。 6. The method according to claim 1, characterized in that In S5, the multimodal incremental learning model is based on a Bayesian neural network, and is modeled by combining a Bayesian one-dimensional convolutional neural network and a Bayesian two-dimensional convolutional neural network, and is further processed by a Bayesian linear network to generate the final output.

7. The method according to claim 6, characterized in that The Bayesian two-dimensional convolutional neural network is used to extract local features, and multiple feature enhancement and extraction are achieved through a SEBlock residual module including a batch normalization layer, an activation function, an adaptive average pooling layer, and a linear layer and sigmoid activation; The attention mechanism helps the incremental learning model focus on important feature areas more effectively, thereby improving the quality of data representation.

8. An incremental learning system for dynamic pulsar data analysis, characterized in that: The system comprises: Analysis module: Analyze the pulsar data set to determine if there is a data drift problem; Adjustment module: uses weight adjustment technology to deal with potential data imbalance caused by excessive non-pulsar candidate data; Replacement module: Replace the neural network in the model with a Bayesian neural network to capture the uncertainty of the data and enhance the robustness of the model; Design module: Design an incremental learning model that allows the incremental learning model to dynamically adjust when receiving new data and adapt to the new data set by updating the posterior distribution without training from scratch; Processing module: The incremental learning model adopts a multimodal structure based on a Bayesian neural network, and combines an adaptive replay mechanism into the incremental learning model training of the current task, so that the incremental learning model can utilize the knowledge of previous tasks and enhance the accuracy and stability when processing new tasks.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Continuous learning method based on Bayesian variational inference

    CN115731396A

  • Comprehensive pulsar time analysis method and system based on iterative filtering

    CN115774835A

  • Alzheimer disease early recognition system based on multi-center structure magnetic resonance image

    CN116310479A

  • Pulsar candidate body identification method and system based on self-normalization classification model

    CN117591961A

  • Multimodal psychological state evaluation system and method for prisoners based on incremental learning

    CN118366653A