Deep neural network analysis method based on progressive thinking and causal reasoning

Abstract

The invention discloses a deep neural network analysis method based on progressive thinking and causal reasoning. The method comprises the following steps: inputting obtained original input data into an input encoder for processing to form a standardized high-dimensional feature vector; inputting the high-dimensional feature vector into a progressive thinking module for progressive thinking, and evaluating the confidence of thinking results of each layer by using a confidence predictor; determining the termination state of the thinking cycle according to the evaluated confidence; and after the determination is completed, decoding the high-dimensional feature vector after thinking through an output decoder to generate a final analysis result. According to the application, through a progressive thinking mechanism, multi-step reasoning can be carried out on complex problems, each step is combined with knowledge of the memory module and analysis of the causal reasoning module, the ability of solving complex tasks is remarkably improved, and meanwhile, through recording the thinking process of each step and confidence assessment, the decision process is more transparent, and the interpretability is enhanced.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and deep learning technology, specifically to a deep neural network analysis method based on progressive thinking and causal reasoning. Background Technology

[0002] With the rapid development of artificial intelligence technology, deep neural networks have made significant breakthroughs in fields such as image recognition and natural language processing. However, existing deep neural network models still have fundamental shortcomings when dealing with complex tasks that require multi-step logical reasoning.

[0003] First, traditional neural networks typically use a single forward propagation for prediction, lacking the ability to think progressively about complex problems, resulting in limited performance when faced with tasks requiring multi-step reasoning. Second, the decision-making process of neural networks is opaque, making it difficult to understand how the network arrives at its conclusions, lacking interpretability. Moreover, traditional neural networks primarily learn statistical correlations in data rather than causal relationships, limiting the model's application in intervention scenarios and counterfactual reasoning. Furthermore, existing models lack effective long-term memory mechanisms, making it difficult to store and retrieve past experiences and knowledge. Summary of the Invention

[0004] In order to solve the above-mentioned technical problems, this application proposes the following technical solution:

[0005] This application provides a deep neural network analysis method based on progressive thinking and causal reasoning, including:

[0006] The acquired raw input data is fed into the input encoder for processing to form a standardized high-dimensional feature vector;

[0007] The high-dimensional feature vector is input into the progressive thinking module for progressive thinking, and the confidence predictor is used to evaluate the confidence of each thinking result.

[0008] The termination state of the thinking loop is determined based on the confidence level of the assessment;

[0009] Once determined, the output decoder decodes the high-dimensional feature vector after consideration to generate the final analysis result.

[0010] In one possible implementation, the step of inputting the acquired raw input data into an input encoder for processing to form a standardized high-dimensional feature vector includes:

[0011] The acquired raw input data undergoes outlier detection and correction.

[0012] After mapping the processed data to a high-dimensional space through linear transformation, normalization is performed using application-layer normalization.

[0013] Subsequently, a nonlinear transformation is introduced through an activation function, and the normalized data is regularized using Dropout technology to form a standardized high-dimensional feature vector.

[0014] In one possible implementation, the progressive thinking module includes multiple layers of thinking units, each layer employing a Transformer encoder layer. Each Transformer encoder layer includes a multi-head self-attention mechanism module and a feedforward neural network. The input of the multi-head self-attention mechanism module is connected to the output of the memory module, and the output of the multi-head self-attention mechanism module is connected to the input of the feedforward neural network. The output of the feedforward neural network is connected to the input of the causal inference module, which includes a causal variable extractor, a causal graph learner, and a causal inference network.

[0015] In one possible implementation, the high-dimensional feature vector is input into a progressive thinking module for progressive thinking, while a confidence predictor is used to evaluate the confidence level of each thinking result, including:

[0016] The high-dimensional feature vector is input into the progressive thinking module and then sends a query request to the memory module.

[0017] The memory module retrieves relevant historical knowledge and experience based on the current hidden state;

[0018] The retrieved memory information is fused with the high-dimensional feature vector and then input into the first-level thinking unit of the progressive thinking module. The multi-head self-attention mechanism module identifies and focuses key information on the fused information.

[0019] A feedforward neural network is used to perform nonlinear feature transformation on the key information after identification and focusing, and the transformation result is input into the causal reasoning module for causal reasoning.

[0020] After fusing the results of causal reasoning with the results of the first-level thinking as the final result of the first-level thinking, a confidence predictor is used to evaluate the confidence level of the final result of the first-level thinking.

[0021] In one possible implementation, the memory module retrieves relevant historical knowledge and experience based on the current hidden state, including:

[0022] Project the current query vector onto the key space to form a query key representation;

[0023] The similarity score between the query key and all memory keys is calculated using an attention mechanism;

[0024] Based on the calculated similarity weights, the corresponding memory values ​​are weighted and summed to obtain the knowledge content most relevant to the current query.

[0025] In one possible implementation, the similarity score between the query key and all memory keys is calculated using an attention mechanism. Based on the calculated similarity weights, the corresponding memory values ​​are weighted and summed to obtain the knowledge content most relevant to the current query, including:

[0026] The query matrix Q, the key matrix K, and the value matrix V are generated through three different linear transformation layers, respectively.

[0027] Calculate the dot product of the query matrix Q and the key matrix K to obtain the attention score matrix;

[0028] After scaling and normalizing the attention score matrix, the scaled score matrix is ​​normalized row by row using the Softmax function to ensure that the weights of each row sum to 1, forming an attention weight matrix in the form of a probability distribution.

[0029] The attention weight matrix is ​​multiplied by the value matrix V to achieve dynamic weighted fusion of information from different locations.

[0030] In one possible implementation, the step of using a feedforward neural network to perform nonlinear feature transformation on the identified and focused key information, and inputting the transformation result into a causal inference module for causal inference, includes:

[0031] After receiving the transformation result, the causal reasoning module identifies and extracts potential key causal variables through the causal variable extractor;

[0032] A causal graph learner is used to analyze the causal relationships between the key causal variables to form a directed acyclic graph structure;

[0033] Simultaneously, the extracted key causal variables are combined with the original features through a causal inference network to perform comprehensive reasoning and generate causally enhanced feature representations.

[0034] In one possible implementation, the step of using a causal graph learner to analyze the causal relationships between the key causal variables to form a directed acyclic graph structure includes:

[0035] Create a causal relationship matrix of a predefined size, where each element represents the potential causal strength between two variables;

[0036] A lower triangular mask matrix is ​​created based on the temporal principle of causal relationships. The upper triangular part of the mask matrix is ​​set to 0, and the lower triangular part is set to 1.

[0037] The lower triangular mask matrix is ​​multiplied element-wise with the original causal relationship matrix, forcibly setting all causal connections in the upper triangular part to zero.

[0038] After limiting the numerical range of the masked causal matrix, L1 regularization is used as a penalty term to encourage the sparsity of the causal graph.

[0039] After each update, we verify whether the generated causal graph satisfies the directed acyclic graph condition and dynamically adjust the variable sorting or causal threshold based on the performance feedback during training, thereby optimizing the expressive power of the causal structure while maintaining the constraints of the directed acyclic graph.

[0040] In one possible implementation, determining the termination state of the thinking loop based on the assessed confidence level includes:

[0041] Obtain the confidence score for the assessment.

[0042] The confidence score of the assessment is judged, and the thinking process is terminated if the confidence score reaches a preset threshold.

[0043] If the confidence score does not reach the preset threshold, the final thinking result of the first layer is used as input to the second layer thinking unit for thinking, and the confidence of the final thinking result of the second layer is evaluated.

[0044] In one possible implementation, the step of taking the final thinking result of the first layer as input to the second layer thinking unit for further thinking, and simultaneously evaluating the confidence level of the final thinking result of the second layer, includes:

[0045] After inputting the final thinking result of the first layer into the progressive thinking module, a query request is sent to the memory module.

[0046] The memory module retrieves relevant historical knowledge and experience based on the current hidden state;

[0047] The retrieved memory information is fused with the final thinking result of the first layer and then input into the second layer thinking unit of the progressive thinking module. The multi-head self-attention mechanism module identifies and focuses key information on the fused information.

[0048] A feedforward neural network is used to perform nonlinear feature transformation on the key information after identification and focusing, and the transformation result is input into the causal reasoning module for causal reasoning.

[0049] The results of causal reasoning and the results of the second-level thinking are integrated to form the final result of the second-level thinking, and the confidence level of the final result of the second-level thinking is evaluated.

[0050] Compared with the prior art, the beneficial effects of this application are as follows:

[0051] This application employs a progressive thinking mechanism, enabling multi-step reasoning for complex problems. Each step combines knowledge from the memory module with analysis from the causal reasoning module, significantly improving the ability to solve complex tasks. Furthermore, by recording each step of the thinking process and confidence assessment, the decision-making process becomes more transparent, enhancing interpretability. The memory module allows the model to store and retrieve relevant knowledge, improving its ability to handle long-term dependent and knowledge-intensive tasks. This application is applicable to various fields requiring complex reasoning, such as medical diagnosis, financial analysis, and scientific research, demonstrating broad applicability. Attached Figure Description

[0052] Figure 1 A flowchart illustrating a deep neural network analysis method based on progressive thinking and causal reasoning, provided for an embodiment of this application;

[0053] Figure 2 A deep neural network structure diagram based on progressive thinking and causal reasoning provided for embodiments of this application;

[0054] Figure 3 A flowchart providing a first-level perspective for embodiments of this application;

[0055] Figure 4 A flowchart illustrating the self-attention mechanism to confidence prediction provided in this application embodiment;

[0056] Figure 5 A flowchart illustrating causal reasoning using a causal reasoning module, provided for an embodiment of this application. Detailed Implementation

[0057] The present solution will now be described in conjunction with the accompanying drawings and specific embodiments.

[0058] Figure 1 A flowchart illustrating a deep neural network analysis method based on progressive thinking and causal reasoning, provided in an embodiment of this application, is shown below. Figure 1 This embodiment of a deep neural network analysis method based on progressive thinking and causal reasoning includes:

[0059] S101, the acquired raw input data is input into the input encoder for processing to form a standardized high-dimensional feature vector.

[0060] See Figure 2In this embodiment, the input encoder, as the first component and entry point of the entire neural network, has the core task of converting the original low-dimensional input data into a high-dimensional feature representation suitable for deep neural network processing. This process simulates the human brain's conversion of external sensory information into internal neural signals. The input encoder uses a linear transformation to map the original data with an input dimension of 100 to a feature space with a hidden dimension of 256. This dimensional expansion not only preserves the structural information of the original data but also provides a rich feature representation space for subsequent complex inference processes. Furthermore, the layer normalization mechanism introduced during the encoding process ensures the stability of the data distribution and avoids gradient explosion or vanishing problems caused by excessively large gradients.

[0061] In this embodiment, when generating standardized high-dimensional feature vectors, the input data is first preprocessed, including outlier detection and correction, to ensure the quality and stability of the input data. Then, a linear transformation maps the processed data to a high-dimensional space to capture richer feature information. Next, layer normalization is applied to ensure the consistency of feature distribution and avoid training instability caused by differences in data scale. Subsequently, a nonlinear transformation is introduced through an activation function, enabling the network to learn and represent complex nonlinear relationships, overcoming the limitation of purely linear models that can only handle linearly separable problems. The introduction of the activation function is key to the powerful expressive ability of neural networks, allowing them to approximate arbitrarily complex function mapping relationships, thereby handling complex reasoning tasks in the real world. Finally, Dropout is used for regularization, randomly disconnecting some neuron connections during training to prevent the model from overfitting the training data and improve the model's generalization ability. Dropout is a regularization technique used to prevent overfitting when training neural networks. In each training iteration, a portion of neurons are randomly dropped, i.e., their outputs are set to zero, thereby reducing the interdependence between neurons, enhancing the model's generalization ability, and avoiding overfitting. This method is called the dropout method. After the complete processing described above, the input encoder outputs a standardized high-dimensional feature vector. This vector preserves the core semantic information of the original data and possesses characteristics of numerical stability, uniform distribution, and rich representation. This high-dimensional feature vector serves as the initial hidden state for subsequent progressive thinking modules, providing a stable and reliable feature foundation for the entire network's multi-step reasoning, memory retrieval, and causal analysis processes.

[0062] S102, input the high-dimensional feature vector into the progressive thinking module to perform progressive thinking, and at the same time use the confidence predictor to evaluate the confidence of the current thinking result.

[0063] In this embodiment, the progressive thinking module includes multiple thinking units, each employing a Transformer encoder layer. Each layer represents a thinking step. Each Transformer encoder layer includes a multi-head self-attention mechanism module and a feedforward neural network. The input of the multi-head self-attention mechanism module is connected to the output of the memory module, and the output of the multi-head self-attention mechanism module is connected to the input of the feedforward neural network. The output of the feedforward neural network is connected to the input of the causal inference module. The causal inference module includes a causal variable extractor, a causal graph learner, and a causal inference network. The causal variable extractor identifies and extracts potential causal variables from the input features. These variables represent key factors influencing the final result. The causal graph learner is responsible for learning the causal relationships between variables and constructing a directed acyclic graph. The causal inference network combines the original features with the causal variables for final inference. To achieve parallel processing and multi-dimensional analysis of information, this embodiment uses an 8-head multi-attention mechanism, with each head focusing on different aspects of the input information. Regarding the number of thinking layers, 3 layers are used for simple tasks, while 6 layers are used for complex tasks, ensuring that the depth of inference matches the complexity of the problem.

[0064] See Figure 3 In this embodiment, after the high-dimensional feature vector is input into the progressive thinking module, a query request is sent to the memory module. The memory module retrieves relevant historical knowledge and experience based on the current hidden state, fuses the retrieved memory information with the high-dimensional feature vector to form an enhanced input representation, and then inputs it into the first-layer thinking unit of the progressive thinking module. A multi-head self-attention mechanism module identifies and focuses key information on the fused information. A feedforward neural network performs nonlinear feature transformation on the identified and focused key information, and the transformation result is input into the causal reasoning module for causal reasoning. The result of the causal reasoning is fused with the result of the first-layer thinking as the final first-layer thinking result. Simultaneously, a confidence predictor evaluates the confidence level of this step. In this embodiment, the confidence predictor is equipped with a threshold control strategy. When the confidence level reaches a preset threshold, the thinking process can be terminated to avoid unnecessary computational overhead. This mechanism improves computational efficiency while ensuring reasoning quality.

[0065] In this embodiment, the memory module does not simply store thought results, but rather enables two-way interaction. It stores key knowledge and experience from the historical reasoning process, and at each thinking step, it retrieves relevant information based on the current query vector, providing background knowledge support for the current reasoning. The retrieval results from the memory module, along with the input data, serve as input to the thinking unit, enabling a knowledge-enhanced reasoning process.

[0066] During memory storage, the system employs an intelligent replacement strategy to maintain memory capacity. When new knowledge needs to be stored, the system prioritizes replacing memory units with lower usage frequency, ensuring that important and frequently used knowledge is retained long-term. Memory updates use a gradual approach; new memory content does not completely overwrite existing information but is integrated with it through a weighted average, maintaining memory stability while adapting to new knowledge input. Simultaneously, the memory module is designed with a fixed capacity of storage space, achieving efficient capacity management through a frequency tracking mechanism. When the storage space reaches its limit, the system automatically cleans up long-unused or less important memory units based on a comprehensive evaluation of usage frequency and access time, freeing up storage space for new, important knowledge and ensuring the memory system always maintains an optimal knowledge storage state.

[0067] In this embodiment, the memory module is designed with a key-value-frequency triplet structure: the memory key is responsible for indexing and retrieving the stored content, similar to a library's classification catalog system, providing a unique identifier for each stored knowledge fragment; the memory value stores the specific knowledge content and feature representation, including key information and experience extracted from the historical reasoning process; and the frequency is used to track the access history and usage statistics of each memory unit, providing an important assessment basis for the memory update strategy.

[0068] When the progressive thinking module requires relevant knowledge, it first projects the current query vector into the key space to form a query key representation. Then, it calculates the similarity score between the query key and all memory keys using an attention mechanism. The higher the similarity score, the stronger the relevance of the memory content to the current reasoning requirement. Based on the calculated similarity weights, the corresponding memory values ​​are weighted and summed to obtain the knowledge content most relevant to the current query. Simultaneously, the system updates the usage frequency count of accessed memory units, providing reference information for subsequent memory management.

[0069] See Figure 4 In this embodiment, dynamic weighting of information at different positions in the input sequence is achieved through the interaction of three matrices: query, key, and value. The first step involves generating the three matrices: the input sequence is processed through three different linear transformation layers to generate a query matrix Q, a key matrix K, and a value matrix V. The query matrix Q represents the information that is desired at the current position, the key matrix K represents the information that each position can provide, and the value matrix V represents the actual information content contained at each position. These three matrices typically have the same dimension to ensure compatibility in subsequent calculations.

[0070] The second step is similarity calculation. This involves calculating the dot product of the query matrix Q and the key matrix K to obtain the attention score matrix. Specifically, this is done by multiplying the transposes of Q and K. Each element in the resulting matrix represents the relevance strength between the corresponding query position and the key position. Position pairs with higher relevance receive higher scores, indicating a stronger association between the information from these two positions.

[0071] Step 3: Scaling and normalization. To prevent the gradient vanishing problem caused by an excessively large dot product, the attention score is divided by a scaling factor, usually the square root of the feature dimension. Then, the scaled score matrix is ​​normalized row by row using the Softmax function to ensure that the sum of the weights in each row is 1, forming an attention weight matrix in the form of a probability distribution.

[0072] Step 4: Dynamic weighted summation. The normalized attention weight matrix is ​​multiplied by the value matrix V to achieve dynamic weighted fusion of information from different locations. In this process, locations with higher attention weights contribute more information to the final output, while locations with lower weights contribute less. This mechanism enables the model to automatically identify and focus on the information most relevant to the current task.

[0073] Step 5: Multi-head parallel processing. To capture different types of relationships from multiple perspectives, the system employs a multi-head attention mechanism. The input features are divided into multiple subspaces, and each subspace independently performs the calculations of the above four steps, resulting in multiple different attention outputs. Each attention head focuses on a specific type of relationship pattern, such as semantic relationships, positional relationships, or other potential feature associations.

[0074] Step 6: Multi-head fusion and output. The outputs of all attention heads are concatenated and then fused through a linear transformation layer to generate the final self-attention output. This fusion process integrates complementary information from different heads, forming a more comprehensive and richer feature representation.

[0075] Step 7: Residual Connections and Layer Normalization. The output of the self-attention layer is residually connected to the original input, and then layer normalization is applied to ensure the stability of the information flow and the effectiveness of gradient propagation. Residual connections preserve the original information, preventing information loss problems in deep networks.

[0076] See Figure 5 Simultaneously with each thinking step, the causal inference module receives the output of the thinking unit and performs causal analysis. The causal variable extractor first identifies potential key causal variables from the feature representation. The causal graph learner then analyzes the causal relationships between these variables and constructs a directed acyclic graph structure. The causal inference network combines the original features with the causal variables to perform inference calculations. The directed acyclic graph constraint mechanism ensures that the learned causal structure conforms to logical norms and avoids the generation of circular causality.

[0077] In this embodiment, the unidirectionality of the causal relationship is ensured through the lower triangular matrix mask, avoiding the generation of circular causality. The specific implementation steps of the constraint mechanism include: The first step: Initialization of the causal graph matrix. The system first constructs an N×N causal relationship matrix, where N represents the total number of causal variables. Each element in the matrix represents the potential causal strength between two variables, and the initial value is obtained through neural network learning.

[0078] The second step: Generation of the lower triangular mask. Create a lower triangular mask matrix, in which the upper triangular part including the diagonal is all set to 0, and the lower triangular part is set to 1. This design is based on the principle of time sequence of causal relationships: if variables are arranged in a certain logical order, only variables with smaller serial numbers can affect variables with larger serial numbers, and vice versa does not hold.

[0079] The third step: Mask application and constraint enforcement. Perform an element-by-element multiplication operation on the lower triangular mask and the original causal relationship matrix, forcing all causal connections in the upper triangular part to be set to zero. This operation ensures the unidirectionality of the causal relationship: variable i can only affect variable j (when i < j), and variable j cannot affect variable i in reverse, fundamentally eliminating the possibility of circular causality.

[0080] The fourth step: Numerical stability processing. Limit the numerical range of the causal matrix after mask processing. Usually, the value range of matrix elements is constrained within the interval [-5, 5] to prevent numerical instability problems caused by excessive causal weights.

[0081] The fifth step: Introduction of sparsity constraint. Encourage the sparsity of the causal graph through the L1 regularization penalty term, that is, tend to learn a simple causal structure with fewer connection edges. This conforms to the Occam's razor principle and avoids overly complex causal relationship networks.

[0082] The sixth step: Causal validity verification. Verify whether the generated causal graph satisfies the condition of a directed acyclic graph after each update: check whether there are self-loops (the diagonal elements should be 0), verify whether it strictly follows the lower triangular structure, and confirm that all causal weights are within a reasonable numerical range.

[0083] The seventh step: Dynamic structure optimization. Dynamically adjust the variable sorting or causal threshold according to the performance feedback during the training process, and optimize the expressive ability of the causal structure under the premise of maintaining the directed acyclic graph constraint. When the weights of certain causal connections are close to zero for a long time, the system will completely cut them off to further simplify the causal graph structure.

[0084] Furthermore, the causal reasoning module also possesses important intervention analysis capabilities. When it is necessary to assess the impact of a change in a certain variable, the module can simulate the change in the specific variable through intervention masks and intervention values, and then predict the degree of impact of this change on the final outcome based on the learned causal graph structure, providing a scientific causal basis for decision-making.

[0085] S103, determine the termination state of the thinking loop based on the assessed confidence level.

[0086] In this embodiment, a confidence score is obtained for the assessed confidence level. After each thinking step, the confidence predictor assesses the confidence level of each level of reasoning result, generating a confidence score between 0 and 1. The system ensures through a constraint mechanism that the confidence level gradually increases as the thinking steps deepen, which aligns with the characteristic of gradually increasing certainty in human thinking. This design provides an interpretability indicator for each step of reasoning, making the entire decision-making process transparent and traceable. By judging the confidence score of the assessed confidence level, if the confidence score reaches 0.8, the thinking process terminates; if the confidence score does not reach 0.8, the final thinking result of the first level is used as input to the second level thinking unit for further thinking, while the confidence level of the final thinking result of the second level is assessed.

[0087] In this embodiment, after the final thinking result of the first layer is input into the progressive thinking module, a query request is sent to the memory module. The memory module retrieves relevant historical knowledge and experience based on the current hidden state. The retrieved memory information is then fused with the final thinking result of the first layer and input into the second layer thinking unit of the progressive thinking module. The multi-head self-attention mechanism module identifies and focuses key information on the fused information. The feedforward neural network is used to perform nonlinear feature transformation on the identified and focused key information, and the transformation result is input into the causal reasoning module for causal reasoning. The result of causal reasoning is fused with the result of the second layer thinking as the final thinking result of the second layer. At the same time, the confidence level of the final thinking result of the second layer is evaluated. This progressive thinking is repeated until the confidence score of the current layer reaches 0.8, at which point the thinking process is terminated.

[0088] S104, once determined, the output decoder decodes the high-dimensional feature vector after consideration to generate the final analysis result.

[0089] In this embodiment, the output decoder is responsible for converting the high-dimensional feature vector representation, after multiple rounds of consideration, into the final output result. This process requires mapping the 256-dimensional internal representation back to the original output space. The output decoder can generate different output formats according to different application scenarios, including classification results, regression values, probability distributions, etc. It can also output the complete inference trajectory, demonstrating the model's thought process.

[0090] The entire analysis process employs an iterative optimization strategy, where the output of each thinking step becomes the input for the next, forming a progressive chain of reasoning. As the depth of thinking increases, the model's understanding of the problem deepens, and the solution becomes more refined. Ultimately, the system integrates the results of all thinking steps, generating the final analysis results through an output decoder, while also providing a complete reasoning trajectory, causal relationship diagram, and confidence level change curve.

[0091] In this embodiment, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0092] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0093] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A deep neural network analysis method based on progressive thinking and causal reasoning, characterized in that, include: The acquired raw input data is fed into the input encoder for processing to form a standardized high-dimensional feature vector; The high-dimensional feature vector is input into the progressive thinking module for progressive thinking, and the confidence predictor is used to evaluate the confidence of each thinking result. The termination state of the thinking loop is determined based on the confidence level of the assessment; Once determined, the output decoder decodes the high-dimensional feature vector after consideration to generate the final analysis result.

2. The deep neural network analysis method based on progressive thinking and causal reasoning according to claim 1, characterized in that, The process of inputting the acquired raw input data into the input encoder for processing to form a standardized high-dimensional feature vector includes: The acquired raw input data undergoes outlier detection and correction. After mapping the processed data to a high-dimensional space through linear transformation, normalization is performed using application-layer normalization. Subsequently, a nonlinear transformation is introduced through an activation function, and the normalized data is regularized using Dropout technology to form a standardized high-dimensional feature vector.

3. The deep neural network analysis method based on progressive thinking and causal reasoning according to claim 1, characterized in that, The progressive thinking module includes multiple thinking units, each employing a Transformer encoder layer. Each Transformer encoder layer includes a multi-head self-attention mechanism module and a feedforward neural network. The input of the multi-head self-attention mechanism module is connected to the output of the memory module, and the output of the multi-head self-attention mechanism module is connected to the input of the feedforward neural network. The output of the feedforward neural network is connected to the input of the causal inference module, which includes a causal variable extractor, a causal graph learner, and a causal inference network.

4. The deep neural network analysis method based on progressive thinking and causal reasoning according to claim 1, characterized in that, The high-dimensional feature vector is input into the progressive thinking module for progressive thinking, and a confidence predictor is used to evaluate the confidence of each thinking result, including: The high-dimensional feature vector is input into the progressive thinking module and then sends a query request to the memory module. The memory module retrieves relevant historical knowledge and experience based on the current hidden state; The retrieved memory information is fused with the high-dimensional feature vector and then input into the first-level thinking unit of the progressive thinking module. The multi-head self-attention mechanism module identifies and focuses key information on the fused information. A feedforward neural network is used to perform nonlinear feature transformation on the key information after identification and focusing, and the transformation result is input into the causal reasoning module for causal reasoning. After fusing the results of causal reasoning with the results of the first-level thinking as the final result of the first-level thinking, a confidence predictor is used to evaluate the confidence level of the final result of the first-level thinking.

5. The deep neural network analysis method based on progressive thinking and causal reasoning according to claim 4, characterized in that, The memory module retrieves relevant historical knowledge and experience based on the current hidden state, including: Project the current query vector onto the key space to form a query key representation; The similarity score between the query key and all memory keys is calculated using an attention mechanism; Based on the calculated similarity weights, the corresponding memory values ​​are weighted and summed to obtain the knowledge content most relevant to the current query.

6. The deep neural network analysis method based on progressive thinking and causal reasoning according to claim 5, characterized in that, The process involves calculating the similarity score between the query key and all memory keys using an attention mechanism, and then weighting and summing the corresponding memory values ​​based on the calculated similarity weights to obtain the knowledge content most relevant to the current query. This includes: The query matrix Q, the key matrix K, and the value matrix V are generated through three different linear transformation layers, respectively. Calculate the dot product of the query matrix Q and the key matrix K to obtain the attention score matrix; After scaling and normalizing the attention score matrix, the scaled score matrix is ​​normalized row by row using the Softmax function to ensure that the weights of each row sum to 1, forming an attention weight matrix in the form of a probability distribution. The attention weight matrix is ​​multiplied by the value matrix V to achieve dynamic weighted fusion of information from different locations.

7. The deep neural network analysis method based on progressive thinking and causal reasoning according to claim 4, characterized in that, The process of using a feedforward neural network to perform nonlinear feature transformation on the identified and focused key information, and then inputting the transformation result into the causal inference module for causal inference, includes: After receiving the transformation result, the causal reasoning module identifies and extracts potential key causal variables through the causal variable extractor; A causal graph learner is used to analyze the causal relationships between the key causal variables to form a directed acyclic graph structure; Simultaneously, the extracted key causal variables are combined with the original features through a causal inference network to perform comprehensive reasoning and generate causally enhanced feature representations.

8. The deep neural network analysis method based on progressive thinking and causal reasoning according to claim 7, characterized in that, The step of using a causal graph learner to analyze the causal relationships between the key causal variables to form a directed acyclic graph structure includes: Create a causal relationship matrix of a predefined size, where each element represents the potential causal strength between two variables; A lower triangular mask matrix is ​​created based on the temporal principle of causal relationships. The upper triangular part of the mask matrix is ​​set to 0, and the lower triangular part is set to 1. The lower triangular mask matrix is ​​multiplied element-wise with the original causal relationship matrix, forcibly setting all causal connections in the upper triangular part to zero. After limiting the numerical range of the masked causal matrix, L1 regularization is used as a penalty term to encourage the sparsity of the causal graph. After each update, we verify whether the generated causal graph satisfies the directed acyclic graph condition and dynamically adjust the variable sorting or causal threshold based on the performance feedback during training, thereby optimizing the expressive power of the causal structure while maintaining the constraints of the directed acyclic graph.

9. The deep neural network analysis method based on progressive thinking and causal reasoning according to claim 1, characterized in that, The determination of the termination state of the thinking loop based on the assessed confidence level includes: Obtain the confidence score for the assessment. The confidence score of the assessment is judged, and the thinking process is terminated if the confidence score reaches a preset threshold. If the confidence score does not reach the preset threshold, the final thinking result of the first layer is used as input to the second layer thinking unit for thinking, and the confidence of the final thinking result of the second layer is evaluated.

10. The deep neural network analysis method based on progressive thinking and causal reasoning according to claim 9, characterized in that, The process of using the final thinking result of the first layer as input to the second layer thinking unit for further thinking, and simultaneously evaluating the confidence level of the final thinking result of the second layer, includes: After inputting the final thinking result of the first layer into the progressive thinking module, a query request is sent to the memory module. The memory module retrieves relevant historical knowledge and experience based on the current hidden state; The retrieved memory information is fused with the final thinking result of the first layer and then input into the second layer thinking unit of the progressive thinking module. The multi-head self-attention mechanism module identifies and focuses key information on the fused information. A feedforward neural network is used to perform nonlinear feature transformation on the key information after identification and focusing, and the transformation result is input into the causal reasoning module for causal reasoning. The results of causal reasoning and the results of the second-level thinking are integrated to form the final result of the second-level thinking, and the confidence level of the final result of the second-level thinking is evaluated.

Images