Attention calibration method, device, medium and product based on prior knowledge constraint

Abstract

Embodiments of the present disclosure disclose a priori knowledge constraint based attention calibration method, device, medium and product. The method comprises: dividing input features obtained after preprocessing of multi-modal brain health data into multiple feature groups and setting prior weights; linearly transforming each feature group and splicing to generate a query matrix, a key matrix and a value matrix; performing scaled dot-product attention calculation on the query matrix and the key matrix to obtain an original attention weight matrix; constructing a prior weight matrix based on the prior weights of the feature groups and calibrating the original attention weight matrix to obtain a calibrated attention weight matrix; and performing weighted summation on the value matrix after normalization to obtain an attention output result, which is used to generate brain health evaluation data of a target object. The method can introduce prior calibration after normalization of attention weights and before weighted summation, thereby improving the accuracy and interpretability of the attention output without damaging feature interaction and gradient propagation.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to an attention calibration method, apparatus, medium, and product based on prior knowledge constraints. Background Technology

[0002] With the rapid development of neuroscience and artificial intelligence technologies, brain health assessment based on multimodal data has become an important direction for cognitive health management, early screening of brain diseases, and postoperative rehabilitation. Existing brain health analysis systems typically collect multimodal data such as electroencephalograms (EEGs), imaging, behavioral data, and cognitive questionnaires, and perform fusion analysis through deep learning models, especially self-attention mechanisms, to fully utilize the correlation between information from different modalities and improve the accuracy of the assessment.

[0003] However, clinical experts are aware that different modalities of data contribute significantly differently to the assessment results (e.g., cognitive scale scores are more diagnostically valuable than demographic information). However, existing methods cannot incorporate such prior knowledge into the self-attention mechanism without disrupting the internal information interaction of the model or causing gradient vanishing. As a result, the accuracy and clinical interpretability of the attention output results are difficult to meet practical needs. Summary of the Invention

[0004] In view of this, embodiments of this disclosure provide an attention calibration method, apparatus, medium, and product based on prior knowledge constraints, which can perform prior calibration and renormalization after scaling dot product attention calculation and before weighted summation of value matrices. Prior knowledge is incorporated without distorting feature interactions or losing gradients, thereby improving the accuracy and interpretability of attention output.

[0005] In a first aspect, embodiments of this disclosure provide an attention calibration method based on prior knowledge constraints, employing the following technical solution: Acquire multimodal brain health data of the target object and preprocess it into input features; The input features are divided into multiple feature groups, and a prior weight is set for each feature group. Each feature group is linearly transformed and then concatenated to generate a query matrix, a key matrix, and a value matrix. Scaled dot product attention calculation is performed on the query matrix and the key matrix to obtain the original attention weight matrix; Based on the prior weights of the feature group, a prior weight matrix of the same size as the original attention weight matrix is ​​constructed. The prior weight matrix is ​​used to calibrate the original attention weight matrix to obtain the calibrated attention weight matrix; The calibrated attention weight matrix is ​​normalized to obtain the final attention weight matrix; The value matrix is ​​weighted and summed using the final attention weight matrix to obtain the attention output result, which is used to generate brain health assessment data for the target object.

[0006] Optionally, dividing the input features into multiple feature groups and setting prior weights for each feature group includes: The input features are divided into feature groups of at least two levels according to clinical knowledge, wherein each feature group of a higher level contains at least two feature subgroups of a lower level. Based on the total number of dimensions of the input features, a target level suitable for the current attention computation is determined from the at least two levels; Configure candidate prior weight sets for multiple brain health assessment sub-scenarios; Obtain the business label of the input feature, and select the appropriate prior weight set from the multiple candidate prior weight sets based on the business label; Based on the aforementioned set of adaptation prior weights, prior weights are assigned to each feature group under the target level.

[0007] Optionally, constructing a prior weight matrix of the same size as the original attention weight matrix based on the prior weights of the feature set includes: Construct an initial matrix of the same size as the original attention weight matrix; Traverse each row of the initial matrix, determine the feature group to which the feature dimension of the current row belongs, and assign all elements of the current row to the prior weights of the feature group to which they belong. By iterating through all rows and assigning values, we obtain the prior weight matrix.

[0008] Optionally, the attention calibration method based on prior knowledge constraints further includes: Set a learnable, dynamically adjustable coefficient for each feature group; Multiply the prior weights of each feature group by the corresponding dynamic adjustment coefficient to obtain the adjusted prior weights. Construct a prior weight matrix based on the adjusted prior weights.

[0009] Optionally, calibrating the original attention weight matrix using the prior weight matrix to obtain a calibrated attention weight matrix includes: The original attention weight matrix is ​​multiplied element-wise with the prior weight matrix to obtain the calibrated attention weight matrix.

[0010] Optionally, normalizing the calibrated attention weight matrix to obtain the final attention weight matrix includes: The calibrated attention weight matrix is ​​normalized row by row so that the sum of the weights in each row is 1, thus obtaining the final attention weight matrix.

[0011] Optionally, the attention calibration method based on prior knowledge constraints further includes: When multiple attention heads are involved, each attention head independently performs attention calculation, calibration, and weighted summation of the logarithmic matrix. The outputs of each attention head are concatenated and then linearly transformed to obtain the multi-head attention output.

[0012] Secondly, this disclosure also provides an attention calibration system based on prior knowledge constraints, employing the following technical solution: The preprocessing module is used to acquire multimodal brain health data of the target object and preprocess it into input features; The feature segmentation module is used to divide the input features into multiple feature groups and set prior weights for each feature group. The linear transformation module is used to linearly transform each feature group and then concatenate them to generate a query matrix, a key matrix, and a value matrix. The attention calculation module is used to perform scaled dot product attention calculation on the query matrix and the key matrix to obtain the original attention weight matrix; The matrix construction module is used to construct a prior weight matrix of the same size as the original attention weight matrix based on the prior weights of the feature group. A matrix calibration module is used to calibrate the original attention weight matrix using the prior weight matrix to obtain a calibrated attention weight matrix. The normalization module is used to normalize the calibrated attention weight matrix to obtain the final attention weight matrix; The weighted summation module is used to sum the value matrix using the final attention weight matrix to obtain the attention output result, which is used to generate brain health assessment data for the target object.

[0013] Thirdly, this disclosure also provides a computer device, which adopts the following technical solution: The computer device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to perform any of the above-described attention calibration methods based on prior knowledge constraints.

[0014] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing computer instructions for causing a computer to execute any of the attention calibration methods based on prior knowledge constraints described above.

[0015] Fifthly, embodiments of this disclosure also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of any of the methods described above.

[0016] The attention calibration method based on prior knowledge constraints provided in this disclosure achieves targeted calibration of the attention distribution by multiplying the prior weight matrix element-wise with the original attention weight matrix and renormalizing it after scaling the dot product attention calculation and before weighted summing of the value matrix. Since the calculation of the query matrix and key matrix is ​​completely unaffected by the weights, the true correlation between features is fully preserved. Simultaneously, the renormalization after calibration strictly maintains the normalization characteristic that the sum of each row of attention weights is 1, ensuring stable output scale. The gradient of the value matrix is ​​not scaled by the business weights during backpropagation, and the gradients of all features are fully preserved, avoiding the gradient vanishing problem.

[0017] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating the attention calibration method based on prior knowledge constraints provided in this embodiment of the disclosure; Figure 2 A flowchart illustrating the feature group division and prior weight setting method provided in this embodiment of the disclosure; Figure 3 A flowchart illustrating the prior weight matrix construction method provided in this embodiment of the disclosure; Figure 4 A schematic diagram of an attention calibration system based on prior knowledge constraints provided in this embodiment of the disclosure; Figure 5This is a schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. Detailed Implementation

[0020] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0021] It should be understood that the following specific examples illustrate the implementation of this disclosure, and those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0022] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0023] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0024] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0025] Methods for injecting prior business knowledge into deep learning models in the form of weights mainly include input layer weighting, output layer weighting, and loss function weighting. Input layer weighting directly multiplies each feature by its corresponding business weight, which distorts the true correlation between features and causes gradient vanishing for low-weight features. Output layer weighting trains each feature group separately and then fuses them at the output layer, completely losing cross-feature group interaction information, and the computational cost increases linearly with the number of feature groups. Loss function weighting only affects the model during the training phase, cannot be applied to the inference process, and easily leads to overfitting of the model to high-weight features.

[0026] None of the above methods can seamlessly integrate structured clinical prior knowledge into the self-attention mechanism without disrupting the internal information interaction of the model or causing gradient vanishing. As a result, the accuracy and clinical interpretability of the attention output results are difficult to meet the needs of practical applications.

[0027] To solve the above problems, refer to Figure 1 This disclosure provides an attention calibration method based on prior knowledge constraints, comprising the following steps: S1: Acquire multimodal brain health data of the target object and preprocess it into input features; S2: Divide the input features into multiple feature groups and set prior weights for each feature group; S3: Linearly transform each feature group and then concatenate them to generate a query matrix, a key matrix, and a value matrix; S4: Perform scaling dot product attention calculation on the query matrix and the key matrix to obtain the original attention weight matrix; S5: Based on the prior weights of the feature groups, construct a prior weight matrix of the same size as the original attention weight matrix; S6: Use the prior weight matrix to calibrate the original attention weight matrix to obtain the calibrated attention weight matrix; S7: Normalize the calibrated attention weight matrix to obtain the final attention weight matrix; S8: Use the final attention weight matrix to perform a weighted summation of the value matrix to obtain the attention output result, which is used to generate brain health assessment data for the target object.

[0028] The attention calibration method based on prior knowledge constraints disclosed herein achieves targeted calibration of the attention distribution by multiplying the prior weight matrix element-wise with the original attention weight matrix and renormalizing it after scaling the dot product attention calculation and before weighted summing of the value matrix. Since the calculation of the query matrix and key matrix is ​​completely unaffected by the weights, the true correlation between features is fully preserved. Furthermore, the renormalization after calibration strictly maintains the normalization property that the sum of each row of attention weights is 1, ensuring stable output scale. The gradient of the value matrix is ​​not scaled by the business weights during backpropagation, and the gradients of all features are fully preserved, avoiding the gradient vanishing problem.

[0029] Compared to existing methods such as input layer weighting, output layer weighting, and loss function weighting, this scheme neither distorts the interaction information between features nor adds extra computational load during the calibration process. While preserving the data-driven global context modeling capability, it effectively incorporates expert prior knowledge. The final attention output results are improved in both accuracy and interpretability, providing a more reliable feature representation for subsequent brain health assessments.

[0030] In S1, multimodal raw data on the target object's brain health are acquired. This multimodal data includes, but is not limited to, blood biochemical indicators, cognitive scale scores, physiological signal data, and quantitative brain imaging features. The multimodal raw data undergoes preprocessing, including data cleaning, missing value imputation, numerical encoding, and normalization. Each modality's data is uniformly converted into numerical feature vectors, and these feature vectors are concatenated in a preset order to form complete input features. Each modality corresponds to a set of consecutively arranged feature dimensions, with each feature dimension within the same modality being adjacent in the sequence.

[0031] In S2, refer to Figure 2 The flowchart illustrating the feature grouping and prior weight setting method shows that "dividing the input features into multiple feature groups and setting prior weights for each feature group" includes the following steps: S21: Divide the input features into feature groups of at least two levels according to clinical knowledge, wherein the upper level feature group contains at least two lower level feature subgroups; S22: Based on the total number of dimensions of the input features, determine the target level that is suitable for the current attention computation from at least two levels; S23: Configure a set of candidate prior weights for multiple brain health assessment sub-scenarios; S24: Obtain the business labels of the input features, and select the appropriate prior weight set from multiple candidate prior weight sets based on the business labels; S25: Assign prior weights to each feature group under the target level based on the adapted prior weight set.

[0032] In S21, based on clinical knowledge, the input features are divided into at least two levels of feature groups, from coarse to fine. The upper level is a coarse-grained large group, and the lower level is a fine-grained small group or individual feature. For example, there are three levels: large group, small group, and individual feature. Each level contains multiple feature groups, and each feature group belongs to only one upper-level feature group. They do not overlap. The number of levels and the division method are predefined by clinical experts based on the actual needs of brain health assessment.

[0033] In S22, the total number of dimensions of the input features determines the size of the attention weight matrix. When the feature dimensions are small, coarse-grained, large-group weights are sufficient to effectively represent prior knowledge. When the feature dimensions are large, finer-grained levels are required. This is because features within the same level share the same weight value. The finer the level, the more rows and blocks in the matrix that take independent values, resulting in a more refined representation of the differences for each feature. Therefore, the total number of dimensions of the input features is the core determining factor in deciding which level of prior weights to use.

[0034] In S23, each candidate prior weight set contains weight values ​​for all feature groups under the target level. The weight values ​​for each feature group are pre-assigned by clinical experts based on the importance of each modality feature to the evaluation result in that sub-scenario. The weight values ​​for the same feature group are different in different sub-scenarios. All candidate sets share the same feature group structure, that is, the group division method is the same, only the weight values ​​corresponding to each group vary from scenario to scenario.

[0035] In S24, the business tag is used to identify the sub-scenario type to which the current evaluation belongs, such as the examination application type or clinical diagnosis direction field from the electronic medical record. Based on the business tag, a scenario-appropriate candidate prior weight set is selected from multiple candidate prior weight sets.

[0036] In step S25, the weight values ​​corresponding to each feature group in the selected set of adaptive prior weights are assigned to the respective feature groups, thus completing the prior weight setting for each feature group. For example, if the total dimension of the input features is 50, and the target level is determined to be the first-level large group level, which includes four feature groups: blood, cognition, physiology, and imaging, and the target object's business label is early screening for cognitive impairment, then an early screening scenario set is selected. The weights of the four large groups in this set are 0.10, 0.60, 0.10, and 0.20, respectively. Based on this, the prior weight of the cognitive scale group is set to 0.60, the brain imaging group to 0.20, and the blood and physiology groups to 0.10 each, thus completing the prior weight assignment for each feature group.

[0037] In S3, each feature group undergoes a linear transformation, which involves multiplying the original feature vector by three independent learnable weight matrices to obtain the query submatrix, key submatrix, and value submatrix for each feature group. After all feature groups have undergone their respective linear transformations, all query submatrices are concatenated along the sequence dimension to obtain the complete query matrix Q; similarly, all key matrices are concatenated to obtain the complete key matrix K, and all value submatrices are concatenated to obtain the complete value matrix V. The number of rows in the concatenated Q, K, and V is equal to the sum of the feature dimensions of all feature groups, i.e., the total dimension of the input features.

[0038] In S4, the dot product of the query matrix Q and the transpose of the key matrix K is calculated to obtain the similarity matrix. The element in the i-th row and j-th column of the similarity matrix represents the original relevance score of the i-th feature position to the j-th feature position. The similarity matrix is ​​scaled by dividing by the square root of the feature dimension to control the variance of the dot product result and prevent the Softmax function from entering the gradient saturation region. The scaled similarity matrix is ​​then Softmax normalized row by row so that the weight values ​​of each row are between 0 and 1 and the sum of all elements in that row is 1, resulting in the original attention weight matrix, which preserves the true relevance between all feature positions. The original attention weight matrix is ​​a D×D square matrix, where the element in the i-th row and j-th column represents the attention weight of the i-th feature position to the j-th feature position, and D is the total number of dimensions of the input features.

[0039] The formula for calculating the similarity matrix is ​​as follows: In the formula, express; Indicates the transpose operation; The feature dimensions represent the query vector and key vector.

[0040] The formula for calculating the original attention weight matrix is ​​as follows: In the formula, This represents the original attention weight matrix; The flexible maximum normalization is a normalization function that maps a real number vector to a probability distribution vector.

[0041] In S5, refer to Figure 3 The flowchart illustrating the prior weight matrix construction method shows that "constructing a prior weight matrix of the same size as the original attention weight matrix based on the prior weights of feature groups" includes the following steps: S51: Construct an initial matrix of the same size as the original attention weight matrix; S52: Traverse each row of the initial matrix, determine the feature group to which the feature dimension of the current row belongs, and assign all elements of the current row to the prior weights of the feature group to which they belong. S53: Traverse all rows to complete the assignment and obtain the prior weight matrix.

[0042] In S51, since the original attention weight matrix is ​​a square matrix with the number of rows and columns equal to the length of the input feature sequence (i.e., the total number of dimensions D of the input features), an initial D×D matrix is ​​constructed. All elements of this matrix are initially empty or zero, and are used to fill in the prior weight values ​​of each feature group later.

[0043] In steps S52 and S53, each row of the initial matrix is ​​traversed. For the i-th row (corresponding to the i-th feature dimension in the input feature sequence), the prior weight value set for that feature group is found based on the feature group to which the feature dimension belongs. All 96 elements in the i-th row are then assigned this prior weight value. After traversing all rows (i=1,2,...,96) and completing the assignment, the complete prior weight matrix is ​​obtained. Rows belonging to the same feature group are assigned the same prior weight value, while rows from different feature groups are assigned their respective corresponding prior weight values.

[0044] Assume the input feature sequence is 6 units long and belongs to the cognitive scale group, brain imaging group, and gene index group, respectively, with prior weights of 0.50, 0.30, and 0.15. Construct a 6×6 initial matrix, where all elements in rows 1-2 are set to 0.50, all elements in rows 3-4 are set to 0.30, and all elements in rows 5-6 are set to 0.15, thus obtaining the prior weight matrix.

[0045] Optionally, a learnable, dynamically adjusted coefficient β is assigned to each feature group. i The initial value of this coefficient is set to 1.0, and it is optimized synchronously during the model training process as a trainable parameter. The prior weights w of each feature group are... prior (i) Multiply by the corresponding dynamic adjustment coefficient β i The adjusted final weight w is obtained. final (i)=w prior (i)·β i This enables the model to make small-scale autonomous adjustments based on expert prior knowledge.

[0046] To ensure the dominance of prior business knowledge, a regularization term L is added to the loss function during model training. reg =λ·Σ(β i -1.0)², where λ is the regularization coefficient used to control the adjustment magnitude. This regularization term applies to β deviating from 1.0. i Apply a squared penalty during training, β iConstrained within the range of 0.8 to 1.2, allowing ±20% data-driven fine-tuning based on prior weights, this approach retains the dominance of expert knowledge while also taking into account individual differences in data distribution. The larger the λ value, the better the β... i The higher the cost of deviating from 1.0, the stronger the constraint of the prior weights.

[0047] In S6, after the scaling dot product attention calculation and before the weighted summation of the logarithmic matrices, the original attention weight matrix is ​​multiplied element-wise with the prior weight matrix to obtain the calibrated attention weight matrix. Because each element in the prior weight matrix represents the importance of the corresponding feature group in the business context, multiplying it element-wise with the original attention weights amplifies the attention given to important feature groups and suppresses the attention given to less important feature groups, thus achieving a targeted adjustment of the attention distribution. The formula for calculating the calibrated attention weight matrix is ​​as follows: In the formula, This represents the calibrated attention weight matrix; This represents the prior weight matrix.

[0048] In S7, the element-wise multiplication during calibration disrupts the normalization property of the original attention weight matrix, where the sum of each row equals 1. This causes the sum of each row to become the prior weight value (not 1) of the corresponding feature group. Therefore, the calibrated attention weight matrix is ​​normalized row by row. This re-normalization restores the mathematical form of the probability distribution, ensuring that the sum of the weights in each row equals 1, resulting in the final attention weight matrix. Otherwise, the scale stability of the output features cannot be maintained during subsequent weighted summation. The formula for calculating the final attention weight matrix is ​​as follows: In the formula, This represents the final attention weight matrix, where the element in the i-th row and j-th column represents the attention weight of the i-th feature position after renormalization to the j-th feature position. This means summing each row of the calibrated attention weight matrix, resulting in a column vector where the i-th element represents the sum of all elements in the i-th row.

[0049] In S8, when there is only one attention head, the attention output obtained by weighted summation of the final attention weight matrix and the value matrix is ​​the output of the entire attention layer. When there are multiple attention heads, each attention head independently performs attention calculation, calibration, and weighted summation of the value matrix based on the predefined feature groups, obtaining the attention output of each attention head. The outputs of each attention head are concatenated and then linearly transformed to obtain the multi-head attention output. This is because the output dimensions of each head are the same before concatenation, so the linear transformation layer maps the concatenated high-dimensional vector back to the original feature dimensions, ensuring that subsequent network layers can receive it correctly.

[0050] Reference Figure 4 This disclosure provides an attention calibration system based on prior knowledge constraints, comprising: Preprocessing module 101 is used to acquire multimodal brain health data of the target object and preprocess it into input features; The feature segmentation module 102 is used to divide the input features into multiple feature groups and set prior weights for each feature group. The linear transformation module 103 is used to linearly transform each feature group and then concatenate them to generate a query matrix, a key matrix, and a value matrix. Attention calculation module 104 is used to perform scaled dot product attention calculation on the query matrix and the key matrix to obtain the original attention weight matrix; The matrix construction module 105 is used to construct a prior weight matrix of the same size as the original attention weight matrix based on the prior weights of the feature group. The matrix calibration module 106 is used to calibrate the original attention weight matrix using the prior weight matrix to obtain the calibrated attention weight matrix. Normalization module 107 is used to normalize the calibrated attention weight matrix to obtain the final attention weight matrix; The weighted summation module 108 is used to perform weighted summation of the value matrix using the final attention weight matrix to obtain the attention output result, which is used to generate brain health assessment data for the target object.

[0051] The various variations and specific examples of the attention calibration method based on prior knowledge constraints provided above are also applicable to the attention calibration system based on prior knowledge constraints provided in this disclosure. Through the foregoing detailed description of the attention calibration method based on prior knowledge constraints, those skilled in the art can clearly understand the implementation method of the attention calibration system based on prior knowledge constraints. For the sake of brevity, it will not be described in detail here.

[0052] A computer device according to embodiments of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0053] The processor may be a central processing unit (CPU) or other processing unit with data processing and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of this disclosure, the processor is used to execute computer-readable instructions stored in the memory, causing the computer device to perform all or part of the steps of the attention calibration method based on prior knowledge constraints described in the foregoing embodiments of this disclosure.

[0054] Those skilled in the art will understand that, in order to solve the technical problem of how to achieve a good user experience, this embodiment may also include well-known structures such as communication buses and interfaces, and these well-known structures should also be included within the protection scope of this disclosure.

[0055] like Figure 5 This is a schematic diagram of a computer device provided for an embodiment of the present disclosure. It illustrates a structural schematic diagram suitable for implementing the computer device in the embodiments of the present disclosure. Figure 5 The computer device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0056] like Figure 5 As shown, a computer device may include a processor (such as a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) or programs loaded from storage devices into random access memory (RAM). The RAM also stores various programs and data required for the operation of the computer device. The processor, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0057] Typically, the following devices can be connected to the I / O interface: input devices, such as sensors or visual information acquisition devices; output devices, such as displays; storage devices, such as magnetic tapes or hard drives; and communication devices. Communication devices allow the computer device to communicate wirelessly or wiredly with other devices (such as edge computing devices) to exchange data. Although Figure 5A computer apparatus with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or included alternatively.

[0058] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from ROM. When the computer program is executed by a processor, all or part of the steps of the attention calibration method based on prior knowledge constraints according to embodiments of this disclosure are performed.

[0059] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0060] A computer-readable storage medium according to embodiments of the present disclosure stores non-transitory computer-readable instructions. When these non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the attention calibration method based on prior knowledge constraints described in the foregoing embodiments of the present disclosure are performed.

[0061] The aforementioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or portable hard drive), media with built-in rewritable non-volatile memory (e.g., memory card), and media with built-in ROM (e.g., ROM cartridge).

[0062] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0063] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0064] In this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0065] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0066] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0067] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0068] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0069] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. An attention calibration method based on prior knowledge constraints, characterized in that, include: Acquire multimodal brain health data of the target object and preprocess it into input features; The input features are divided into multiple feature groups, and a prior weight is set for each feature group. Each feature group is linearly transformed and then concatenated to generate a query matrix, a key matrix, and a value matrix. Scaled dot product attention calculation is performed on the query matrix and the key matrix to obtain the original attention weight matrix; Based on the prior weights of the feature group, a prior weight matrix of the same size as the original attention weight matrix is ​​constructed. The prior weight matrix is ​​used to calibrate the original attention weight matrix to obtain the calibrated attention weight matrix; The calibrated attention weight matrix is ​​normalized to obtain the final attention weight matrix; The value matrix is ​​weighted and summed using the final attention weight matrix to obtain the attention output result, which is used to generate brain health assessment data for the target object.

2. The attention calibration method based on prior knowledge constraints according to claim 1, characterized in that, The step of dividing the input features into multiple feature groups and setting prior weights for each feature group includes: The input features are divided into feature groups of at least two levels according to clinical knowledge, wherein each feature group of a higher level contains at least two feature subgroups of a lower level. Based on the total number of dimensions of the input features, a target level suitable for the current attention computation is determined from the at least two levels; Configure candidate prior weight sets for multiple brain health assessment sub-scenarios; Obtain the business label of the input feature, and select the appropriate prior weight set from the multiple candidate prior weight sets based on the business label; Based on the aforementioned set of adaptation prior weights, prior weights are assigned to each feature group under the target level.

3. The attention calibration method based on prior knowledge constraints according to claim 2, characterized in that, The construction of a prior weight matrix of the same size as the original attention weight matrix based on the prior weights of the feature set includes: Construct an initial matrix of the same size as the original attention weight matrix; Traverse each row of the initial matrix, determine the feature group to which the feature dimension of the current row belongs, and assign all elements of the current row to the prior weights of the feature group to which they belong. By iterating through all rows and assigning values, we obtain the prior weight matrix.

4. The attention calibration method based on prior knowledge constraints according to claim 2, characterized in that, Also includes: Set a learnable, dynamically adjustable coefficient for each feature group; Multiply the prior weights of each feature group by the corresponding dynamic adjustment coefficient to obtain the adjusted prior weights. Construct a prior weight matrix based on the adjusted prior weights.

5. The attention calibration method based on prior knowledge constraints according to claim 1, characterized in that, The step of calibrating the original attention weight matrix using the prior weight matrix to obtain the calibrated attention weight matrix includes: The original attention weight matrix is ​​multiplied element-wise with the prior weight matrix to obtain the calibrated attention weight matrix.

6. The attention calibration method based on prior knowledge constraints according to claim 1, characterized in that, The normalization of the calibrated attention weight matrix to obtain the final attention weight matrix includes: The calibrated attention weight matrix is ​​normalized row by row so that the sum of the weights in each row is 1, thus obtaining the final attention weight matrix.

7. The attention calibration method based on prior knowledge constraints according to claim 1, characterized in that, Also includes: When multiple attention heads are involved, each attention head independently performs attention calculation, calibration, and weighted summation of the logarithmic matrix. The outputs of each attention head are concatenated and then linearly transformed to obtain the multi-head attention output.

8. A computer device, characterized in that, The computer device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the attention calibration method based on prior knowledge constraints as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the attention calibration method based on prior knowledge constraints as described in any one of claims 1-7.

10. A computer program product comprising computer instructions, characterized in that, When executed by a processor, the computer instructions implement the steps of the attention calibration method based on prior knowledge constraints as described in any one of claims 1-7.

Images