Expert knowledge driven large model customized data processing method and related equipment

Abstract

The application provides an expert knowledge driven large model customization data processing method and related equipment. The method comprises: inputting acquired business data into a large model trained with a dynamically customizable expert submodel; in the large model, determining a knowledge embedding vector based on the business data; determining the weight of each adapter expert based on the knowledge embedding vector, customizing the expert submodel; and processing the business data based on the expert submodel. The embodiment of the application constructs a business knowledge hierarchy based on large-scale business data and uses multi-dimensional business information and expert knowledge. In combination with large model efficient fine-tuning technology and mixed expert strategy, dynamic expert combination adaptation is realized under the guidance of business data knowledge, so that different expert models are customized for different business data, automatic adaptation in multiple scenarios is realized, and more professional data processing is realized.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an expert knowledge-driven method for customized data processing of large models and related equipment. Background Technology

[0002] Current intelligent analytics is evolving towards multi-task capabilities. Traditional single-task models are gradually being replaced by generalist large models, which possess the potential to handle cross-domain, cross-modal, and cross-task tasks. However, existing technologies still face a contradiction between versatility and specialization in practical applications. Generalist large models have strong generalization capabilities, but their accuracy on specific tasks may be lower than that of specialized models. This is due to semantic conflicts between tasks during multi-task mixed training; conflicting tasks interfere with each other during training, resulting in negative performance transfer. Summary of the Invention

[0003] In view of this, the purpose of this application is to propose an expert knowledge-driven method for customized data processing of large models and related equipment.

[0004] To achieve the above objectives, this application provides an expert knowledge-driven method for customized data processing of large models, comprising:

[0005] The acquired business data is input into a large model that has been trained and can dynamically customize expert sub-models;

[0006] In the large model with dynamically customizable expert sub-models, a knowledge embedding vector is determined based on the business data; based on the knowledge embedding vector, the weight of each adapter expert is determined, and the expert sub-model is customized.

[0007] The business data is processed based on the expert sub-model.

[0008] In one possible implementation, the training process of the large model with the trained, dynamically customizable expert sub-models includes:

[0009] A knowledge tree is constructed based on the big data of the corresponding business to obtain the data hierarchical structure;

[0010] A context embedding prior is constructed based on the data hierarchy; in the context embedding prior, each data sample corresponds to a set of learnable context embedding vectors to represent the knowledge prior of the sample.

[0011] Construct a hybrid adapter expert network; the hybrid adapter expert network includes a gating network and at least two adapter experts;

[0012] The context embedding prior and the hybrid adapter expert network are embedded into the encoder of the large model to obtain the large model to be trained;

[0013] Based on the results of the large model to be trained and the corresponding labels of the big data, the large model to be trained is trained to obtain the trained large model with dynamically customizable expert sub-models.

[0014] In one possible implementation, the construction of a knowledge tree based on big data related to the corresponding business to obtain a data hierarchy includes:

[0015] Terminology is extracted from the acquired big data to obtain entity nodes;

[0016] The entity nodes are divided into a hierarchical structure;

[0017] Based on the hierarchical structure, data samples in the big data are mapped to each leaf node to construct the knowledge tree and obtain the data hierarchical structure.

[0018] In one possible implementation, determining the knowledge embedding vector based on the business data includes:

[0019] Based on the tags corresponding to the business data, determine the node position of the business data in the data hierarchy;

[0020] Based on the node location, the knowledge embedding vector of the business data is determined.

[0021] In one possible implementation, determining the weights of each adapter expert based on the knowledge embedding vector and customizing the expert sub-model includes:

[0022] The data is characterized to obtain a data representation;

[0023] Based on the data representation and the knowledge embedding vector, a gating vector is obtained;

[0024] The weights of each adapter expert are determined based on the gating vector, and an expert sub-model is customized.

[0025] In one possible implementation, the processing of the business data based on the expert sub-model is performed using the following formula:

[0026]

[0027] in, Indicates the processing result. Indicates a feedforward network. Indicates the first An adapter expert, express The first in The entry represents the first entry. An adapter expert Contribution To represent the data, the gate vector is defined as follows: .

[0028] Based on the same inventive concept, embodiments of this application also provide an expert knowledge-driven large-model customized data processing device, including:

[0029] The input module is configured to input acquired business data into a large model that has been trained and can dynamically customize expert sub-models.

[0030] The large model module with dynamically customizable expert sub-models is configured to determine a knowledge embedding vector based on the business data within the large model with dynamically customizable expert sub-models; and to determine the weight of each adapter expert based on the knowledge embedding vector, and customize the corresponding expert sub-model.

[0031] The processing module is configured to process the business data based on the expert sub-model.

[0032] Based on the same inventive concept, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the expert knowledge-driven large model customized data processing method as described in any of the above.

[0033] Based on the same inventive concept, embodiments of this application also provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute any of the expert knowledge-driven large model customized data processing methods described above.

[0034] Based on the same inventive concept, embodiments of this application also provide a computer program product, which includes computer program instructions, the computer instructions being used to cause the computer program product to execute any of the expert knowledge-driven large model customized data processing methods described above.

[0035] As described above, the expert knowledge-driven large-scale model customized data processing method and related equipment provided in this application input acquired business data into a trained large model with dynamically customizable expert sub-models. Within this large model, a knowledge embedding vector is determined based on the business data. Based on the knowledge embedding vector, the weight of each adapter expert is determined, and an expert sub-model is customized. The business data is then processed based on the expert sub-model, dynamically customizing a dedicated model for different business scenarios. This approach balances the generalization and specialization capabilities of the large model within business scenarios, enabling a single model to solve multiple business problems and providing business availability across various scenarios. Specifically, this solution is based on large-scale business data, utilizing multi-dimensional business information and expert knowledge to construct a hierarchical structure of business knowledge. By combining efficient large-scale model fine-tuning techniques and hybrid expert strategies, dynamic expert group matching is achieved under the guidance of business data knowledge, thereby customizing different expert sub-models for different business data, achieving automatic adaptation across multiple scenarios, and realizing more professional data processing. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a schematic diagram of the expert knowledge-driven large model customized data processing method according to an embodiment of this application.

[0038] Figure 2 This is a schematic diagram of the large model structure of the dynamically customizable expert sub-model in an embodiment of this application.

[0039] Figure 3 This is a schematic diagram illustrating the construction process of the medical knowledge hierarchy structure in an embodiment of this application;

[0040] Figure 4 This is a schematic diagram of a hierarchical context embedding prior according to an embodiment of this application;

[0041] Figure 5 This is a schematic diagram of the structure of an expert knowledge-driven large model customized data processing device according to an embodiment of this application.

[0042] Figure 6 This is a schematic diagram of the electronic device structure according to an embodiment of this application. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0044] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0045] It is understood that before using the technical solutions of the various embodiments in this disclosure, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0046] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations of this disclosed technical solution.

[0047] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0048] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0049] As described in the background section, current intelligent analytics is evolving towards multi-task capabilities. Traditional single-task models are gradually being replaced by generalist large models, which possess the potential to handle cross-domain, cross-modal, and cross-task tasks. However, existing technologies still face a contradiction between versatility and specialization in practical applications. Generalist large models have strong generalization capabilities, but their accuracy on specific tasks may be lower than that of specialized models. This is due to semantic conflicts between tasks during multi-task mixed training; conflicting tasks interfere with each other during training, resulting in negative performance transfer.

[0050] Based on the above considerations, this application proposes an expert knowledge-driven large-scale model customized data processing method. This method involves inputting acquired business data into a pre-trained large-scale model with dynamically customizable expert sub-models. Within this large-scale model, a knowledge embedding vector is determined based on the business data. Based on the knowledge embedding vector, the weight of each adapter expert is determined, and an expert sub-model is customized. The business data is then processed using these expert sub-models to dynamically customize specialized models for different business scenarios. This approach balances the generalization and specialization capabilities of the large-scale model within business scenarios, enabling a single model to solve multiple business problems and providing business availability across various scenarios. Specifically, this solution utilizes large-scale business data and multi-dimensional business information and expert knowledge to construct a hierarchical structure of business knowledge. By combining efficient large-scale model fine-tuning techniques and hybrid expert strategies, dynamic expert group matching is achieved under the guidance of business data knowledge. This allows for the customization of different expert sub-models for different business data, achieving automatic adaptation across multiple scenarios and enabling more professional data processing.

[0051] The technical solutions of the embodiments of this application will be described in detail below through specific examples.

[0052] refer to Figure 1 The expert knowledge-driven large model customized data processing method of this application includes the following steps:

[0053] Step S101: Input the acquired business data into the trained, dynamically customizable expert sub-model;

[0054] Step S102: In the large model with dynamically customizable expert sub-models, determine the knowledge embedding vector based on the business data; determine the weight of each adapter expert based on the knowledge embedding vector, and customize the expert sub-model.

[0055] Step S103: Process the business data based on the expert sub-model.

[0056] refer to Figure 2 This is a schematic diagram of the large model structure of the dynamically customizable expert sub-model in an embodiment of this application.

[0057] In the embodiments of this application, taking the visual large model SAM (Segment AnythingModel) in a general segmentation scenario as an example, it demonstrates how to dynamically customize medical expert sub-models for tasks of different modalities and organs in a medical scenario, thereby ensuring the specialized capabilities of the large model in different clinical scenarios. Specifically, the above problem can be solved by constructing a Contextual Mixture of Adapter Experts (ConMoAE) module. ConMoAE consists of a hierarchical structure-based contextual embedding prior (HSCEP) and a context-guided mixture of adapters network (CMoAN). Figure 2 As shown, by integrating ConMoAE into the feed-forward network (FFN) of each Transformer layer in the SAM image encoder, contextual medical knowledge can be introduced into the large model, and separate parameters can be provided for conflicting tasks. Semantically relevant sub-models can be generated for different tasks to mitigate the "negative transfer" phenomenon, thereby ensuring the specialization capability in medical scenarios.

[0058] In some embodiments, for step S101, the training process of the large model with the trained dynamically customizable expert sub-model includes: constructing a knowledge tree based on big data of the corresponding business to obtain a data hierarchical structure; constructing a context embedding prior based on the data hierarchical structure; in the context embedding prior, each data sample corresponds to a set of learnable context embedding vectors to represent the knowledge prior of the sample; constructing a hybrid adapter expert network; the hybrid adapter expert network includes a gating network and at least two adapter experts; embedding the context embedding prior and the hybrid adapter expert network into the encoder of the large model to obtain the large model to be trained; training the large model to be trained based on the results of the large model to be trained inputting the big data and the labels corresponding to the big data to obtain the large model with the trained dynamically customizable expert sub-model.

[0059] In some embodiments, constructing a knowledge tree based on big data for the corresponding business to obtain a data hierarchy includes: extracting terms from the acquired big data to obtain entity nodes; dividing the entity nodes into a hierarchical structure; and mapping data samples from the big data to each leaf node based on the hierarchical structure to construct the knowledge tree and obtain the data hierarchy.

[0060] In this embodiment, the first step is to construct a hierarchical structure of medical knowledge by combining a medical ontology knowledge base and expert annotations. This hierarchical structure can not only systematically organize large-scale medical datasets, but also provide knowledge guidance for large models, thereby ensuring specialized capabilities in different medical scenarios.

[0061] refer to Figure 3 This is a schematic diagram illustrating the construction process of the medical knowledge hierarchy structure in an embodiment of this application.

[0062] like Figure 3 As shown, taking a general medical segmentation scenario as an example, this hierarchical structure can be constructed based on RadLex, a publicly available terminology dictionary developed by the Radiological Society of North America (RSNA). Using RadLex identifiers (RIDs), anatomical-related terms from the collected medical segmentation dataset are mapped to the RadLex vocabulary, and relevant terms are extracted as entity nodes. With the help of imaging experts, these terms are integrated into the medical knowledge hierarchy based on the "Has_part" and "Contains" relationships. This allows the construction of an anatomical knowledge tree, MedHierarchy, encompassing the human body, body parts, body sub-regions, and organs and tissues. By fully utilizing this constructed medical knowledge hierarchy, medical data can be systematically organized. Large-scale medical data can be allocated to corresponding leaf nodes in the hierarchy according to different medical tasks, thus dynamically invoking relevant anatomical knowledge for different data. Under each leaf node, data samples for the same task are grouped together, resulting in a positive transfer effect during model training.

[0063] Specifically, common anatomical terms (such as human organs, tissues, and parts) in medical segmentation tasks can be associated with the RadLex vocabulary through annotation and mapping methods.

[0064] Design knowledge nodes using RadLex's logical relationships (such as the hierarchical structure of parts). For example:

[0065] Top-level node (Root): Human Body.

[0066] Child nodes: Body Parts (body parts, such as head, neck, torso, etc.).

[0067] Secondary child nodes: Body Sub-regions (such as the head, abdomen, etc.).

[0068] Third-level child nodes: Organs and Tissues (organs and tissues, such as heart, liver, lungs, etc.).

[0069] Entity relationship annotation:

[0070] With the help of imaging experts, the relationships between different terms were analyzed, for example:

[0071] Torso (trunk) Has_part abdomen (abdomen)

[0072] The head contains the pituitary gland. Based on these relationships, a complete anatomical knowledge tree, MedHierarchy, is constructed.

[0073] Dynamic mapping and extension:

[0074] For continuously updated medical datasets, new anatomical terms are dynamically mapped to existing knowledge structures. For example, when introducing new segmented datasets involving terms not yet included, the knowledge tree can be expanded based on RadLex or other authoritative sources.

[0075] Furthermore, after obtaining the hierarchical structure of medical knowledge, a Hierarchical Structure-based Contextual Embedding Prior (HSCEP) can be constructed based on the hierarchical structure, thereby providing medical knowledge dynamically for large models.

[0076] refer to Figure 4 This is a schematic diagram of the hierarchical context embedding prior in an embodiment of this application.

[0077] Specifically, HScEP explicitly maps knowledge in the MedHierarchy to context embedding vectors:

[0078]

[0079] In some embodiments, such as Figure 4 As shown, HScEP can be composed of five layers of embedded vector points. The highest-level anatomical embedding prior represents the human body in the MedHierarchy. and The anatomical embedding priors for the second and third layers represent body parts and body sub-regions in the MedHierarchy, respectively. This represents the embedding priors of specific organs and tissues in the fourth layer of MedHierarchy. The fifth layer of embedding priors represents the task to which the data sample belongs. Each embedding point consists of a learnable vector.

[0080] Given a data sample, HScEP will assign a set of context embedding vectors to each data sample based on its position in the MedHierarchy. This set of context embedding vectors assigns different prior knowledge to different tasks, decoupling irrelevant tasks and thus avoiding task conflicts during multi-task training.

[0081] Specifically, the core design of HScEP is based on a knowledge hierarchy (MedHierarchy), which is mapped layer by layer to context embedding vectors, including five layers of embedding points:

[0082] Top-level embedding ( : Human Body.

[0083] It represents the highest level of anatomical knowledge, defining the embedded priors of the entire human body as a whole.

[0084] It captures general medical background knowledge for the model from a holistic perspective, such as the global structure of the human body and common sense of anatomy.

[0085] Second layer embedding ( : Body Parts.

[0086] This represents the embedded priors of the main parts of the human body (such as head and neck, trunk, bones, blood vessels, and skin).

[0087] It provides the model with knowledge about different body parts, such as the morphological and functional differences in specific areas of the head, neck, and trunk.

[0088] Third layer embedding ( : Body Sub-regions.

[0089] Capture more fine-grained regions of specific body parts, such as the chest and abdomen in the torso.

[0090] It helps the model identify more specific medical structures, thereby enhancing the understanding of the features within the region.

[0091] Fourth layer embedding ( : Organs and tissues.

[0092] This represents the embedded priors of specific organs (such as the heart, liver, and lungs) and their tissues.

[0093] This layer is directly related to the segmentation task of medical data, providing the model with knowledge guidance about specific anatomical regions.

[0094] Fifth layer embedding ( : Specific task (Task).

[0095] This represents the specific task to which the data sample belongs, such as the embedding prior for tasks like lung segmentation or skull segmentation.

[0096] It explicitly encodes the goal of a task as an embedded prior, thus avoiding conflicts between different tasks.

[0097] In one feasible implementation, the corresponding node in the MedHierarchy is found by using the anatomical location label of the data sample (e.g., "Lungs" for lung). For example:

[0098] If a sample is labeled "Lungs", its position in the knowledge tree is: human body -> Torso -> Chest -> Lung -> Lung Segmentation.

[0099] Assignment of context embedding vectors:

[0100] Assign a corresponding context embedding vector based on the sample's position in the knowledge tree.

[0101] For example, for samples related to "lungs", the assigned context embedding vector is:

[0102] in:

[0103] : Global embedding of the human body.

[0104] : The part of the torso (Torso) is embedded.

[0105] : Subregional embedding of the chest.

[0106] The lung is an organ embedded within the lung.

[0107] Task embedding for the "Lung Segmentation" task.

[0108] During training, the model dynamically calls the corresponding context embedding vector for each sample input. This is then combined with the features of the input image. Embedded vector It provides prior information on medical knowledge and task objectives, making it easier for the model to capture anatomical semantics.

[0109] By constructing and applying HScEP, the MedHierarchy of medical knowledge can be explicitly mapped into contextual embedding vectors, providing the model with dynamic, task-related prior knowledge. Taking lung segmentation as an example, HScEP's five-layer embedding vectors can capture hierarchical knowledge from the global human body to the local lungs, and, combined with the task objective, clarify the model's learning direction. This method is not only applicable to medical image segmentation scenarios but can also be extended to other business scenarios requiring multi-task collaboration or knowledge injection, thereby providing artificial intelligence with stronger knowledge representation capabilities and generalization.

[0110] Furthermore, after obtaining the context embedding prior HScEP, a context-guided Mixture of Adapters Network (CMoAN) can be constructed. This network consists of several adapter experts and a gating network. Given a data sample, the gating network, guided by the contextual knowledge prior provided by HScEP, dynamically assigns different weights to each adapter expert, thereby constructing different expert combinations to serve different tasks.

[0111] Specifically, each adapter expert consists of a bottleneck structure. (bottleneck structure), a downsampling layer (down-projection layer), an upsampling layer It consists of an up-projection layer and a ReLU (non-linear) activation layer. Given a data representation The adapted features can be represented as .

[0112] CMoAN is represented by data. and embedded priors As input, guide the gating network to determine the expertise of each adapter for the first... The contribution weight of each data sample. The number of adapter experts is defined as... The gate vector is defined as Given a data representation and context-embedded priors The gating vector can be obtained through the following calculation:

[0113]

[0114] in, For the concatenation operator, This represents a gated network consisting of a linear layer, with the output... Each score represents the contribution weight of each adapter expert. In summary, the final output of CMoAN... The following can be calculated:

[0115]

[0116] in, Represents a feedforward network. For the first An adapter expert, yes The first in The entry represents the first entry. An adapter expert . contributions.

[0117] In a feasible embodiment, still using the chest CT image mentioned above as an example, after determining the embedding vector, the data is represented... and context-embedded priors The data is concatenated, and the weight of each adapter expert is calculated through a gating network. Assuming there are K=3 adapter experts, the output weight G=[0.2,0.5,0.3] indicates that the second adapter expert contributes the most to the current task. The final output is the combined feature of each adapter expert after weighting.

[0118] The constructed context embedding prior and hybrid adapter expert network are then embedded into the encoder of the large model to obtain the large model to be trained. Further, based on the results of the large model to be trained and the corresponding labels of the big data input, the large model to be trained is trained to obtain a trained large model with dynamically customizable expert sub-models. The specific training process will not be described in detail here.

[0119] After the model training is completed, in step S101, the acquired business data is input into the trained large model with dynamically customizable expert sub-models. Further, in steps S102 and S103, in the large model with dynamically customizable expert sub-models, a knowledge embedding vector is determined based on the business data; the weight of each adapter expert is determined based on the knowledge embedding vector, and the expert sub-model is customized; the business data is processed based on the expert sub-model.

[0120] In some embodiments, determining the knowledge embedding vector based on the business data includes: determining the node position of the business data in the data hierarchy according to the tag corresponding to the business data; and determining the knowledge embedding vector of the business data based on the node position.

[0121] In some embodiments, determining the weight of each adapter expert and customizing the expert sub-model based on the knowledge embedding vector includes: performing feature representation on the data to obtain a data representation; obtaining a gating vector based on the data representation and the knowledge embedding vector; determining the weight of each adapter expert based on the gating vector and customizing the expert sub-model.

[0122] In some embodiments, the processing of the business data based on the expert sub-model is performed using the following formula:

[0123]

[0124] in, Indicates the processing result. Indicates a feedforward network. Indicates the first An adapter expert, express The first in The entry represents the first entry. An adapter expert Contribution To represent the data, the gate vector is defined as follows: .

[0125] The following explanation uses lung CT lesion segmentation as an example to illustrate the above steps.

[0126] First, acquire CT images:

[0127] Obtain a set of lung CT images to be analyzed from a hospital or medical imaging database (such as a PACS system). These images may be in DICOM format. Ensure the quantity and quality of the images are suitable for the analysis, and avoid using low-quality or blurry images.

[0128] Subsequent image preprocessing:

[0129] Denoising: Use filtering algorithms (such as Gaussian filtering or median filtering) to remove noise from the image.

[0130] Standardization: The process of standardizing an image to make its pixel values ​​uniform (usually from 0 to 1).

[0131] Size adjustment: Adjust the image to a fixed input size, such as 256x256 or 512x512, to meet the input requirements of the model.

[0132] Generate data representations:

[0133] The preprocessed CT images are input into the model to obtain the data representation. .

[0134] Using the MedHierarchy of medical knowledge, appropriate knowledge embedding vectors are selected based on the specific content of the image (such as whether it contains obvious lung structures).

[0135] For example, if the lesion is located in the lung, the matched knowledge embedding vector can be the context embedding vector corresponding to the above model during training:

[0136]

[0137] in:

[0138] :human body.

[0139] :trunk.

[0140] :Chest.

[0141] :lung.

[0142] "Lung segmentation" task.

[0143] Furthermore, data representation and knowledge are embedded in vector concatenation to form a complete input vector. This concatenated vector is then input into a gating network to obtain a gating vector. The corresponding adapter expert is then matched, and the corresponding weights are calculated to customize the expert sub-model. Finally, the business data is processed based on this expert sub-model to calculate the final lesion segmentation result.

[0144] As can be seen from the above embodiments, the expert knowledge-driven large-scale model customized data processing method described in this application involves inputting acquired business data into a trained large model with dynamically customizable expert sub-models; within the large model, a knowledge embedding vector is determined based on the business data; based on the knowledge embedding vector, the weight of each adapter expert is determined, and an expert sub-model is customized; the business data is processed based on the expert sub-model, dynamically customizing a dedicated model for different business scenarios. This approach balances the generalization and specialization capabilities of the large model within business scenarios, enabling a single model to solve multiple business problems and providing business availability across multiple scenarios. Specifically, this solution is based on large-scale business data, utilizing multi-dimensional business information and expert knowledge to construct a hierarchical structure of business knowledge. Combining efficient large-scale model fine-tuning techniques and hybrid expert strategies, dynamic expert group matching is achieved under the guidance of business data knowledge, thereby customizing different expert sub-models for different business data, achieving automatic adaptation across multiple scenarios, and realizing more professional data processing.

[0145] In medical settings, this approach dynamically customizes specialized models for different clinical scenarios. It balances the generalization and specialized capabilities of large-scale models within a medical context, enabling a single model to solve multiple clinical tasks and ensuring clinical usability across various scenarios. Specifically, this solution leverages large-scale medical data and utilizes multi-dimensional medical information and expert knowledge to construct a hierarchical structure of medical knowledge. Combining efficient fine-tuning techniques for large-scale models with hybrid expert strategies, it achieves dynamic expert group matching guided by medical knowledge. This allows for the customization of different expert sub-models for different medical data, enabling automatic adaptation across multiple scenarios.

[0146] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0147] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0148] Based on the same inventive concept, and corresponding to any of the above embodiments, this application also provides an expert knowledge-driven large model customized data processing device.

[0149] refer to Figure 5 The expert knowledge-driven large model customized data processing device includes:

[0150] Input module 51 is configured to input acquired business data into a large model that has been trained and can dynamically customize expert sub-models.

[0151] The large model module 52, which can dynamically customize expert sub-models, is configured to determine a knowledge embedding vector based on the business data in the large model of the large model with dynamically customizable expert sub-models; and determine the weight of each adapter expert based on the knowledge embedding vector, and customize the corresponding expert sub-model.

[0152] The processing module 53 is configured to process the business data based on the expert sub-model.

[0153] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0154] The apparatus described above is used to implement the corresponding expert knowledge-driven large model customized data processing method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0155] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the expert knowledge-driven large model customized data processing method described in any of the above embodiments.

[0156] Figure 6 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0157] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0158] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0159] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0160] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0161] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0162] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0163] The electronic devices described above are used to implement the corresponding expert knowledge-driven large model customized data processing method in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0164] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the expert knowledge-driven large model customized data processing method as described in any of the above embodiments.

[0165] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0166] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the expert knowledge-driven large model customized data processing method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0167] Based on the same inventive concept, corresponding to the expert knowledge-driven large model customized data processing method described in any of the above embodiments, this disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processor to execute the expert knowledge-driven large model customized data processing method. Corresponding to the execution entity for each step in each embodiment of the expert knowledge-driven large model customized data processing method, the processor executing the corresponding step can belong to the corresponding execution entity.

[0168] The computer program product of the above embodiments is used to cause the computer and / or the processor to execute the expert knowledge-driven large model customization data processing method as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0169] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0170] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0171] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0172] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A method for customized data processing of large models driven by expert knowledge, characterized in that, include: The acquired medical segmentation dataset is input into a large model that has been trained and can dynamically customize medical expert sub-models; In the large model with dynamically customizable medical expert sub-models, medical knowledge embedding vectors are determined based on the medical segmentation dataset; Based on the medical knowledge embedding vector, the weight of each adapter expert is determined, and a medical expert sub-model is customized, including: performing feature representation on the medical segmentation dataset to obtain a medical data representation; obtaining a gating vector based on the medical data representation and the medical knowledge embedding vector; determining the weight of each adapter expert based on the gating vector, and customizing the medical expert sub-model. The medical segmentation dataset is segmented based on the aforementioned medical expert sub-model; The training process of the large model, which features a pre-trained, dynamically customizable medical expert sub-model, includes: Based on the big data of corresponding medical segmentation business, an anatomical knowledge tree is constructed to obtain the knowledge hierarchy structure of medical segmentation data; A context embedding prior is constructed based on the knowledge hierarchy structure of the medical segmentation data; in the context embedding prior, each medical segmentation data sample corresponds to a set of learnable context embedding vectors to represent the knowledge prior of the medical segmentation data sample; the context embedding prior dynamically provides medical knowledge for the large model; the knowledge prior includes anatomical sites and tasks; Construct a hybrid adapter expert network; the hybrid adapter expert network includes a gating network and at least two adapter experts; The context embedding prior and the hybrid adapter expert network are embedded into the encoder of the large model to obtain the large model to be trained; Based on the results of the large model to be trained and the corresponding labels of the big data, the large model to be trained is trained to obtain the trained large model of dynamically customizable medical expert sub-models.

2. The method according to claim 1, characterized in that, The construction of an anatomical knowledge tree based on the big data of corresponding medical segmentation business yields a knowledge hierarchy structure for the medical segmentation data, including: Analytical terms are extracted from the acquired big data to obtain entity nodes; Based on partial and inclusion relationships, the entity nodes are divided into a hierarchical structure according to the logic of human body - body part - body subregion - specific organ and tissue; Based on the hierarchical structure, the medical segmentation data samples in the big data are displayed and mapped to each leaf node, and the anatomical knowledge tree is constructed to obtain the knowledge hierarchy structure of the medical segmentation data.

3. The method according to claim 1, characterized in that, The process of determining the medical knowledge embedding vector based on the medical segmentation dataset includes: Based on the labels corresponding to the medical segmentation dataset, determine the node position of the medical segmentation dataset in the knowledge hierarchy structure of the medical segmentation data; Based on the node positions, the medical knowledge embedding vector of the medical segmentation dataset is determined.

4. The method according to claim 1, characterized in that, The segmentation of the medical segmentation dataset based on the medical expert sub-model is performed using the following formula: in, Indicates the processing result. Indicates a feedforward network. Indicates the first An adapter expert, express The first in The entry represents the first entry. An adapter expert Contribution To represent data, the medical gating vector is defined as follows: .

5. A knowledge-driven, large-scale model customized data processing device, characterized in that, include: The input module is configured to feed the acquired medical segmentation dataset into a large model that is trained and can dynamically customize medical expert sub-models; A large model module with dynamically customizable medical expert sub-models is configured to determine medical knowledge embedding vectors based on the medical segmentation dataset within the large model with dynamically customizable medical expert sub-models. Based on the medical knowledge embedding vector, the weight of each adapter expert is determined, and a corresponding medical expert sub-model is customized, including: performing feature representation on the medical segmentation dataset to obtain a medical data representation; obtaining a gating vector based on the medical data representation and the medical knowledge embedding vector; determining the weight of each adapter expert based on the gating vector, and customizing the medical expert sub-model. The processing module is configured to perform segmentation processing on the medical segmentation dataset based on the medical expert sub-model; The training process of the large model, which features a pre-trained, dynamically customizable medical expert sub-model, includes: Based on the big data of corresponding medical segmentation business, an anatomical knowledge tree is constructed to obtain the knowledge hierarchy structure of medical segmentation data; A context embedding prior is constructed based on the knowledge hierarchy structure of the medical segmentation data; in the context embedding prior, each medical segmentation data sample corresponds to a set of learnable context embedding vectors to represent the knowledge prior of the medical segmentation data sample; the context embedding prior dynamically provides medical knowledge for the large model; the knowledge prior includes anatomical sites and tasks; Construct a hybrid adapter expert network; the hybrid adapter expert network includes a gating network and at least two adapter experts; The context embedding prior and the hybrid adapter expert network are embedded into the encoder of the large model to obtain the large model to be trained; Based on the results of the large model to be trained and the corresponding labels of the big data, the large model to be trained is trained to obtain the trained large model of dynamically customizable medical expert sub-models.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 4.

8. A computer program product comprising computer program instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 4.

Images