Expert knowledge-driven large-model customized data processing method and related equipment

Abstract

The invention provides an expert knowledge-driven large-model customized data processing method and related equipment. The method comprises the following steps: inputting acquired business data into a trained large model capable of dynamically customizing an expert sub-model; in the large model, based on the business data, determining a knowledge embedding vector; based on the knowledge embedding vector, determining the weight of each adapter expert, and customizing an expert sub-model; and processing the business data based on the expert sub-model. According to the embodiment of the invention, the business knowledge hierarchical structure is constructed by using the multi-dimensional business information and the expert knowledge based on the large-scale business data. In combination with a large-model efficient fine tuning technology and a mixed expert strategy, dynamic expert group adaptation is realized under the guidance of business data knowledge, so that different expert models are customized for different business data, automatic adaptation under multiple scenes is realized, and more professional data processing is realized.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an expert knowledge-driven large-model customized data processing method and related equipment. Background Art

[0002] Currently, intelligent analysis is developing in the direction of multi-task. Traditional single-task models have gradually been replaced by generalist large models, which have the potential to handle cross-domain, cross-modal, and cross-task. However, existing technologies still face the contradiction between generality and specialization in practical applications. Generalist large models have strong generalization capabilities, but their accuracy on specific tasks may not be as good as that of special models. This is due to the semantic conflict between tasks during multi-task mixed training. Conflicting tasks will interfere with each other during training, resulting in negative transfer in performance. Summary of the invention

[0003] In view of this, the purpose of this application is to propose an expert knowledge-driven large-model customized data processing method and related equipment.

[0004] Based on the above objectives, this application provides an expert knowledge-driven large model customized data processing method, including: Input the acquired business data into the trained big model with dynamically customizable expert sub-models; In the large model of the dynamically customizable expert sub-model, based on the business data, a knowledge embedding vector is determined; based on the knowledge embedding vector, a weight of each adapter expert is determined to customize the expert sub-model; The business data is processed based on the expert sub-model.

[0005] In a possible implementation, the training process of the large model of the trained dynamically customizable expert sub-model includes: Build a knowledge tree based on the big data of the corresponding business to obtain a data hierarchy; Building a context embedding prior based on the data hierarchy; in the context embedding prior, each data sample corresponds to a set of learnable context embedding vectors to represent the knowledge prior of the sample; Constructing a hybrid adapter expert network; the hybrid adapter expert network includes a gating network and at least two adapter experts; Embedding the context embedding prior and the hybrid adapter expert network into the encoder of the large model to obtain the large model to be trained; Based on the result of inputting the big data into the big model to be trained and the labels corresponding to the big data, the big model to be trained is trained to obtain the trained big model of the dynamically customizable expert sub-model.

[0006] In a possible implementation, the building of a knowledge tree based on big data of the corresponding business to obtain a data hierarchy structure includes: Extracting terms from the acquired big data to obtain entity nodes; Dividing the entity nodes into a hierarchical structure; Based on the hierarchical structure, data samples in the big data are explicitly mapped to each leaf node, the knowledge tree is constructed, and the data hierarchical structure is obtained.

[0007] In a possible implementation manner, determining the knowledge embedding vector based on the business data includes: Determine, according to the label corresponding to the business data, the node position corresponding to the business data in the data hierarchy; Based on the node position, the knowledge embedding vector of the business data is determined.

[0008] In a possible implementation, determining the weight of each adapter expert based on the knowledge embedding vector and customizing the expert sub-model includes: Characterizing the data to obtain data characterization; Obtaining a gating vector based on the data representation and the knowledge embedding vector; The weight of each adapter expert is determined based on the gating vector to customize the expert sub-model.

[0009] In a possible implementation, the processing of the service data based on the expert sub-model is performed by the following formula:

[0010] in, Indicates the processing result. represents a feed-forward network, Indicates Adapter experts, express The Entries, representing Adapter Experts Contribution Represents the data representation, and the gate vector is defined as .

[0011] Based on the same inventive concept, the embodiment of the present application also provides an expert knowledge-driven large model customized data processing device, including: An input module, configured to input the acquired business data into the trained large model of the dynamically customizable expert sub-model; The large model module of the dynamically customizable expert sub-model is configured to determine a knowledge embedding vector based on the business data in the large model of the dynamically customizable expert sub-model; determine a weight of each adapter expert based on the knowledge embedding vector, and customize the corresponding expert sub-model; The processing module is configured to process the business data based on the expert sub-model.

[0012] Based on the same inventive concept, an embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the expert knowledge-driven large-model customized data processing method as described in any one of the above items is implemented.

[0013] Based on the same inventive concept, an embodiment of the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute any of the above-mentioned expert knowledge-driven large-model customized data processing methods.

[0014] Based on the same inventive concept, an embodiment of the present application also provides a computer program product, which includes computer program instructions, and the computer instructions are used to enable the computer program product to execute any of the above-mentioned expert knowledge-driven large-model customized data processing methods.

[0015] From the above, it can be seen that the expert knowledge-driven large model customized data processing method and related equipment provided by this application are implemented by inputting the acquired business data into the large model of the trained dynamically customizable expert sub-model; in the large model of the dynamically customizable expert sub-model, based on the business data, the knowledge embedding vector is determined; based on the knowledge embedding vector, the weight of each adapter expert is determined to customize the expert sub-model; based on the expert sub-model, the business data is processed to dynamically customize the dedicated model for different business scenarios. In the business scenario, the generalization ability and dedicated ability of the large model are taken into account to realize a large model to solve multiple businesses and have business availability in multiple scenarios. Specifically, the solution is based on large-scale business data and uses multi-dimensional business information and expert knowledge to construct a business knowledge hierarchy. Combined with the efficient fine-tuning technology of the large model and the hybrid expert strategy, dynamic expert group matching is achieved under the guidance of business data knowledge, so as to customize different expert sub-models for different business data, realize automatic adaptation in multiple scenarios, and realize more professional data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A schematic diagram of the process flow of a large model customized data processing method driven by expert knowledge according to an embodiment of the present application; Figure 2 A schematic diagram of a large model structure of a dynamically customizable expert sub-model according to an embodiment of the present application; Figure 3 A schematic diagram of the construction process of the medical knowledge hierarchy structure of an embodiment of the present application; Figure 4 A schematic diagram of a priori context embedding based on a hierarchical structure according to an embodiment of the present application; Figure 5 A schematic diagram of the structure of a large model customized data processing device driven by expert knowledge in an embodiment of the present application; Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0019] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0020] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0021] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0022] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0023] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0024] As mentioned in the background technology section, current intelligent analysis is developing in the direction of multi-task. Traditional single-task models have gradually been replaced by generalist large models, which have the potential to handle cross-domain, cross-modal, and cross-task. However, the existing technology still faces the contradiction between generality and specificity in practical applications. Generalist large models have strong generalization capabilities, but their accuracy on specific tasks may not be as good as that of specialized models. This is due to the semantic conflict between tasks during multi-task mixed training. Conflicting tasks will interfere with each other during training, resulting in negative transfer in performance.

[0025] Based on the above considerations, the embodiment of the present application proposes a large model customized data processing method driven by expert knowledge, which is achieved by inputting the acquired business data into the large model of the trained dynamically customizable expert sub-model; in the large model of the dynamically customizable expert sub-model, based on the business data, determining the knowledge embedding vector; based on the knowledge embedding vector, determining the weight of each adapter expert, and customizing the expert sub-model; processing the business data based on the expert sub-model, and dynamically customizing the dedicated model for different business scenarios. In the business scenario, the generalization ability and dedicated ability of the large model are taken into account, so that a large model can solve multiple businesses and have business availability in multiple scenarios. Specifically, the solution is based on large-scale business data, and uses multi-dimensional business information and expert knowledge to construct a business knowledge hierarchy. Combined with the efficient fine-tuning technology of the large model and the hybrid expert strategy, dynamic expert group matching is achieved under the guidance of business data knowledge, so as to customize different expert sub-models for different business data, realize automatic adaptation in multiple scenarios, and realize more professional data processing.

[0026] The technical solutions of the embodiments of the present application are described in detail below through specific examples.

[0027] refer to Figure 1 The expert knowledge-driven large model customized data processing method of the embodiment of the present application includes the following steps: Step S101, inputting the acquired business data into the trained large model of the dynamically customizable expert sub-model; Step S102, in the large model of the dynamically customizable expert sub-model, determining a knowledge embedding vector based on the business data; determining the weight of each adapter expert based on the knowledge embedding vector, and customizing the expert sub-model; Step S103: Process the business data based on the expert sub-model.

[0028] refer to Figure 2 , which is a schematic diagram of the large model structure of the dynamically customizable expert sub-model in an embodiment of the present application.

[0029] In the embodiments of the present application, taking the visual large model SAM (Segment Anything Model) in a general segmentation scenario as an example, it is demonstrated how to dynamically customize medical expert sub-models for tasks of different modalities and organs in medical scenarios, thereby ensuring the dedicated capabilities of the large model in different clinical scenarios. Specifically, the above problems can be solved by constructing a contextual mixture of adapter experts (ConMoAE). ConMoAE consists of a hierarchical structure-based contextual embedding prior (HScEP) and a context-guided mixture of adapters network (CMoAN). As Figure 2 As shown, by integrating ConMoAE into the feed forward network (FFN) of each Transformer layer in the SAM image encoder, contextual medical knowledge can be introduced into the large model, and then separate parameters can be provided for conflicting tasks, and semantically related sub-models can be generated for different tasks to alleviate the "negative transfer" phenomenon, thereby ensuring dedicated capabilities in medical scenarios.

[0030] In some embodiments, with respect to step S101, the training process of the large model of the trained dynamically customizable expert sub-model includes: constructing a knowledge tree based on the big data of the corresponding business to obtain a data hierarchy; constructing a context embedding prior based on the data hierarchy; in the context embedding prior, each data sample corresponds to a set of learnable context embedding vectors to characterize the knowledge prior of the sample; constructing a hybrid adapter expert network; the hybrid adapter expert network includes a gating network and at least two adapter experts; embedding the context embedding prior and the hybrid adapter expert network into the encoder of the large model to obtain the large model to be trained; based on the result of inputting the big data into the large model to be trained and the label corresponding to the big data, training the large model to be trained to obtain the large model of the trained dynamically customizable expert sub-model.

[0031] In some embodiments, the construction of a knowledge tree based on the big data of the corresponding business to obtain a data hierarchy includes: extracting terms from the acquired big data to obtain entity nodes; dividing the entity nodes into a hierarchical structure; based on the hierarchical structure, displaying and mapping data samples in the big data to each leaf node to construct the knowledge tree and obtain the data hierarchy.

[0032] In this embodiment, it is necessary to first combine the medical ontology knowledge base and expert annotation to build a medical knowledge hierarchy. This hierarchy can not only systematically organize large-scale medical data sets, but also provide knowledge guidance for large models, thereby ensuring dedicated capabilities in different medical scenarios.

[0033] refer to Figure 3 , which is a schematic diagram of the construction process of the medical knowledge hierarchy structure of the embodiment of the present application.

[0034] like Figure 3As shown in the figure, taking the general medical segmentation scenario as an example, the hierarchy can be constructed based on RadLex, a public terminology dictionary developed by the Radiological Society of North America (RSNA). According to the RadLex identifier RID, the anatomical related words in the collected medical segmentation dataset are mapped to the RadLex vocabulary, and the relevant terms are extracted from it as entity nodes. With the help of imaging experts, the terms are integrated into the medical knowledge hierarchy according to the "Has_part" and "Contains" relationships. Thus, an anatomical knowledge tree MedHierarchy including human body, body parts, body sub-regions, organs and tissues can be constructed. By making full use of the constructed medical knowledge hierarchy, medical data can be systematically organized, and large-scale medical data can be assigned to the corresponding leaf nodes in the hierarchy according to different medical tasks, so as to dynamically call the corresponding anatomical knowledge for different data. Under each leaf node, data samples of the same task are organized together, so as to bring positive migration effects to each other during model training.

[0035] Specifically, common anatomical terms (such as human organs, tissues, parts, etc.) in medical segmentation tasks can be associated with the RadLex vocabulary through annotation and mapping methods.

[0036] Use RadLex's logical relationships (such as the hierarchical structure of parts) to design knowledge nodes. For example: Top-level node (Root): Human Body.

[0037] Subnode: Body Parts (body parts, such as head, neck, torso, etc.).

[0038] Secondary child nodes: Body Sub-regions (body sub-regions, such as head, abdomen, etc.).

[0039] Third-level subnode: Organs and Tissues (such as heart, liver, lungs, etc.).

[0040] Annotation of entity relationships: With the help of an imaging expert, analyze the relationship between different terms, such as: Torso (torso) Has_part abdomen (abdomen) Head contains pituitary. Based on the relationship, a complete anatomical knowledge tree MedHierarchy is constructed.

[0041] Dynamic mapping and expansion: Dynamically map new anatomical terms to existing knowledge structures for ever-changing medical datasets. For example, when a new segmentation dataset is introduced that involves terms not included in it, the knowledge tree can be expanded based on RadLex or other authoritative sources.

[0042] Furthermore, after obtaining the hierarchical structure of medical knowledge, a contextual embedding prior (Hierarchical Structure-based Contextual Embedding Prior, HScEP) can be constructed according to the hierarchical structure to dynamically provide medical knowledge for the large model.

[0043] refer to Figure 4 , which is a schematic diagram of a hierarchical context embedding prior according to an embodiment of the present application.

[0044] Specifically, HScEP explicitly maps the knowledge in the medical hierarchy (MedHierarchy) into contextual embedding vectors:

[0045] In some embodiments, Figure 4 As shown, HScEP can be composed of five layers of embedded vector points. is the top-level anatomical embedding prior, representing the human body in MedHierarchy. and The second and third layers are anatomical embedding priors, representing body parts and body sub-regions in MedHierarchy, respectively. Represents the embedding priors of specific organs and tissues in the fourth layer MedHierarchy. is the fifth layer embedding prior, which represents the task to which the data sample belongs. Each embedding point consists of a learnable vector.

[0046] When a data sample is given, HScEP will assign a set of context embedding vectors to each data sample according to the position of the sample in MedHierarchy. This set of context embedding vectors assigns different knowledge priors to different tasks, decoupling unrelated tasks and thus avoiding task conflicts in the model during multi-task training.

[0047] Specifically, the core design of HScEP is based on the knowledge hierarchy (MedHierarchy), which is hierarchically mapped into context embedding vectors, specifically including five layers of embedding points: Top-level embedding ( ): Human Body.

[0048] Represents the highest level of anatomical knowledge, defining an embedded prior for the entire human body as a whole.

[0049] It captures common medical background knowledge for the model from a holistic perspective, such as the global structure and anatomical common sense of the human body.

[0050] The second layer embedding ( ): Body Parts.

[0051] Embedding priors representing the main parts of the human body (such as head and neck, torso, bones, blood vessels, and skin).

[0052] It provides the model with knowledge about different body parts, such as the morphological and functional differences in specific areas of the head, neck, and trunk.

[0053] The third layer embedding ( ): Body Sub-regions.

[0054] Capturing finer-grained regional embedding priors for specific parts of the body, such as the chest and abdomen in the torso.

[0055] It helps the model to identify more specific medical structures, thus enhancing the understanding of the internal characteristics of the region.

[0056] The fourth layer embedding ( ): Organs and tissues (Organs&Tissues).

[0057] Represents embedding priors for specific organs (e.g., heart, liver, lungs) and their tissues.

[0058] This layer is directly related to the segmentation task of medical data, providing the model with knowledge guidance about specific anatomical regions.

[0059] The fifth layer embedding ( ): A specific task (Task).

[0060] Represents the specific task to which the data sample belongs, such as the embedding prior of lung segmentation, skull segmentation, etc.

[0061] It explicitly encodes the goal of the task as an embedding prior to avoid conflicts between different tasks.

[0062] In a feasible embodiment, the corresponding node is found in MedHierarchy by the anatomical part label of the data sample (such as "Lungs" for lungs). For example: The sample is labeled "Lungs", so its position in the knowledge tree is: human body ->Torso ->Chest ->Lung ->Lung Segmentation.

[0063] Assignment of context embedding vector: According to the position of the sample in the knowledge tree, the corresponding context embedding vector is assigned.

[0064] For example, for samples related to “lung”, the assigned context embedding vector is:

[0065] in: : Global embedding of the human body.

[0066] : The torso part is embedded.

[0067] : Subregional embedding of the chest.

[0068] :Organ embedding of the lung. .

[0069] : Task Embedding for the Lung Segmentation Task.

[0070] During the training process, the model dynamically calls the corresponding context embedding vector for each sample input , which is combined with the input image features. The embedding vector Prior information of medical knowledge and task objectives is provided, making it easier for the model to capture anatomical semantics.

[0071] Through the construction and application of HScEP, the medical knowledge hierarchy MedHierarchy can be explicitly mapped into contextual embedding vectors, providing the model with dynamic, task-related knowledge priors. Taking the lung segmentation task as an example, the five-layer embedding vector of HScEP can capture hierarchical knowledge from the global human body to the local lung, and clarify the learning direction of the model in combination with the task objectives. This method is not only applicable to medical image segmentation scenarios, but can also be extended to other business scenarios that require multi-task collaboration or knowledge injection, thereby providing artificial intelligence with stronger knowledge expression capabilities and generalization.

[0072] Furthermore, after obtaining the context embedding prior HScEP, a context-guided mixture of adapters network (CMoAN) can be constructed. The network consists of several adapter experts and a gating network. Given a data sample, the gating network dynamically assigns different weights to each adapter expert under the guidance of the context knowledge prior provided by HScEP, thereby constructing different expert combinations to serve different tasks.

[0073] Specifically, each adapter expert consists of a bottleneck structure (bottleneck structure), a downsampling layer (down-projection layer), an upsampling layer (up-projection layer), and a non-linear activation layer ReLU (non-linear layer). Given a data representation , the adapted features can be expressed as .

[0074] CMoAN is characterized by data and embedding prior As input, the gating network is guided to determine the The contribution weight of data samples. The number of adapter experts is defined as , the gating vector is defined as Given a data representation and contextual embedding priors , the gate vector can be obtained by the following calculation:

[0075] in, is the concatenation operator, represents a gating network consisting of a linear layer, outputting The scores represent the contribution weight of each adapter expert. In summary, the final output of CMoAN is It can be calculated as follows:

[0076] in, represents a feed-forward network, For the Adapter experts, yes The Entries, representing Adapter Experts contribution.

[0077] In a feasible embodiment, still taking the above-mentioned chest CT image as an example, after the embedding vector is determined, the data representation and contextual embedding priors The weight of each adapter expert is calculated through the gating network. Assuming K=3 adapter experts, the output weight G=[0.2, 0.5, 0.3] means that the second adapter expert contributes the most to the current task. The final output is the weighted combined feature of each adapter expert.

[0078] Afterwards, the above-constructed context embedding prior and hybrid adapter expert network are embedded into the encoder of the large model to obtain the large model to be trained. The large model to be trained is further trained based on the result of inputting big data into the large model to be trained and the labels corresponding to the big data to obtain a trained large model with dynamically customizable expert sub-model. The specific training process will not be repeated here.

[0079] After the model training is completed, for step S101, the acquired business data is input into the large model of the trained dynamically customizable expert sub-model. Furthermore, for steps S102 and S103, in the large model of the dynamically customizable expert sub-model, based on the business data, a knowledge embedding vector is determined; based on the knowledge embedding vector, the weight of each adapter expert is determined to customize the expert sub-model; and based on the expert sub-model, the business data is processed.

[0080] In some embodiments, determining the knowledge embedding vector based on the business data includes: determining the node position corresponding to the business data in the data hierarchy according to the label corresponding to the business data; and determining the knowledge embedding vector of the business data based on the node position.

[0081] In some embodiments, determining the weight of each adapter expert based on the knowledge embedding vector and customizing the expert sub-model includes: characterizing the data to obtain data representation; obtaining a gating vector based on the data representation and the knowledge embedding vector; and determining the weight of each adapter expert based on the gating vector and customizing the expert sub-model.

[0082] In some embodiments, the processing of the business data based on the expert sub-model is performed by the following formula:

[0083] in, Indicates the processing result. represents a feed-forward network, Indicates Adapter experts, express The Entries, representing Adapter Experts Contribution Represents the data representation, and the gate vector is defined as .

[0084] The following takes lung CT image lesion segmentation as an example to illustrate the above steps.

[0085] First, obtain CT images: Obtain a set of lung CT image data to be analyzed from a hospital or medical image database (such as a PACS system). These images may be in DICOM format. Ensure that the number and quality of images are suitable for analysis, and avoid using low-quality or blurry images.

[0086] Then preprocess the image: Denoising: Use filtering algorithms (such as Gaussian filtering or median filtering) to remove noise from images.

[0087] Normalization: Normalize an image so that its pixel values ​​have a uniform range (usually 0 to 1).

[0088] Resizing: Resize the image to a fixed input size, such as 256x256 or 512x512, to meet the input requirements of the model.

[0089] Generate data representation: The preprocessed CT images are input into the model to obtain data representation .

[0090] Using the medical knowledge hierarchy (MedHierarchy), the corresponding knowledge embedding vector is selected according to the specific content of the image (such as whether it contains obvious lung structure).

[0091] For example, if the lesion is located in the lungs, the matched knowledge embedding vector can be the corresponding context embedding vector in the above model training process:

[0092] in: :human body.

[0093] :trunk.

[0094] :Chest.

[0095] :lung.

[0096] : “Lung segmentation” task.

[0097] Furthermore, the data representation and knowledge embedding vector are connected to form a complete input vector, which is then input into the gating network to obtain a gating vector. The corresponding adapter expert is further matched, and the corresponding weights are calculated to customize the expert sub-model. The business data is then processed based on the expert sub-model to calculate the final lesion segmentation result.

[0098] It can be seen from the above embodiments that the expert knowledge-driven large model customized data processing method described in the embodiment of the present application is to input the acquired business data into the large model of the trained dynamically customizable expert sub-model; in the large model of the dynamically customizable expert sub-model, based on the business data, determine the knowledge embedding vector; based on the knowledge embedding vector, determine the weight of each adapter expert and customize the expert sub-model; based on the expert sub-model, process the business data to dynamically customize the dedicated model for different business scenarios. In the business scenario, the generalization ability and dedicated ability of the large model are taken into account to realize a large model to solve multiple businesses and have business availability in multiple scenarios. Specifically, the solution is based on large-scale business data and uses multi-dimensional business information and expert knowledge to construct a business knowledge hierarchy. Combined with the efficient fine-tuning technology of the large model and the hybrid expert strategy, dynamic expert group matching is achieved under the guidance of business data knowledge, so as to customize different expert sub-models for different business data, realize automatic adaptation in multiple scenarios, and realize more professional data processing.

[0099] In medical scenarios, special models are dynamically customized for different clinical scenarios. In medical scenarios, the generalization and specialization capabilities of large models are taken into account to achieve a large model to solve multiple clinical tasks and have clinical usability in multiple scenarios. Specifically, the solution is based on large-scale medical data and uses multi-dimensional medical information and expert knowledge to build a medical knowledge hierarchy. Combined with the efficient fine-tuning technology of large models and hybrid expert strategies, dynamic expert group matching is achieved under the guidance of medical knowledge, so as to customize different expert sub-models for different medical data and achieve automatic adaptation in multiple scenarios.

[0100] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0101] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0102] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an expert knowledge-driven large-model customized data processing device.

[0103] refer to Figure 5 , the expert knowledge-driven large model customized data processing device comprises: An input module 51 is configured to input the acquired business data into the trained large model of the dynamically customizable expert sub-model; The large model module 52 of the dynamically customizable expert sub-model is configured to determine a knowledge embedding vector based on the business data in the large model of the dynamically customizable expert sub-model; determine the weight of each adapter expert based on the knowledge embedding vector, and customize the corresponding expert sub-model; The processing module 53 is configured to process the business data based on the expert sub-model.

[0104] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0105] The device of the above embodiment is used to implement the corresponding expert knowledge-driven large-model customized data processing method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0106] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the expert knowledge-driven large-model customized data processing method described in any of the above embodiments is implemented.

[0107] Figure 6 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0108] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0109] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0110] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0111] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0112] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0113] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0114] The electronic device of the above-mentioned embodiment is used to implement the corresponding expert knowledge-driven large-model customized data processing method in any of the above-mentioned embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0115] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the expert knowledge-driven large-model customized data processing method as described in any of the above embodiments.

[0116] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0117] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the expert knowledge-driven large-model customized data processing method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0118] Based on the same inventive concept, corresponding to the expert knowledge-driven large-model customized data processing method described in any of the above embodiments, the present disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer so that the computer and / or the processor executes the expert knowledge-driven large-model customized data processing method. Corresponding to the execution subject corresponding to each step in each embodiment of the expert knowledge-driven large-model customized data processing method, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0119] The computer program product of the above embodiment is used to enable the computer and / or the processor to execute the expert knowledge-driven large-model customized data processing method as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0120] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0121] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power / ground connections to the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device may be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present application are to be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0122] Although the present application has been described in conjunction with specific embodiments of the present application, many alternatives, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the discussed embodiments.

[0123] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A large model customized data processing method driven by expert knowledge, characterized in that: include: Input the acquired business data into the trained big model with dynamically customizable expert sub-models; In the large model of the dynamically customizable expert sub-model, determining a knowledge embedding vector based on the business data; Based on the knowledge embedding vector, determining the weight of each adapter expert and customizing the expert sub-model; The business data is processed based on the expert sub-model.

2. The method according to claim 1, characterized in that The training process of the trained large model with dynamically customizable expert sub-model includes: Build a knowledge tree based on the big data of the corresponding business to obtain a data hierarchy; Building a context embedding prior based on the data hierarchy; in the context embedding prior, each data sample corresponds to a set of learnable context embedding vectors to represent the knowledge prior of the sample; Constructing a hybrid adapter expert network; the hybrid adapter expert network includes a gating network and at least two adapter experts; Embedding the context embedding prior and the hybrid adapter expert network into the encoder of the large model to obtain the large model to be trained; Based on the result of inputting the big data into the big model to be trained and the labels corresponding to the big data, the big model to be trained is trained to obtain the trained big model of the dynamically customizable expert sub-model.

3. The method according to claim 2, characterized in that The knowledge tree is constructed based on the big data of the corresponding business to obtain a data hierarchy structure, including: Extracting terms from the acquired big data to obtain entity nodes; Dividing the entity nodes into a hierarchical structure; Based on the hierarchical structure, data samples in the big data are explicitly mapped to each leaf node, the knowledge tree is constructed, and the data hierarchical structure is obtained.

4. The method according to claim 2, characterized in that: The step of determining a knowledge embedding vector based on the business data includes: Determine, according to the label corresponding to the business data, the node position corresponding to the business data in the data hierarchy; Based on the node position, the knowledge embedding vector of the business data is determined.

5. The method according to claim 1, characterized in that The method of determining the weight of each adapter expert based on the knowledge embedding vector and customizing the expert sub-model includes: Characterizing the data to obtain data characterization; Obtaining a gating vector based on the data representation and the knowledge embedding vector; The weight of each adapter expert is determined based on the gating vector to customize the expert sub-model.

6. The method according to claim 5, characterized in that The processing of the business data based on the expert sub-model is performed by the following formula: in, Indicates the processing result. represents a feed-forward network, Indicates Adapter experts, express The Entries, representing Adapter Experts Contribution Represents the data representation, and the gate vector is defined as .

7. A large model customized data processing device driven by expert knowledge, characterized in that: include: An input module, configured to input the acquired business data into the trained large model of the dynamically customizable expert sub-model; A large model module of the dynamically customizable expert sub-model is configured to determine a knowledge embedding vector based on the business data in the large model of the dynamically customizable expert sub-model; Based on the knowledge embedding vector, determine the weight of each adapter expert and customize the corresponding expert sub-model; The processing module is configured to process the business data based on the expert sub-model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 6.

10. A computer program product, comprising computer program instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 6.

Images