Federal text open intention classification method based on granular ball calculation

Abstract

The invention discloses a federated text open intention classification method based on granular ball calculation, which comprises a client training step, a server aggregation step and a client classification step which are completed in sequence, and is characterized in that the client training step uploads obtained pre-training language model parameters and a local granular ball knowledge base to a server; the server aggregation step constructs a global model and a global granular ball knowledge base, and issues the global model and the global granular ball knowledge base to each client; a client classification step: inputting a to-be-classified sample into the global model to obtain a global feature; and comparing the global features with a global granular ball knowledge base to realize intention classification. According to the method, the granular ball structured knowledge is constructed at the client side to realize efficient representation learning, the model and the knowledge are aggregated at the server side to support global reasoning and open recognition, and the method is suitable for a privacy protection scene of multi-client and multi-source heterogeneous text data; and accurate identification of known intentions and effective rejection of unknown intentions can still be realized.

Description

Technical Field

[0001] The present invention relates to the field of electronic data processing, and in particular to a method for classifying open intent of federated texts based on granular spherical computing. Background Art

[0002] In the field of natural language processing, intent classification is a key task in achieving semantic understanding between humans and machines. It is widely used in systems such as intelligent customer service, voice assistants, and conversational robots. Traditional intent classification methods typically assume that all intent categories in the training set will appear during the inference phase, a so-called "closed-world assumption." However, in real-world applications, users may make new types of requests not seen during training at any time, resulting in the model failing to correctly identify or misclassifying them, impacting user experience and system reliability.

[0003] To address this issue, researchers proposed the Open Intent Classification (OIC) task, whose goal is to detect and reject unknown intents (i.e., classify them as "unknown") while simultaneously identifying known intents. Existing OIC methods fall into two main categories: one introduces pseudo-unknown samples during the training phase, treating them as the "K+1" class to train multi-classification models; the other uses only known classes for training, and during inference, utilizes mechanisms such as thresholds, boundaries, or distances to reject input samples that fall outside the known class boundaries. These methods generally perform well in experimental settings, but most rely on centralized training, assuming that all training data can be stored and accessed uniformly.

[0004] However, in real-world applications, user data is often stored dispersed across multiple devices or organizations. Centralized data aggregation is impractical due to privacy protection, data sovereignty, and legal and regulatory considerations. For example, in scenarios like finance, healthcare, and education, user data held by different institutions or terminals is isolated and cannot be freely shared. In this context, the traditional OIC approach's reliance on centralized data severely limits its practical application.

[0005] Therefore, there is an urgent need for an open intent classification method that can simultaneously meet the following requirements: (1) be compatible with distributed and privacy-restricted training scenarios; (2) effectively integrate the knowledge representations of heterogeneous clients; and (3) build a controllable and structured open space model to improve the ability to recognize unknown intents. Summary of the Invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method for classifying open intent of federated texts based on granular sphere computing.

[0007] The object of the present invention is achieved through the following technical solutions:

[0008] In a first aspect, the present invention provides a method for classifying open intent in federated text based on granular computing, comprising a client training step, a server aggregation step, and a client classification step, wherein the client training step includes the following sub-steps:

[0009] S11: Use the pre-trained language model to extract sample features of the local training set to obtain a real sample set containing the sample feature vectors and sample category labels;

[0010] S12: Modeling the real sample set using an unsupervised sphere clustering method to obtain a first sphere set that is an approximate expression of the class distribution;

[0011] S13: constructing pseudo unknown samples based on the first ball set, and uniformly assigning the category labels of the pseudo unknown samples to the unknown intent class to obtain a pseudo unknown sample set;

[0012] S14: Locally train the pre-trained language model using the real sample set and the pseudo unknown sample set. After the training is completed, upload the obtained pre-trained language model parameters and the first ball set as the local ball knowledge base to the server;

[0013] The server-side aggregation step includes the following sub-steps:

[0014] S21: Aggregate the pre-trained language model parameters uploaded by each client to construct a global model;

[0015] S22: Aggregate the local granular knowledge bases uploaded by each client to construct a global granular knowledge base;

[0016] S23: Send the global model and global granular knowledge base to each client;

[0017] The client classification step includes the following sub-steps:

[0018] S31: Inputting the sample to be classified into the global model to obtain global features;

[0019] S32: Compare the global features with the global granular knowledge base to achieve intent classification.

[0020] Furthermore, in step S11, the sample features of the local training set are extracted using the pre-trained language model to obtain a real sample set containing the sample feature vectors and the sample category labels, including:

[0021] Client The local training set on ;in, Represents the client The number of local intent text samples, i represents the i-th sample, Represents the client The i-th intent text sample, To represent the client The category label of the i-th intent text sample;

[0022] Extraction using pre-trained language models The characteristics of the client The feature vector of the i-th intent text sample ; and further obtain the real sample set .

[0023] Furthermore, in step S12, the first ball set as an approximate expression of the category distribution , the jth ball Has the following properties: Particle size : The number of samples contained in the sphere; the centroid of the sphere : the mean of all sample features within the sphere; the mean radius of the sphere : the average Euclidean distance from all samples in the sphere to the center of mass; the maximum radius of the sphere : The maximum Euclidean distance from all samples in the sphere to the center of mass; the sphere radius; the sphere label : The label of the category with the highest proportion in the pellet; pellet purity :The ball belongs to the ball label The sample proportion of

[0024] The real sample set is modeled using unsupervised sphere clustering to obtain a first sphere set that approximates the category distribution, including:

[0025] If the purity of the lth ball is Below the set threshold And the sample size Exceeding the minimum threshold , then the division operation is triggered: for the particle ball , count all the label types that appear in it ; The ball is labeled Divide into several sub-spheres, each of which uses a sample within the label as the pseudo-centroid; distribute the samples in the original sphere to the sub-sphere represented by the nearest pseudo-centroid; re-evaluate the purity and size of each sub-sphere, and retain it if it meets the termination condition; otherwise, continue recursive division;

[0026] Finally, the set of all particles that meet the conditions Will be used to represent the client The known class structure on the sphere, the centroid and radius of the sphere constitute the sphere prototype of each category, which is used as an approximate expression of the category distribution under privacy protection.

[0027] Furthermore, in step S13, a pseudo unknown sample is constructed based on the first ball set, and the category labels of the pseudo unknown samples are uniformly assigned as unknown intent classes, thereby obtaining a pseudo unknown sample set, including:

[0028] S131: Initialize candidate samples, including:

[0029] For any two balls with different labels in the first ball set , calculate the center of mass of the two The midpoint of is used as the initial representation of the pseudo unknown sample:

[0030] ;

[0031] And take all the above midpoints as candidate samples to form the initial candidate set;

[0032] S132: Genetic evolution and diversity enhancement, including:

[0033] Keep the candidate sample with the highest current fitness; perform crossover operations to generate new samples; add Gaussian noise to the candidate samples To simulate mutation:

[0034] ;

[0035] In the formula, t represents the evolutionary number of the current genetic algorithm, Indicates that the mean is zero vector and the covariance matrix is ​​σ 2 Multidimensional Gaussian distribution multiplied by the identity matrix I, In the tth generation, the initial candidate and The evolutionary process results obtained by continuously applying crossover, selection and Gaussian noise mutation operations to the offspring, that is, It is the starting point of generation 0. is the offspring expression of generation t;

[0036] Each generation of candidate samples is evaluated for fitness using the following fitness function:

[0037] ;

[0038] The further away from all known class prototypes, the higher the score, which is more likely to represent the true unknown intention; where represents the result after evolution, In the process of evolution, Indicates the number of spheres corresponding to all known classes on client c, The fitness function is expressed as the candidate sample and the centroid of all particles The minimum distance in ;

[0039] S133: Filtering and label assignment, including:

[0040] Keep all fitness above the threshold , and the pseudo samples satisfy the following conditions:

[0041] ;

[0042] That is, the pseudo sample must be outside the coverage of all spheres;

[0043] Finally, a preset proportion is selected from the retained pseudo samples according to fitness ranking The pseudo samples are constructed as pseudo unknown samples; and the category labels of the pseudo unknown samples are uniformly assigned to represent the unknown intent class. , and participate in subsequent training.

[0044] Furthermore, in step S14, the pre-trained language model is locally trained using the real sample set and the pseudo unknown sample set. After the training is completed, the obtained pre-trained language model parameters and the local granular knowledge base are uploaded to the server, including:

[0045] S141: Establish basic classification loss, including:

[0046] To the client The known class sample set is the real sample set and pseudo unknown sample set , using standard cross entropy loss for supervised learning:

[0047] ;

[0048] in Represents the category probability distribution output by the pre-trained language model, categories, of which Indicates unknown class;

[0049] S142: Establishing spherical structure constraint loss, including:

[0050] In order to enhance the intra-class compactness of known class samples, the granular structure loss is introduced , defined as follows:

[0051] ;

[0052] in For samples The nearest sphere's center of mass:

[0053] ;

[0054] S143: Establish an adaptive weighting mechanism and total loss function, including:

[0055] Introducing dynamic weight factors , adaptively adjust the uncertainty of unknown class discrimination based on the pre-trained language model:

[0056] ;

[0057] The final total loss function is defined as:

[0058] ;

[0059] S144: Use the total loss function to perform local training on the pre-trained language model. After the training is completed, the pre-trained language model parameters are obtained. and local spherical knowledge base Upload to the server.

[0060] Furthermore, in step S21, the pre-trained language model parameters uploaded by each client are aggregated to construct a global model including:

[0061] After each round of local training, the client Upload pre-trained language model parameters to the server , aggregated by weighted average method, and the global model is constructed:

[0062] ;

[0063] in For the client The number of samples, is the total number of samples, and C is the number of clients.

[0064] Furthermore, in step S22, the local granular knowledge bases uploaded by each client are aggregated to construct a global granular knowledge base, including:

[0065] S221: For public classes , if two balls from different clients and satisfy:

[0066] ;

[0067] They are considered overlapping and need to be merged into a new sphere:

[0068] ;

[0069] ;

[0070] If it does not meet the requirements, the corresponding spheres are retained directly; the merged spheres form a global public knowledge base ;

[0071] S222: For private classes , leaving each client's sphere intact and building directly ;

[0072] S223: Finally build the global granular knowledge base:

[0073] .

[0074] Furthermore, in step S31, the samples to be classified are Input into the global model to obtain global features ;

[0075] In step S32, the global feature With the global granular knowledge base Compare and implement intent classification, including:

[0076] Traverse all balls , judge the global features Whether it meets:

[0077] ;

[0078] like If it falls into at least one ball, it will be assigned the label corresponding to the nearest ball. If any sphere coverage condition is not met, the sample is judged as an unknown intent class.

[0079] The beneficial effects of the present invention are:

[0080] In an exemplary embodiment of the present invention, granular structured knowledge is constructed on the client side to achieve efficient representation learning, and the model and knowledge are aggregated on the server side to support global reasoning and open recognition. This exemplary embodiment is particularly suitable for privacy protection scenarios with multi-client, multi-source heterogeneous text data. Under the premise that user data cannot be centrally aggregated, it can still achieve accurate recognition of known intents and effective rejection of unknown intents. This method can be widely used in conversational artificial intelligence systems such as intelligent customer service systems, voice assistants, financial dialogue systems, and medical question-and-answer platforms to improve their intent understanding capabilities and security in open environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1The present invention provides a flowchart of a method for classifying open intent of federated text based on granular sphere computing in an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0082] The technical solution of the present invention is described clearly and completely below with reference to the accompanying drawings. It is apparent that the embodiments described are only a portion of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0083] In the description of the present invention, it should be noted that the directions or positional relationships indicated by terms such as "center," "up," "down," "left," "right," "vertical," "horizontal," "inside," and "outside" are based on the directions or positional relationships described in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, terms such as "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0084] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention.

[0085] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0086] See also Figure 1 , Figure 1 A flowchart of a method for classifying open intent in federated text based on granular sphere computing, provided in an exemplary embodiment of the present invention, is shown. The method includes a client training step, a server aggregation step, and a client classification step, which are completed in sequence. The client training step includes the following sub-steps:

[0087] S11: Use the pre-trained language model to extract sample features of the local training set to obtain a real sample set containing the sample feature vectors and sample category labels;

[0088] S12: Modeling the real sample set using an unsupervised sphere clustering method to obtain a first sphere set that is an approximate expression of the class distribution;

[0089] S13: constructing pseudo unknown samples based on the first ball set, and uniformly assigning the category labels of the pseudo unknown samples to the unknown intent class to obtain a pseudo unknown sample set;

[0090] S14: Locally train the pre-trained language model using the real sample set and the pseudo unknown sample set. After the training is completed, the obtained pre-trained language model parameters and are uploaded to the server as the first ball set of the local ball knowledge base;

[0091] The server-side aggregation step includes the following sub-steps:

[0092] S21: Aggregate the pre-trained language model parameters uploaded by each client to construct a global model;

[0093] S22: Aggregate the local granular knowledge bases uploaded by each client to construct a global granular knowledge base;

[0094] S23: Send the global model and global granular knowledge base to each client;

[0095] The client classification step includes the following sub-steps:

[0096] S31: Inputting the sample to be classified into the global model to obtain global features;

[0097] S32: Compare the global features with the global granular knowledge base to achieve intent classification.

[0098] Specifically, in this exemplary embodiment:

[0099] (1) First, in the client training step:

[0100] (1-1) Using a Granular-Ball Structure to Model Known Class Distributions and Implement Structured Local Representation: On the client side, this exemplary embodiment introduces the concept of granular-ball computing to abstract the representation of known classes into a number of granular balls with centers and radii. This structure effectively captures local class distribution characteristics, enables structured and irreversible information sharing without leaking the original samples, and provides high-quality structural units for subsequent knowledge aggregation and decision-making.

[0101] (1-2) Generate pseudo-unknown samples using a granular structure to improve open space simulation capabilities: Traditional pseudo-unknown sample generation often relies on shallow perturbations or random reconstructions, lacking semantic or spatial guidance and easily confusing with known classes. In this exemplary embodiment, a pseudo-unknown sample generation mechanism based on a granular structure is proposed to simulate the characteristic distribution of unknown intents. This pseudo-unknown sample generation method fully utilizes the granular structure information of known classes to effectively simulate unknown intents, improving the model's ability to discriminate unknown classes and its generalization ability in open environments.

[0102] (1-3) The pre-trained language model will then be trained locally using the real sample set and the pseudo unknown sample set. After the training is completed, the pre-trained language model parameters and the local granular knowledge base will be uploaded to the server.

[0103] (2) In the server-side aggregation step:

[0104] Proposing a granular knowledge aggregation mechanism: To achieve open classification and discrimination from a global perspective in federated scenarios, this exemplary embodiment proposes a server-side module that combines model parameter aggregation and granular structure aggregation. This module is responsible for integrating granular knowledge uploaded by each client, constructing global category boundaries, and performing non-parametric reasoning based on this knowledge. Granular structured knowledge is constructed on the client side to achieve efficient representation learning, while models and knowledge are aggregated on the server side to support global reasoning and open recognition.

[0105] (3) In the client classification step:

[0106] Utilize global granular boundaries for non-parametric reasoning to achieve open intent classification: This mechanism defines category boundaries through a multi-granular granular structure, achieving open discrimination capabilities at the sample level without the need for additional classifiers, effectively improving the performance of unknown intent detection.

[0107] In summary, this exemplary embodiment is particularly suitable for privacy protection scenarios involving multi-client, multi-source, heterogeneous text data. Even when user data cannot be centrally aggregated, it can still accurately identify known intents and effectively reject unknown intents. This method can be widely applied to conversational AI systems, such as intelligent customer service systems, voice assistants, financial dialogue systems, and medical question-and-answer platforms, improving their intent understanding capabilities and security in open environments.

[0108] It should be noted that the pre-trained language model preferably uses a shared global encoder (12-layer BERT-BaseTransformer) + a local classification head. The global model only contains a common set of encoder weights, without a decoder or classification head, and is continuously synchronized via federated averaging. Because there is no representation transmission, there is no decoder. Furthermore, the pre-trained language models on each client are of the same type before training, meaning they are all initially pre-trained BERT models.

[0109] The following content will describe the specific implementation of each exemplary embodiment:

[0110] Steps S11 and S12 model client-side spheres. To enhance the structural expressiveness of distribution modeling while maintaining user privacy, this paper designs a sphere-based known class representation method for the client's local modeling phase. This sphere structure simultaneously expresses distribution centers and coverage, effectively modeling class distributions without leaking the original data.

[0111] More preferably, in an exemplary embodiment, in step S11, the sample features of the local training set are extracted using a pre-trained language model to obtain a real sample set including the feature vectors of the samples and the category labels of the samples, including:

[0112] Client The local training set on ;in, Represents the client The number of local intent text samples, i represents the i-th sample, Represents the client The i-th intent text sample, To represent the client The category label of the i-th intent text sample;

[0113] Extraction using pre-trained language models The characteristics of the client The feature vector of the i-th intent text sample ; and further obtain the real sample set .

[0114] More preferably, in an exemplary embodiment, in step S12, the first ball set as an approximate expression of the class distribution , the jth ball Has the following properties: Particle size : The number of samples contained in the sphere; the centroid of the sphere : the mean of all sample features within the sphere; the mean radius of the sphere : the average Euclidean distance from all samples in the sphere to the center of mass; the maximum radius of the sphere : The maximum Euclidean distance from all samples in the sphere to the center of mass; the sphere radius; the sphere label : The label of the category with the highest proportion in the pellet; pellet purity :The ball belongs to the ball label The sample proportion of

[0115] The real sample set is modeled using unsupervised sphere clustering to obtain a first sphere set that approximates the category distribution, including:

[0116] If the purity of the lth ball is Below the set threshold And the sample size Exceeding the minimum threshold , then the division operation is triggered: for the particle ball , count all the label types that appear in it ; The ball is labeled Divide into several sub-spheres, each of which uses a sample within the label as the pseudo-centroid; distribute the samples in the original sphere to the sub-sphere represented by the nearest pseudo-centroid; re-evaluate the purity and size of each sub-sphere, and retain it if it meets the termination condition; otherwise, continue recursive division;

[0117] Finally, the set of all particles that meet the conditions Will be used to represent the client The known class structure on the sphere, the centroid and radius of the sphere constitute the sphere prototype of each category, which is used as an approximate expression of the category distribution under privacy protection.

[0118] Specifically, in this exemplary embodiment, Granular-Ball is an adaptive clustering unit that can characterize data distribution at multiple granularity levels. ,in is the feature vector of the sample, is the category label of the sample. By clustering the sample set, we can get a set of particles consisting of a series of balls , where each ball Depend on The attributes are as described above.

[0119] To enhance the structural expressiveness of distribution modeling while maintaining user privacy, this exemplary embodiment designs a granular sphere-based representation of known classes for use in the local modeling phase of the client. This granular sphere structure simultaneously expresses both distribution centers and coverage, effectively modeling class distributions without leaking the original data.

[0120] After obtaining a real sample set as a feature set in step S11, step S12 models the feature set based on an unsupervised sphere clustering method. This clustering process uses the entire feature set as the initial sphere and generates multiple sphere units using a recursive partitioning method. The specific strategy is as described above.

[0121] More preferably, in an exemplary embodiment, to simulate the characteristic distribution of unknown intent, a pseudo-unknown sample generation mechanism based on a sphere structure is proposed to enhance open space modeling capabilities. Specifically, in step S13, pseudo-unknown samples are constructed based on the first sphere set, and the class labels of the pseudo-unknown samples are uniformly assigned to the unknown intent class, resulting in a pseudo-unknown sample set including:

[0122] S131: Initialize candidate samples, including:

[0123] For any two balls with different labels in the first ball set , calculate the center of mass of the two The midpoint of is used as the initial representation of the pseudo unknown sample:

[0124] ;

[0125] And take all the above midpoints as candidate samples to form the initial candidate set;

[0126] S132: Genetic evolution and diversity enhancement, including:

[0127] In order to improve the diversity of pseudo unknown samples and avoid samples being concentrated in a fixed area, a genetic algorithm is introduced to evolve the candidate set. The specific process includes: retaining the candidate sample with the highest current fitness; performing a crossover operation to generate new samples; adding Gaussian noise to the candidate samples. To simulate mutation:

[0128] ;

[0129] Where t represents the evolutionary generation of the current genetic algorithm, Indicates that the mean is zero vector and the covariance matrix is ​​σ 2 Multidimensional Gaussian distribution multiplied by the identity matrix I, In the tth generation, the initial candidate and The evolutionary process results obtained by continuously applying crossover, selection and Gaussian noise mutation operations to the offspring, that is, It is the starting point of generation 0. is the offspring expression of generation t;

[0130] Each generation of candidate samples is evaluated for fitness using the following fitness function:

[0131] ;

[0132] The further away from all known class prototypes, the higher the score, which is more likely to represent the true unknown intention; where represents the result after evolution, In the process of evolution, Indicates the number of spheres corresponding to all known classes on client c, The fitness function is expressed as the candidate sample and the centroid of all particles The minimum distance in ;

[0133] S133: Filtering and label assignment, including:

[0134] Keep all fitness above the threshold , and the pseudo samples satisfy the following conditions:

[0135] ;

[0136] That is, the pseudo sample must be outside the coverage of all spheres;

[0137] Finally, a preset proportion is selected from the retained pseudo samples according to fitness ranking The pseudo samples with a value of 0.5 are constructed as pseudo unknown samples; and the category labels of the pseudo unknown samples are uniformly assigned to represent the unknown intent class. , and participate in subsequent training.

[0138] Specifically, the generation of pseudo unknown samples in this exemplary embodiment makes full use of the geometric relationship between particles and spheres, generates initial candidates based on the midpoint between the centroids of particles of different categories, and combines genetic algorithms for evolutionary search and Gaussian perturbation. It cooperates with the distance-based fitness function and boundary exclusion strategy to effectively control the position and diversity of pseudo samples, ensuring that they are located in open space and are representative, achieving effective simulation of unknown intentions, and improving the model's ability to discriminate unknown categories and generalize in open environments.

[0139] More preferably, in an exemplary embodiment, to uniformly utilize real samples and pseudo-unknown samples while simultaneously improving both discrimination and structural expression capabilities, this exemplary embodiment proposes a granular-guided training objective, including: a basic classification loss, a granular structural constraint loss, and an adaptive weighting mechanism. Specifically, in step S14, the pre-trained language model is locally trained using the real sample set and the pseudo-unknown sample set. After training, the obtained pre-trained language model parameters and the local granular knowledge base are uploaded to the server, including:

[0140] S141: Establish basic classification loss, including:

[0141] To the client The known class sample set is the real sample set and pseudo unknown sample set , using standard cross entropy loss for supervised learning:

[0142] ;

[0143] in Represents the category probability distribution output by the pre-trained language model, categories, of which Indicates unknown class;

[0144] S142: Establishing spherical structure constraint loss, including:

[0145] In order to enhance the intra-class compactness of known class samples, the granular structure loss is introduced , encourages the sample to move closer to the center of mass of the nearest sphere, which is defined as follows:

[0146] ;

[0147] in For samples The nearest sphere's center of mass:

[0148] ;

[0149] This loss is only applicable to real known class samples, and pseudo unknown samples do not participate in this structural constraint;

[0150] S143: Establish an adaptive weighting mechanism and total loss function, including:

[0151] In order to achieve a dynamic balance between compression of known classes and separation of unknown classes, a dynamic weight factor is introduced , adaptively adjust the uncertainty of unknown class discrimination based on the pre-trained language model:

[0152] ;

[0153] The final total loss function is defined as:

[0154] ;

[0155] S144: Use the total loss function to perform local training on the pre-trained language model. After the training is completed, the pre-trained language model parameters are obtained. and local spherical knowledge base Upload to the server.

[0156] Specifically, in this exemplary embodiment, through this training mechanism, each client not only obtains a pre-trained language model but also builds a structured knowledge base containing intra-class spheres, which can be subsequently used for server aggregation and inference. Furthermore, a structure-aware sphere supervision loss is designed to enhance intra-class compactness and inter-class separation. This exemplary embodiment constructs a joint loss function during the training phase. It performs standard classification training based on existing labels while also introducing a sphere supervision term to encourage similar samples to converge toward their nearest sphere centroid, thereby improving intra-class compactness and discriminative ability. Furthermore, to balance the trade-off between known recognition and unknown detection tasks, a dynamic adjustment coefficient is introduced to adaptively adjust the strength of structural supervision based on the model's confidence in the unknown sample.

[0157] In federated scenarios, to achieve open classification and discrimination from a global perspective, the following two exemplary embodiments propose a server-side module (steps S21 and S22) that aggregates model parameters and granular structure. This module is responsible for integrating granular knowledge uploaded by clients, constructing global category boundaries, and performing non-parametric reasoning based on this information.

[0158] More preferably, in an exemplary embodiment, in step S21, aggregating the pre-trained language model parameters uploaded by each client to construct a global model includes:

[0159] After each round of local training, the client Upload pre-trained language model parameters to the server , aggregated by weighted average method, and the global model is constructed:

[0160] ;

[0161] in For the client The number of samples, is the total number of samples, and C is the number of clients.

[0162] In step S22, in order to construct a global category distribution representation, the local granular knowledge bases uploaded by each client are aggregated to construct a global granular knowledge base, including:

[0163] S221: For public classes , if two balls from different clients and satisfy:

[0164] ;

[0165] They are considered overlapping and need to be merged into a new sphere:

[0166] ;

[0167] ;

[0168] If it does not meet the requirements, the corresponding spheres are retained directly; the merged spheres form a global public knowledge base ;

[0169] S222: For private classes , leaving each client's sphere intact and building directly ;

[0170] S223: Finally build the global granular knowledge base:

[0171] .

[0172] Specifically, in this exemplary embodiment, the structure provides a cross-client category structure view while ensuring data privacy, which can be directly used for open intent judgment in the reasoning stage.

[0173] More specifically, this exemplary embodiment not only uses the classic Federated Averaging (FedAvg) algorithm to fuse model parameters uploaded by clients, but also designs a sphere aggregation strategy based on geometric overlap. Public spheres are spatially merged to improve coverage and consistency, while private spheres retain their local structural features. Ultimately, a global sphere knowledge base is constructed that covers the category distribution of all clients and improves the model's generalization capabilities.

[0174] After obtaining the global model and spherical knowledge (steps S21 and S22), the following exemplary embodiment proposes a non-parametric reasoning mechanism for identifying whether a test sample belongs to a known class:

[0175] More preferably, in an exemplary embodiment, in step S31, the sample to be classified Input into the global model to obtain global features ;

[0176] In step S32, the global feature With the global granular knowledge base Compare and implement intent classification, including:

[0177] Traverse all balls , judge the global features Whether it meets:

[0178] ;

[0179] like If it falls into at least one ball, it will be assigned the label corresponding to the nearest ball. If none of the sphere coverage conditions are met, the sample is classified as an unknown intent class:

[0180] .

[0181] This mechanism defines category boundaries through a multi-granularity sphere structure, achieving open discrimination capabilities at the sample level without the need for additional classifiers, and effectively improving the performance of unknown intent detection.

[0182] The method proposed based on the above-mentioned preferred exemplary embodiment is innovative in structural modeling, knowledge aggregation and reasoning strategy, significantly enhancing the practicality and stability of the model in real-world scenarios such as multi-client, privacy protection and inconsistent distribution between classes. It has broad application value and is particularly suitable for natural language understanding systems in finance, medical care, intelligent customer service, etc. that have high privacy requirements and heterogeneous data distribution.

[0183] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will readily appreciate that other variations or modifications can be made based on the above descriptions. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications derived therefrom remain within the scope of protection of the present invention.

Claims

1. A method for classifying open intent of federated text based on granular spherical computing, characterized by: The process includes a client training step, a server aggregation step, and a client classification step, which are completed in sequence. The client training step includes the following sub-steps: S11: Use the pre-trained language model to extract sample features of the local training set to obtain a real sample set containing the sample feature vectors and sample category labels; S12: Modeling the real sample set using an unsupervised sphere clustering method to obtain a first sphere set that approximates the category distribution; S13: constructing pseudo unknown samples based on the first ball set, and uniformly assigning the category labels of the pseudo unknown samples to the unknown intent class to obtain a pseudo unknown sample set; S14: Locally train the pre-trained language model using a real sample set and a pseudo unknown sample set. After the training is completed, upload the obtained pre-trained language model parameters and the first ball set as the local ball knowledge base to the server; The server-side aggregation step includes the following sub-steps: S21: Aggregate the pre-trained language model parameters uploaded by each client to construct a global model; S22: Aggregate the local granular knowledge bases uploaded by each client to construct a global granular knowledge base; S23: Send the global model and global granular knowledge base to each client; The client classification step includes the following sub-steps: S31: Inputting the sample to be classified into the global model to obtain global features; S32: Compare the global features with the global granular knowledge base to achieve intent classification.

2. The method for classifying open intent of federated text based on granular computing according to claim 1, characterized in that: In step S11, the sample features of the local training set are extracted using the pre-trained language model to obtain a real sample set containing the sample feature vectors and the sample category labels, including: Client The local training set on ;in, Represents the client The number of local intent text samples, i represents the i-th sample, Represents the client The i-th intent text sample, To represent the client The category label of the i-th intent text sample; Extraction using pre-trained language models The characteristics of the client The feature vector of the i-th intent text sample ; and further obtain the real sample set .

3. The method for classifying open intent of federated text based on granular computing according to claim 2, characterized in that: In step S12, the first ball set as an approximate expression of the category distribution , the jth ball Has the following properties: Particle size : The number of samples contained in the sphere; the centroid of the sphere : the mean of all sample features within the sphere; the mean radius of the sphere : the average Euclidean distance from all samples in the sphere to the center of mass; the maximum radius of the sphere : The maximum Euclidean distance from all samples in the sphere to the center of mass; the sphere radius; the sphere label : The label of the category with the highest proportion in the pellet; pellet purity :The ball belongs to the ball label The sample proportion of The real sample set is modeled using unsupervised sphere clustering to obtain a first sphere set that approximates the category distribution, including: If the purity of the lth ball is Below the set threshold And the sample size Exceeding the minimum threshold , then the division operation is triggered: for the particle ball , count all the label types that appear in it ; The ball is labeled Divide into several sub-spheres, each of which uses a sample within the label as the pseudo-centroid; distribute the samples in the original sphere to the sub-sphere represented by the nearest pseudo-centroid; re-evaluate the purity and size of each sub-sphere, and retain it if it meets the termination condition; otherwise, continue recursive division; Finally, the set of all particles that meet the conditions Will be used to represent the client The known class structure on the sphere, the centroid and radius of the sphere constitute the sphere prototype of each category, which is used as an approximate expression of the category distribution under privacy protection.

4. The method for classifying open intent of federated text based on granular computing according to claim 3 is characterized by: In step S13, a pseudo unknown sample is constructed based on the first ball set, and the category labels of the pseudo unknown samples are uniformly assigned to the unknown intent class to obtain a pseudo unknown sample set, including: S131: Initialize candidate samples, including: For any two balls with different labels in the first ball set , calculate the center of mass of the two The midpoint of is used as the initial representation of the pseudo unknown sample: ； And take all the above midpoints as candidate samples to form the initial candidate set; S132: Genetic evolution and diversity enhancement, including: Keep the candidate sample with the highest current fitness; perform crossover operations to generate new samples; add Gaussian noise to the candidate samples To simulate mutation: ； Where t represents the evolutionary generation of the current genetic algorithm, Indicates that the mean is zero vector and the covariance matrix is ​​σ 2 Multidimensional Gaussian distribution multiplied by the identity matrix I, In the tth generation, the initial candidate and The evolutionary process results obtained by continuously applying crossover, selection and Gaussian noise mutation operations to the offspring, that is, It is the starting point of generation 0. is the offspring expression of generation t; Each generation of candidate samples is evaluated for fitness using the following fitness function: ； The further away from all known class prototypes, the higher the score, which is more likely to represent the true unknown intention; where represents the result after evolution, In the process of evolution, Indicates the number of spheres corresponding to all known classes on client c, The fitness function is expressed as the candidate sample and the centroid of all particles The minimum distance in ; S133: Filtering and label assignment, including: Keep all fitness above the threshold , and the pseudo samples satisfy the following conditions: ； That is, the pseudo sample must be outside the coverage of all spheres; Finally, a preset proportion is selected from the retained pseudo samples according to fitness ranking The pseudo samples are constructed as pseudo unknown samples; and the category labels of the pseudo unknown samples are uniformly assigned to represent the unknown intent class. , and participate in subsequent training.

5. The method for classifying open intent of federated text based on granular spherical computing according to claim 4 is characterized by: In step S14, the pre-trained language model is locally trained using the real sample set and the pseudo unknown sample set. After the training is completed, the obtained pre-trained language model parameters and the local granular knowledge base are uploaded to the server, including: S141: Establish basic classification loss, including: To the client The known class sample set is the real sample set and pseudo unknown sample set , using standard cross entropy loss for supervised learning: ； in Represents the category probability distribution output by the pre-trained language model, categories, of which Indicates unknown class; S142: Establishing spherical structure constraint loss, including: In order to enhance the intra-class compactness of known class samples, the granular structure loss is introduced , defined as follows: ； in For samples The nearest sphere's center of mass: ； S143: Establish an adaptive weighting mechanism and total loss function, including: Introducing dynamic weight factors , adaptively adjust the uncertainty of unknown class discrimination based on the pre-trained language model: ； The final total loss function is defined as: ； S144: Use the total loss function to perform local training on the pre-trained language model. After the training is completed, the pre-trained language model parameters are obtained. and local spherical knowledge base Upload to the server.

6. The method for classifying open intent of federated text based on granular spherical computing according to claim 5, characterized in that: In step S21, the pre-trained language model parameters uploaded by each client are aggregated to construct a global model including: After each round of local training, the client Upload pre-trained language model parameters to the server , aggregated by weighted average method, and the global model is constructed: ； in For the client The number of samples, is the total number of samples, and C is the number of clients.

7. The method for classifying open intent of federated text based on granular computing according to claim 6, characterized in that: In step S22, the local granular knowledge bases uploaded by each client are aggregated to construct a global granular knowledge base, including: S221: For public classes , if two balls from different clients and satisfy: ； They are considered overlapping and need to be merged into a new sphere: ； ； If it does not meet the requirements, the corresponding spheres are retained directly; the merged spheres form a global public knowledge base ; S222: For private classes , leaving each client's sphere intact and building directly ; S223: Finally build the global granular knowledge base: 。 8. The method for classifying open intent of federated text based on granular spherical computing according to claim 7 is characterized by: In step S31, the samples to be classified are Input into the global model to obtain global features ; In step S32, the global feature With the global granular knowledge base Compare and implement intent classification, including: Traverse all balls , judge the global features Whether it meets: ； like If it falls into at least one ball, it will be assigned the label corresponding to the nearest ball. If any sphere coverage condition is not met, the sample is judged as an unknown intent class.

Images