Global multi-source data semantic fusion insight method and system based on large language model

Abstract

The application discloses a global multi-source data semantic fusion insight method and system based on a large language model, relates to the technical field of semantic fusion, and comprises the following steps: calculating a source proportion based on a customer voice record set, calculating a co-site mixing index of an entry access point corresponding to the customer voice record set in a time slice based on the source proportion; calculating a sample number based on an entry access point identifier field and a data source type field of each customer voice record in the customer voice record set; generating a global dealiasing semantic vector based on the sample number and the co-site mixing index; generating a plurality of theme clusters based on the global dealiasing semantic vector, and calculating a sample proportion of the theme clusters. The application improves the basic accuracy of global semantic analysis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of semantic fusion, in particular to a global multi-source data semantic fusion insight method and system based on a large language model. BACKGROUND

[0002] In the process of enterprise digital operation, customers continuously generate a large amount of feedback information in the form of text through various channels. These information includes not only user opinions from the enterprise private domain touch points, but also structured feedback from sample research libraries, and also texts distributed in public network space. The above-mentioned data of different sources often enter the analysis process through a unified access path, and in the time dimension, they show a continuous arrival state, and in the space dimension, they show a centralized gathering form. With the expansion of business scale and the increase of user interaction frequency, customer voice data pours in a large amount in a short time, so that the text content has formed a complex mixed state before analysis. Enterprises hope to accurately identify the core theme of user attention based on these global multi-source text data, understand the real proportion of different sources in various themes, and further generate insight results with explainability to support decision analysis, product optimization and service improvement and other practical application scenarios.

[0003] The prior art directly performs semantic analysis and theme aggregation on the gathered text when processing multi-source customer voice text, and cannot effectively depict the source mixing degree occurring at the entrance position of the data, nor can it effectively correct the semantic bias brought by mixing. This processing method is easy to magnify the local advantage of a certain source in quantity or expression intensity, thereby forming a deviation in the semantic space, resulting in that the generated theme result cannot truly reflect the original distribution structure of the global data. Especially in the case where multi-source data is centrally entered into the analysis process through the same access point, the existing method cannot distinguish the real contribution proportion of different sources to the theme formation, and the final insight result has obvious deficiencies in accuracy and explainability. SUMMARY

[0004] The purpose of the present application is to solve the problem in the prior art that in the case where multi-source data is centrally entered into the analysis process through the same access point, the existing method cannot distinguish the real contribution proportion of different sources to the theme formation, and the present application proposes a global multi-source data semantic fusion insight method and system based on a large language model.

[0005] In order to solve the problems existing in the prior art, the present application adopts the following technical solutions:

[0006] The global multi-source data semantic fusion insight method based on a large language model comprises:

[0007] S1, calculate source proportion based on the customer voice record set, calculate co-location mixing index of the entry access point corresponding to the customer voice record set in the time slice based on the source proportion;

[0008] S2, calculate sample number based on the entry access point identifier field and the data source type field of each customer voice record in the customer voice record set;

[0009] S3, generate global de-mixing semantic vector based on the sample number and the co-location mixing index;

[0010] S4, generate multiple topic clusters based on the global de-mixing semantic vector, and calculate sample proportion of the topic clusters;

[0011] S5, generate structured insight items of the topic clusters based on the sample proportion of the topic clusters.

[0012] Preferably, the source proportion is calculated based on the customer voice record set, comprising:

[0013] Obtain the customer voice record set, wherein each customer voice record in the customer voice record set comprises a text content field, a data source type field, an entry access point identifier field, and a timestamp field;

[0014] Map the timestamp field of each customer voice record to a time slice in the time slice set;

[0015] Sum the count values of each data source type field corresponding to the same entry access point identifier field in the same time slice to obtain the total count of the entry access point identifier field in the time slice;

[0016] Divide the count values of each data source type field corresponding to the same entry access point identifier field in the same time slice by the total count of the entry access point identifier field in the time slice to obtain the source proportion.

[0017] Preferably, the co-location mixing index of the entry access point corresponding to the customer voice record set in the time slice is calculated based on the source proportion, comprising:

[0018] Multiply the source proportion and the logarithmic value of the source proportion to obtain the source type information contribution;

[0019] Sum the source type information contribution to obtain the total information contribution;

[0020] Take the negative of the total information contribution to obtain the mixing entropy of the entry access point in the time slice;

[0021] Sum all the mixing entropies to obtain the sum of the mixing entropies;

[0022] Divide the sum of the mixing entropies by the number of time slices to obtain the co-location mixing index of the entry access point.

[0023] Preferably, the sample number is calculated based on the entry access point identifier field and the data source type field in each customer voice record in the customer voice record set, including:

[0024] The text content field is subjected to semantic encoding processing to obtain a semantic vector set;

[0025] The semantic vectors in the semantic vector set are grouped according to the entry access point identifier field and the data source type field of each customer voice record to obtain a semantic vector sub-set;

[0026] The semantic vectors in the semantic vector sub-set are counted to obtain the sample number corresponding to the combination of the data source type field and the entry access point identifier field.

[0027] Preferably, the global dealiasing semantic vector is generated based on the sample number and the co-location mixing index, including:

[0028] The co-location mixing index is added to 1 to obtain a co-location mixing adjustment value;

[0029] The sample number is divided by the co-location mixing adjustment value to obtain a dealiasing weight initial value;

[0030] The sum of all dealiasing weight initial values is obtained by summing all dealiasing weight initial values to obtain a weight initial value sum;

[0031] The dealiasing weight initial value is divided by the weight initial value sum to obtain a normalized dealiasing weight;

[0032] The semantic vectors in the semantic vector sub-set are subjected to average operation to obtain a local semantic center vector;

[0033] The normalized dealiasing weight and the local semantic center vector are subjected to multiplication operation to obtain a weighted semantic vector;

[0034] The global dealiasing semantic vector is obtained by vector sum operation on the weighted semantic vector.

[0035] Preferably, the plurality of topic clusters is generated based on the global dealiasing semantic vector, including:

[0036] The semantic vector set is constructed based on the local semantic center vector and the global dealiasing semantic vector;

[0037] The semantic vector set is subjected to clustering division to obtain a plurality of topic clusters.

[0038] Preferably, the sample proportion of the topic cluster is calculated, including:

[0039] The sample number of the semantic vector sub-set in the topic cluster is counted;

[0040] The sample numbers of the sub-sets of semantic vectors in the theme cluster are classified and summarized to obtain sample numbers corresponding to the data source type fields in the theme cluster;

[0041] Based on the sample numbers corresponding to the data source type fields in the theme cluster, sample proportions corresponding to the data source type fields in the theme cluster are calculated.

[0042] Preferably, the structured insight item of the theme cluster is generated based on the sample proportion of the theme cluster, including:

[0043] Based on the sample proportion of the theme cluster and the global de-mixing semantic vector, theme semantic description information of the theme cluster is generated.

[0044] The theme identification of the theme cluster, the theme semantic description information, and the sample proportion of the theme cluster are combined to obtain the structured insight item of the theme cluster.

[0045] Preferably, the semantic vector set is clustered and divided to obtain a plurality of theme clusters, including:

[0046] The semantic distance of any two semantic vectors in the semantic vector set is calculated to obtain a semantic distance value.

[0047] Based on the semantic distance value, the semantic vectors of the semantic vector set are aggregated to obtain a plurality of semantic clustering groups.

[0048] The semantic clustering group is defined as a theme cluster.

[0049] To solve the above problems, the application also provides a global multi-source data semantic fusion insight system based on a large language model, which comprises:

[0050] A mixing index module is configured to calculate a source proportion based on the customer voice record set, and calculate a co-site mixing index of the entry access point corresponding to the customer voice record set in a time slice based on the source proportion.

[0051] A sample counting module is configured to calculate a sample number based on the entry access point identification field and the data source type field of each customer voice record in the customer voice record set.

[0052] A de-mixing semantic module is configured to generate a global de-mixing semantic vector based on the sample number and the co-site mixing index.

[0053] A theme clustering module is configured to generate a plurality of theme clusters based on the global de-mixing semantic vector, and calculate a sample proportion of the theme cluster.

[0054] An insight generation module is configured to generate a structured insight item of the theme cluster based on the sample proportion of the theme cluster.

[0055] Compared with the prior art, the application has the following advantages:

[0056] 1. By calculating the proportion of customer voice records at the entry point and time slice dimensions, and further obtaining the co-location mixing index, it is possible to quantify the degree of mixing of multi-source data at the same access location. This allows for the identification of the data mixing status carried by different entry points before semantic analysis, so that subsequent processing is no longer based on simple aggregated data distribution, but on the basis of existing understanding of the degree of source dispersion. This effectively avoids the problem of indistinguishable sources caused by centralized data import, and improves the basic accuracy of global semantic analysis.

[0057] 2. This invention combines the number of samples and the co-location mixing index to generate a global dealiasing semantic vector. By adjusting the weights of local semantic information from different sources, the sample size and mixing degree of the sources participate in the semantic fusion process simultaneously. This reduces the semantic shift caused by local quantity advantage or entry point mixing, and enables the fused semantic expression to more realistically reflect the overall semantic structure of multi-source data. This solves the defect in the prior art where the topic results are easily dominated by a single source.

[0058] 3. This invention generates topic clusters and calculates sample proportions based on global dealiasing semantic vectors, and then combines them with a large language model to generate structured insight items. This enables the final output to not only accurately present the core topics that users care about, but also clearly reflect the contribution ratio of different data sources in each topic, thereby improving the usability of the insight results and providing more reliable data support for subsequent decision analysis. Attached Figure Description

[0059] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0060] Figure 1 This is a flowchart illustrating a method for semantic fusion and insight of multi-source data across the entire domain based on a large language model, provided in an embodiment of the present invention.

[0061] Figure 2 This is a functional module diagram of a full-domain multi-source data semantic fusion insight system based on a large language model, provided in an embodiment of the present invention. Detailed Implementation

[0062] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0063] Example: This example provides a method for semantic fusion and insight of multi-source data across the entire domain based on a large language model. See [link / reference]. Figure 1, specifically, comprising:

[0064] S1, calculating a source proportion based on the customer voice record set, calculating a co-site mixing index of the entry access point corresponding to the customer voice record set in the time slice based on the source proportion;

[0065] In an embodiment of the application, the source proportion is calculated based on the customer voice record set, comprising:

[0066] Obtain a customer voice record set, wherein each customer voice record in the customer voice record set includes a text content field, a data source type field, an entry access point identification field, and a timestamp field;

[0067] Specifically, the process of obtaining the customer voice record set includes the steps of data access, parsing and regularization of external collection sources and internal collection sources. First, receive user-submitted text information from private domain touchpoints deployed by the enterprise, including comment input on application terminal interfaces, feedback records from questionnaire submission portals, and fill-in content from activity research forms. Then, read structured research sample data from a digital sample library, and collect public text content from opinion collection channels in public networks. For the original text data from the above different sources, the text part is extracted through a preset data extraction process to form an independent text content field. At the same time, in the process of data reaching the access node, according to the data source category corresponding to the source channel record, it is written into the data source type field to represent the original source channel of the record. When the data is accessed, the corresponding access location identification is generated according to its arrival access node, and the identification is written into the entry access point identification field to reflect its entry position in the real collection path. In addition, the collection time is recorded as the timestamp field at the moment when each text data is received, to describe the generation time of the record. Through the above steps, the customer text information from different channels is sorted into multiple customer voice records containing the text content field, the data source type field, the entry access point identification field and the timestamp field, thereby forming the customer voice record set.

[0068] Specifically, the customer voice record set refers to the combination of customer expression content continuously acquired and stored by the system from multiple data collection channels during the operation of the enterprise user feedback collection and analysis oriented global multi-source data processing system. These information are discrete interaction records generated between users and enterprises in physical space, such as user input text in application interface, comments posted on social platforms or answers submitted in survey questionnaires. These records are uniformly extracted and stored as structured data items. The text content field refers to the original text information in natural language form in each customer voice record, which is derived from the real expression of the user and is represented as a character sequence that can be parsed by a semantic processing module in the system. The data source type field refers to the source category to which the record belongs when it is generated, which is used to represent the attribution relationship of different collection channels in the physical world, such as data from enterprise private domain touchpoints, data from digital sample library and public text data from public domain opinions, etc. The entry access point identifier field refers to the physical access node corresponding to the customer voice record when it enters the system. These nodes are usually located in the distributed collection network and are used to represent the access location of data in the collection path.

[0069] Mapping the timestamp field of each customer voice record to a time slice in the time slice set;

[0070] Specifically, the process of mapping the timestamp field of each customer voice record to the time slice set includes the steps of parsing, comparing and interval attribution determination of time information. First, a time slice set composed of multiple continuous or discrete time intervals is established, wherein each time slice corresponds to an explicit start time and end time to carry the customer voice records generated within the time range. Then, the timestamp field of each record in the customer voice record set is read one by one to parse the year, month, day and specific time in the timestamp into comparable time units. Then, the parsed time units are sequentially compared with the start time and end time of each time slice in the time slice set. When the timestamp falls within the start and end range of a time slice, the customer voice record is attributed to the time slice. After the timestamp is matched with all time slices, each customer voice record is determined a unique time slice attribution relationship, so that the customer voice record set is discretely divided in the time dimension, providing a directly processable time structure basis for subsequent time slice based statistical calculation.

[0071] Specifically, the timestamp field refers to the specific time when the customer voice record is collected or generated, which is used to provide the time information necessary for constructing time series and identifying the timing relationship of data. The time slice set refers to a plurality of continuous or discrete time intervals pre-constructed by the system for segmentation processing, which is used to map different customer voice records to the corresponding time slices in the time dimension, so as to subsequently perform source proportion calculation, sample statistics, and co-location mixing index calculation and the like.

[0072] Summing up the count values of each data source type field corresponding to the same entry access point identification field in the same time slice, to obtain the total count of the entry access point identification field in the time slice;

[0073] Specifically, the set of customer voice records that have completed time slice mapping is read, and all records belonging to the specified time slice and having the same entry access point identification field are filtered out. Subsequently, the value of the data source type field of each filtered record is read, and the records are classified into corresponding data source type categories according to different values of the field. After classification is completed, the number of records appearing in each data source type category is counted respectively, thereby obtaining the count value of each data source type field corresponding to the entry access point identification field in the time slice. Then, the count values of all data source type fields are sequentially accumulated, and the accumulation process is executed one by one in the order from the first data source type to the last data source type, and a total count reflecting the total number of records of the entry access point identification field in the time slice is obtained after the accumulation is completed. Through the above steps, the sum calculation of the count values of each data source type field corresponding to the same entry access point identification field in the same time slice is completed, which provides basic data for subsequent proportion calculation.

[0074] Dividing the count values of each data source type field corresponding to the same entry access point identification field in the same time slice by the total count of the entry access point identification field in the time slice, to obtain the source proportion.

[0075] Specifically, the count values of each data source type field obtained in the foregoing sum calculation are read one by one, and the total count in the time slice corresponding to the entry access point identification field is read at the same time. Subsequently, the count value of each data source type field is sequentially taken as the dividend, and the total count is taken as the divisor, to perform division operation, thereby obtaining the proportion value of each data source type field in the time slice. After all data source type fields complete the division calculation, each proportion value is recorded item by item according to the different values of the data source type field, so that each proportion value can establish a clear corresponding relationship with the corresponding data source type field. The proportion value is used to represent the contribution degree of different source types of data to the total amount of data of the entry access point under the condition of the same entry access point identification field and the same time slice.

[0076] Specifically, the same time slice refers to a continuous time interval obtained by dividing a fixed time length in the data collection process, which is used to carry all customer voice records generated in the time range, so that the system can realize discrete statistics in the time dimension. The same entry access point identification field refers to the identification information of the physical access node through which the data enters the collection system. These access nodes are usually collection terminals deployed at the network edge or service interface, and are used to represent the physical source path of different data streams. The data source type field refers to the data source category corresponding to the customer voice record when it is generated, which reflects the difference between the data collection channels in the real deployment structure, including enterprise private domain touchpoints, digital sample library, public domain opinion collection system and other sources. The count value refers to the discrete numerical value obtained by the system after counting the number of customer voice records meeting the given conditions, which is used to represent the number of records of a specific entry access point and a specific data source type in a time slice. The total count of the entry access point identification field in the time slice refers to the summation result of the record number corresponding to all data source types in the same time slice range. The total count is used to express the total data amount carried by the entry access point in the specified time slice.

[0077] Specifically, the source ratio refers to the proportion of the number of records of a certain data source type in a certain entry access point and a certain time slice relative to the total number of records of the entry access point in the time slice, which is used to reflect the contribution degree of different source data in the access point.

[0078] In an embodiment of the present application, the co-site mixing index of the entry access point corresponding to the customer voice record set in the time slice is calculated based on the source ratio, comprising:

[0079] The product operation is performed on the source ratio and the logarithmic value of the source ratio to obtain the source type information contribution amount;

[0080] The summation is performed on the source type information contribution amount to obtain the total information contribution amount;

[0081] Specifically, the logarithmic value of the source ratio refers to the value obtained by logarithmic transformation of the aforementioned source ratio, which is used to amplify the difference between the ratios, so that the subsequent mixed entropy calculation can more sensitively reflect the influence of source difference. The source type information contribution amount refers to the value obtained by multiplying the source ratio and the logarithmic value of the source ratio, which reflects the individual contribution of a certain data source type to the overall mixing degree under the condition of the entry access point and the time slice. The total information contribution amount refers to the cumulative value obtained by summing all source type information contribution amounts, which is used to reflect the overall information mixing degree formed by different source data at the position of the entry access point.

[0082] The negative operation is performed on the total information contribution amount to obtain the mixed entropy of the entry access point in the time slice.

[0083] summing up all the mixed entropies to obtain a sum value of the mixed entropies;

[0084] Specifically, the implementation process of the mixed entropy of the entry access point in the time slice includes continuous operations of numerical reading, sign conversion and result registration by taking the negative operation on the total information contribution. First, the total information contribution obtained by summing up all the source type information contributions in the same time slice is read, and the total information contribution is input as a to-be-processed numerical value into the numerical conversion process. Then, according to the definition requirement of the entropy function, the sign of the numerical value is inverted to convert it from a positive value to a corresponding negative value, thereby forming a mixed entropy value capable of representing the source balance degree. Next, the mixed entropies obtained in multiple time slices are read in sequence, and are sequentially accumulated according to the time slice order, so that the mixed entropy of each time slice participates in the accumulation process. After the accumulation is completed, a sum value of the mixed entropies for representing the overall source mixing degree is formed. Through the above continuous steps, the time slice-level mixed entropy calculation and the cross-time slice mixed entropy accumulation processing are completed, thereby providing stable basic data for subsequent co-location mixing index calculation.

[0085] dividing the sum value of the mixed entropies by the number of time slices to obtain the co-location mixing index of the entry access point.

[0086] Specifically, the mixed entropy refers to the result obtained by taking the negative value of the total information contribution, and is used to represent the data source diversity and balance degree of the entry access point in the time slice. The greater the value is, the more dispersed the source distribution is. The sum value of the mixed entropies refers to the value obtained by summing up the mixed entropies of multiple time slices respectively in the analysis period, and is used to reflect the overall mixed value of the entry access point in the entire time span.

[0087] Specifically, the co-location mixing index of the entry access point refers to an average index obtained by quantifying the data source mixing degree of the entry access point in multiple time slices. The index is calculated by the proportion of the record number of each time slice in different data source types, and the mixed entropy values of the time slices are obtained based on the information entropy principle. Then, the mixed entropy values are accumulated and divided by the number of time slices participating in the analysis. The index is used to reflect whether the data sources received by the entry access point in a long time span are balanced. If the value is large, it indicates that the entry access point receives data from multiple sources in a long time and the source distribution is relatively dispersed. If the value is small, it indicates that the data source concentration is high, mainly from a small number of data source types. The index provides a basis for subsequent data fusion and semantic dealiasing, so that the high-mixing access point and the low-mixing access point can be distinguished in the processing strategy.

[0088] Specifically, the calculation process can obtain the co-site mixing index of the entry access point because the calculation method follows the common principle of information mixing phenomenon and energy distribution law. The information contribution quantity constructed by the product of the source proportion and its logarithmic value can depict the distribution discrete degree of different data source types at the same entry access point. The product of the source proportion and its logarithmic value reflects the diffusion state of information between different sources, which has the same mathematical structure as the entropy function used to describe the particle distribution imbalance in thermodynamics. Therefore, the summation of the contribution amounts of all sources can obtain the total contribution amount reflecting the discrete degree of the overall sources. On this basis, the negative value of the total contribution amount is taken to obtain the mixing entropy. The greater the mixing entropy is, the more uniform the sources are, and the smaller the mixing entropy is, the more concentrated the sources are. Further, the mixing entropies of multiple time slices are summed and normalized by the number of time slices to make the source mixing conditions of different time periods comparable, thereby generating the co-site mixing index which can stably represent the strength of source diversity of a certain entry access point in a long-term range. The ability of the index to reflect the source mixing degree comes from its same mathematical properties as the entropy function, so it can be used as a reliable source mixing measurement index.

[0089] Specifically, in the case where the voices of multiple types of customers from private domain channels, digital sample libraries and public domain speeches are unified and gathered at the same entry access point, due to the combined effects of factors such as link queuing, buffer accumulation, batch forwarding and burst traffic of different sources, the data of each source at the entry access point presents a significant mixed state, which makes the time distribution and quantity proportion inconsistent with the real structure of the sources. At this time, the semantic expression at the entry access point will lose the difference between sources, which is manifested as the emotional style, topic popularity or expression manner of a certain source being excessively amplified, thereby covering up the real semantic features of other sources, and finally leading to the spatial semantics being incorrectly gathered as a unified theme, forming semantic aliasing bias. In order to identify the mixing distortion degree generated by the entry access point, it is necessary to calculate the co-site mixing index of the entry access point to quantify the mixing degree of different data sources at the entry access point into a comparable numerical value, so that in the subsequent semantic processing and fusion link, a differentiated dealiasing strategy is adopted for the entry with serious mixing to avoid the deviation of the global semantic insight result caused by the mixing effect at the entry.

[0090] S2, calculating the sample number based on the entry access point identifier field and the data source type field of each customer voice record in the customer voice record set;

[0091] In the embodiment of the application, the calculation of the sample number based on the entry access point identifier field and the data source type field of each customer voice record in the customer voice record set comprises:

[0092] performing semantic encoding processing on the text content field to obtain a semantic vector set;

[0093] Specifically, the text content field in the customer voice record is read item by item, and meaningless characters, repeated spaces, irregular symbols and other interference contents in the text are removed to make the text meet the standard format required for semantic processing. Subsequently, the normalized text is input into the semantic coding process, and the language expression in the text is converted into a semantic vector composed of multiple dimensions of values through word segmentation, sentence structure analysis and context semantic extraction. Then, the semantic vector corresponding to each piece of text is stored in sequence to form a semantic vector set composed of multiple semantic vectors, so that subsequent semantic analysis and grouping processing can be carried out in the form of vectors.

[0094] According to the entry access point identifier field and the data source type field of each customer voice record, the semantic vectors in the semantic vector set are grouped to obtain a semantic vector sub-set;

[0095] Specifically, the entry access point identifier field and the data source type field of each customer voice record are read one by one to determine the access location and source category corresponding to the record. Subsequently, the above fields are compared with the source information of each semantic vector in the semantic vector set one by one, and the semantic vectors with the same entry access point identifier field and the same data source type field are filtered out and classified according to the combination values of the two fields as the classification basis. By sequentially performing the same filtering and classification actions on all semantic vectors, multiple semantic vector sub-sets corresponding to different combinations of entry access points and data sources can be generated, and each sub-set represents a semantic expression fragment under specific source conditions.

[0096] The number of semantic vectors in the semantic vector sub-set is counted to obtain the sample number corresponding to the combination of the data source type field and the entry access point identifier field.

[0097] Specifically, each semantic vector sub-set is traversed item by item, and the semantic vectors contained in the sub-set are read one by one in the order of vector appearance, and the count value is increased by one each time a semantic vector is read. Subsequently, after the traversal of all semantic vectors is completed, the cumulative value is taken as the sample number of the semantic vector sub-set to represent the number of records of the combination of the entry access point and the data source type in actual data collection. Finally, the sample number is established in explicit correspondence with the entry access point identifier field and the data source type field corresponding to the sub-set, thereby obtaining the data basis for subsequent semantic statistics or weight calculation.

[0098] Specifically, the semantic vector set refers to a combination of all vector data obtained after semantic encoding processing, used to describe the semantic difference and semantic association between different customer expressions. The semantic vector subset refers to a vector subset obtained by grouping the semantic vector set according to the entry access point identification field and the data source type field in the customer voice record, each subset corresponding to the joint condition of a certain entry access point and a certain data source, reflecting the semantic expression characteristics under the source condition. The sample number refers to the number of semantic vectors contained in a certain semantic vector subset, used to quantify the record size of the combination of the entry access point and the data source type in the real collection process. The entry access point identification field refers to the identification assigned to the data when it enters the access node in the collection path, used to represent the entry source of the data to the physical collection location.

[0099] S3, generating a global dealiasing semantic vector based on the sample number and the co-location mixing index;

[0100] In an embodiment of the present application, generating a global dealiasing semantic vector based on the sample number and the co-location mixing index comprises:

[0101] adding 1 to the co-location mixing index to obtain a co-location mixing adjustment value;

[0102] dividing the sample number by the co-location mixing adjustment value to obtain a dealiasing weight initial value;

[0103] summing all the dealiasing weight initial values to obtain a weight initial value sum;

[0104] dividing the dealiasing weight initial value by the weight initial value sum to obtain a normalized dealiasing weight;

[0105] Specifically, the dealiasing weight initial value refers to a value obtained by dividing the sample number of a certain entry access point source by its corresponding co-location mixing adjustment value, used to represent the initial estimate of the contribution degree of the source to the real semantics under the condition of spatial mixing, so that the source with larger sample number and lower mixing degree obtains a higher share of influence.

[0106] Specifically, the co-location mixing adjustment value refers to a value obtained by adding a fixed increment to the original co-location mixing index, which is used to prevent the subsequent calculation from failing when the index is zero, and can maintain the relative difference of the mixing degree in the calculation process, thereby reflecting the basic adjustment ability of the entry access point in the long-term source mixing state. The weight initial value sum refers to the cumulative value obtained by summing the demixing weight initial values of all sources, which is used to reflect the overall contribution of all sources after mixing adjustment. The normalized demixing weight refers to the proportion value obtained by dividing the demixing weight initial value of each source by the weight initial value sum, which is used as the weighting coefficient of the semantic vector of different sources in semantic fusion, so that the contribution relationship between sources can be compared on a unified scale, and the weighted result is ensured to be numerically stable as a whole, thereby providing a reliable weight basis for subsequent global semantic fusion.

[0107] Specifically, after obtaining the demixing weight initial value by dividing the sample number by the co-location mixing adjustment value, the demixing weight initial values of all sources are summed and used as the normalization reference, and then each demixing weight initial value is divided by the reference to obtain the normalized demixing weight, because this calculation process follows the common processing rule of physical quantity normalization, which converts a group of values with different dimensions or different scales into proportional quantities that can be compared with each other through total quantization, thereby ensuring that the sum of the weights of all sources is constant, similar to scaling the contribution of multiple forces according to the total force to obtain the contribution ratio of each force. The weight obtained in this way can reflect the relative contribution of each source in the overall semantic recovery process, so that the weight can not only retain the scale information of the original sample number, but also offset the deviation caused by co-location mixing, thereby obtaining stable and interpretable normalized demixing weights that can be used for weighted fusion calculation.

[0108] averaging the semantic vectors in the semantic vector sub-set to obtain a local semantic center vector;

[0109] Specifically, after obtaining the semantic vector sub-set, each semantic vector in the set is taken out one by one, and the values of each dimension are accumulated according to the fixed dimension order to form a dimension-by-dimension cumulative sum. After accumulating all semantic vectors, the number of vectors in the semantic vector sub-set is recorded, and each dimension value in the cumulative sum is divided by the vector number, thereby obtaining a new vector composed of average numbers. All dimensions of the new vector are the average of the semantic vectors of the corresponding dimension in the same set, which is used to express the typical position of the set in the semantic space, and serves as the local semantic center vector corresponding to the combination of the data resource field and the entry access point identification field.

[0110] performing multiplication operation on the normalized demixing weight and the local semantic center vector to obtain a weighted semantic vector;

[0111] After obtaining the normalized dealiasing weight and the corresponding local semantic center vector, each dimension value of the local semantic center vector is taken out respectively, and each dimension value is multiplied by the normalized dealiasing weight in the order of vector dimensions to obtain a set of dimension-by-dimension multiplication results, thereby forming a new weighted semantic vector. Each dimension value of the weighted semantic vector represents the contribution share of the local semantic center vector in the global semantic recovery process, and the size is controlled by the normalized dealiasing weight, so that the source with lower mixing degree or larger sample quantity has more significant semantic influence after weighting.

[0112] The weighted semantic vectors are subjected to vector summation operation to obtain a global dealiasing semantic vector.

[0113] Specifically, after obtaining the weighted semantic vectors corresponding to the data resource fields of all sources, each weighted semantic vector is taken out in turn, and the same dimension values of the weighted vectors are subjected to dimension-by-dimension accumulation operation in a fixed dimension order. After traversing all the weighted semantic vectors, the accumulated multi-dimensional values are combined into a new vector, which is the global dealiasing semantic vector. The vector integrates the weighted semantic information of all sources and can reflect the unified semantic expression formed by different sources after mixing adjustment, which is used to describe the global semantic features recovered under the mixed situation of entry access points.

[0114] Specifically, the local semantic center vector refers to a vector obtained by averaging the semantic vectors belonging to the same entry access point and the same data source type, which is used to represent the typical semantic expression position of the entry access point under the data source type. The weighted semantic vector refers to a vector obtained by multiplying the local semantic center vector by the normalized dealiasing weight of the corresponding source, which reflects the relative contribution strength of the source semantic expression in the global fusion process, so that the semantics of different sources can participate in fusion with different amplitudes according to the influence difference. The global dealiasing semantic vector refers to a fusion vector obtained by vector summation of the weighted semantic vectors of all sources, which is used to represent the overall semantic trend position of multiple sources after mixing correction, so that the semantic bias caused by the mixing of entry access points is compensated, thereby obtaining a unified expression representing the global multi-source semantic rule.

[0115] Specifically, the global dealiasing semantic vector refers to an overall semantic vector obtained by multiplying the local semantic center vectors of all sources by the corresponding normalized dealiasing weight and then performing vector summation on the obtained weighted semantic vectors, which is used to reflect the unified semantic expression result after multi-source fusion, thereby compensating for the mixing effect of the entry access point and recovering the real global semantic structure.

[0116] Specifically, since the customer voice data comes from three sources of private domain, sample library and public domain, and the data of the three sources will experience processes such as queuing, caching and forwarding on the link of the same entry access point before entering the semantic analysis, the semantic expressions of different sources overlap in time sequence and quantity distribution, thereby producing a mixed phenomenon that the source is invisible. The mixed phenomenon can cause abnormal amplification or weakening of high-frequency expressions of a source in the semantic space, form spatial aliasing semantic bias, and cause the topics, emotions or hotspots obtained by analysis to be incorrectly attributed to the source that is not the real dominant source. In order to eliminate the spatial level interference caused by the co-sited mixing of the entry access point, it is necessary to adjust the local semantic information of different sources according to the mixing degree thereof, and to weight and fuse the adjusted semantic vectors in the global range to obtain a global demixing semantic vector that can represent the contribution proportion of the real source. The vector can restore the real semantic structure that is covered by the mixing in the three-source co-sited environment, so that the semantic insight formed finally is no longer affected by the spatial structure distortion of the entry access point, thereby accurately reflecting the real distribution and dominant content of the global customer voice.

[0117] S4, generating a plurality of topic clusters based on the global demixing semantic vector, and calculating a sample proportion of the topic clusters;

[0118] In the embodiments of the present application, generating a plurality of topic clusters based on the global demixing semantic vector comprises:

[0119] constructing a semantic vector set based on the local semantic center vector and the global demixing semantic vector;

[0120] Specifically, after obtaining the local semantic center vector and the global demixing semantic vector, first, each local semantic center vector is sequentially read, and the vector is taken as a local semantic element that can represent the semantic characteristics of the corresponding entry access point and data source, and then the global demixing semantic vector is read, and it is taken as a global semantic element that can reflect the overall semantic distribution trend. Subsequently, all local semantic center vectors are sequentially added to a set structure for storing semantic vectors, so that the set contains a plurality of local semantic expression units; then, the global demixing semantic vector is added to the same set structure, so that the set contains semantic vectors for describing local characteristics and semantic vectors for describing overall semantic trends.

[0121] performing clustering and division on the semantic vector set to obtain a plurality of topic clusters.

[0122] In the embodiments of the present application, performing clustering and division on the semantic vector set to obtain a plurality of topic clusters comprises:

[0123] performing semantic distance calculation on any two semantic vectors in the semantic vector set to obtain a semantic distance value;

[0124] Specifically, for any two semantic vectors in the semantic vector set, first, a vector is sequentially read from the semantic vector set as a target vector, and another vector different from the target vector is read from the set as a comparison vector. Then, the target vector and the comparison vector are compared one by one according to the corresponding dimensions, the vector component difference of each dimension is calculated respectively, the differences of all dimensions are squared and summed to obtain a total sum of squares, and a square root operation is performed on the sum of squares to obtain a semantic distance value between the two vectors. By repeating the above process, each pair of vectors in the semantic vector set obtains a corresponding semantic distance value, thereby forming a semantic distance result covering all vector pairs, providing a quantifiable semantic difference basis for subsequent aggregation operations.

[0125] Based on the semantic distance value, the semantic vectors in the semantic vector set are aggregated to obtain a plurality of semantic clustering groups;

[0126] Specifically, after obtaining the semantic distance values covering all vector pairs, first, the mutual proximity relationship between any two semantic vectors is determined, and the vector pairs with smaller semantic distance values are classified into the same initial aggregation unit, so that these vectors exhibit close semantic expressions in the semantic space. Subsequently, for the formed initial aggregation unit, any one vector in the unit is taken as a center vector, and then the semantic distance between the remaining semantic vectors outside the aggregation unit and the center vector is sequentially judged whether it is lower than the aggregation condition formed by the distance judgment mode adopted in the aggregation operation. If the aggregation condition is met, the vector is added to the aggregation unit, and the above steps are continued until there is no new vector meeting the aggregation condition. By continuously repeating the above aggregation process, the semantic vector set is split into a plurality of semantic clustering groups with internal closeness and external difference. Each semantic clustering group exhibits a natural collection structure with similar semantic expressions and serves as a basic semantic unit for subsequent topic cluster generation.

[0127] The semantic clustering group is defined as a topic cluster.

[0128] Specifically, in the semantic vector space, any semantic vector can be regarded as a semantic coordinate point representing the voice content of a certain customer, and if the distance of multiple vectors in the space is significantly close, it indicates that these contents have high consistency in expressing intention, emotional tendency or focus. The semantic clustering group obtained by aggregating the semantic vectors essentially forms a number of vector groups that are closely connected semantically, and the similarity within the group is much higher than that between the groups. This structure is consistent with the law that particles with common attributes in nature form stable cluster distribution in the potential energy minimum area. In order to convert the stable distribution structure in the semantic space into an interpretable analysis unit, the semantic clustering group needs to be assigned a clear semantic label so that it can be used for subsequent insight analysis. Therefore, defining the semantic clustering group as a topic cluster can make each cluster represent a relatively clear semantic topic, so that data from different sources can be organized, presented and interpreted according to the real focus after fusion, thereby forming a topic result that can be used for business insight, avoiding the dispersion and isolation of semantic fragments in analysis, and ensuring that the final insight result has aggregation, readability and interpretability.

[0129] Specifically, the semantic vector set refers to the combination of all semantic vectors extracted from the customer voice text, which is used to represent the differences between different text expressions in the semantic space; the semantic distance value refers to the difference measurement result between two semantic vectors, which is usually reflected by the directional difference or similarity change between the vectors, and is used to reflect the closeness of semantic content; the semantic clustering group refers to the vector set formed by merging the semantic vectors close to each other into the same category based on the semantic distance value, which can be used to identify the natural distribution structure of similar semantics; the topic cluster refers to the conceptual semantic group further defined by the semantic clustering group, which is used to describe the semantic topic that can represent a type of customer focus content.

[0130] In an embodiment of the present application, the sample proportion of the topic cluster is calculated, comprising:

[0131] The sample number of the sub-set of semantic vectors in the topic cluster is counted;

[0132] Specifically, first, each topic cluster is sequentially read, and all semantic vectors contained in the topic cluster are identified one by one. Then for each semantic vector, its sample identifier is read, and the number of identified samples is accumulated one by one, so that each topic cluster obtains a corresponding sample total number. After all semantic vectors are counted, an independent sample number result is formed for each topic cluster, which is used to represent the size of the semantic content covered by the topic cluster and serves as the basic number information for subsequent data source classification statistics.

[0133] The sample number of the sub-set of semantic vectors in the topic cluster is classified and summarized to obtain the sample number corresponding to each data source type field in the topic cluster;

[0134] Specifically, first, the data source type field corresponding to each semantic vector is read, and semantic vectors belonging to the same data source type field are classified into the same classification unit. Then, the number of semantic vectors in each classification unit is counted, and the number of samples corresponding to each semantic vector in the classification unit is added one by one, so that each classification unit obtains a number value representing the sample size. Through the above classification and accumulation operations, each topic cluster obtains the number of samples from different data source type fields, which is used to represent the difference in participation of different sources in the topic semantics.

[0135] Based on the number of samples corresponding to each data source type field in the topic cluster, the sample proportion of each data source type field in the topic cluster is calculated.

[0136] Specifically, first, the total number of samples in each topic cluster is read, and then the number of samples corresponding to any data source type field in the topic cluster is read. Then, the number of samples of the data source type field is taken as the numerator, and the total number of samples of the topic cluster is taken as the denominator, and a ratio operation is performed to obtain the sample proportion of the data source type field in the topic cluster. By performing the above ratio operation on all data source type fields one by one, a complete sample proportion result is formed for each topic cluster to reflect the relative contribution of different sources in the topic semantics and provide a quantitative basis for subsequent source difference analysis and insight generation.

[0137] Specifically, the number of samples of the subset of semantic vectors in the topic cluster refers to the number of samples obtained by counting all semantic vectors belonging to the same topic cluster according to their source texts, which can be used to represent the size distribution of the semantic content covered by the topic cluster; the number of samples corresponding to each data source type field in the topic cluster refers to the statistical value obtained by further identifying the data source type corresponding to each semantic vector in the topic cluster and accumulating the number of semantic vectors belonging to the same data source type, which reflects the participation degree of different external sources in the semantic space of the topic cluster; the sample proportion corresponding to each data source type field in the topic cluster refers to the proportion value obtained by performing a ratio operation on the number of samples of the same data source type and the total number of samples of the topic cluster, which can reflect the relative contribution of the data source in the topic cluster, so that the topic cluster can show the real semantic structure from different sources when analyzed.

[0138] S5, generating a structured insight item of the topic cluster based on the sample proportion of the topic cluster.

[0139] In an embodiment of the present application, the structured insight item of the topic cluster is generated based on the sample proportion of the topic cluster, comprising:

[0140] The topic semantic description information of the topic cluster is generated based on the sample proportion of the topic cluster and the global dealiasing semantic vector.

[0141] Specifically, first, the global dealiasing semantic vector corresponding to the topic cluster is read, and the vector is taken as the basic expression of the overall semantic trend of the topic. Then, the sample proportions of the data source type fields in the topic cluster are read, and the data source with a high proportion is taken as the main semantic contribution direction of the topic, and the data source with a low proportion is taken as auxiliary semantic information. Next, the dimension semantic components in the global dealiasing semantic vector are compared with the semantic features of the main contribution source, the semantic keywords reflecting the main meaning of the topic are extracted, and these keywords are organized into coherent semantic description sentences. Then, combined with the proportion sorting structure, the semantic features from different sources and having supplementary meaning to the topic are added to the description information, so that the topic semantic description can fully reflect the main semantic trend and source difference of the topic cluster.

[0142] The structured insight item of the topic cluster is obtained by combining the topic identification of the topic cluster, the topic semantic description information, and the sample proportion of the topic cluster.

[0143] Specifically, to combine the topic identification of the topic cluster, the topic semantic description information, and the sample proportion of the topic cluster, first, the topic identification of each topic cluster is read, which is taken as the semantic title of the topic cluster. Then, the text content associated with all semantic vectors contained in the topic cluster is organized, and a large language model is used to perform semantic summarization operation. By inputting the high-frequency semantic keywords, representative text segments, and topic identification in the topic cluster into the large language model, the large language model generates a complete semantic explanation text according to the input content. The explanation text is used to describe the core meaning reflected by the topic cluster, the semantic category involved, and the possible user focus. Then, the ratio of the sample number of the topic cluster to the total sample number of all topic clusters is calculated to obtain a value reflecting the proportion of the topic, and the value is converted into a proportion format for easy reading, which is used to represent the importance and coverage of the topic, so that the subsequent insight result has statistical support.

[0144] Specifically, after the above basic information is prepared, first, a fixed content framework of the structured insight item is set, including three parts of a theme title field, a theme semantic description field and a theme proportion field; then the theme identifier is filled into the theme title field in sequence, the semantic description text generated by the large language model is filled into the theme semantic description field, and the calculated theme cluster sample proportion is filled into the theme proportion field; then the content of the three fields is subjected to format normalization processing, including unifying the text style, adjusting the sentence connection, keeping the overall expression clear, so that the finally generated content can be presented in the form of clear structure, complete semantics and strong readability; finally, the above three fields are combined into a complete structured insight item in a predetermined order, so that the item can simultaneously show the core meaning of the theme, the specific semantic explanation and the proportion of the theme in the corpus, thereby providing a directly usable structured insight result for subsequent semantic analysis, business judgment or trend observation.

[0145] Specifically, the sample proportion of the theme cluster refers to the proportional relationship between the number of samples belonging to a certain data source type in the same theme cluster and the total number of samples in the theme cluster, which can reflect the participation strength of different data sources in the theme semantic structure; the theme semantic description information of the theme cluster generated by the global dealiasing semantic vector refers to the descriptive information used to represent the core semantic content of the theme obtained by jointly analyzing the dealiasing semantic vector representing the overall semantic trend and the semantic vector inside the theme cluster, which is used to reflect the main semantic direction of the theme cluster; the theme identifier of the theme cluster refers to the label formed by extracting the main semantic content of the theme cluster, which is used to provide an explicit semantic name for the theme cluster; the structured insight item of the theme cluster refers to arranging multiple semantic dimensions into an itemized output form according to the theme identifier, the theme semantic description information and the sample proportion in the theme cluster, thereby constructing a structured expression result that can reflect semantic topics, source contributions and trend information, which is used to support subsequent business insight and analysis.

[0146] Specifically, the large language model refers to a deep learning model trained using large-scale corpus, which can understand semantics, reason semantic relationships and generate natural language text. Its internal structure encodes the semantic features of the input text through multiple layers of neural networks, and can identify, associate and abstract the meanings of words, sentences and paragraphs based on the language rules learned in training; when the semantic keywords, representative text fragments or other semantic information of the theme cluster are input into the model, the large language model can generate coherent, accurate and semantically logical explanatory text according to the language patterns mastered inside, thereby used to construct the semantic description information of the theme cluster.

[0147] Specifically, to obtain the theme identifier of a theme cluster, firstly, all semantic vector subsets constituting the theme cluster are read, and the text content corresponding to each semantic vector is subjected to unified semantic feature extraction to obtain a set of semantic keywords that can represent the core meaning of the text. Then, all semantic keywords within the theme cluster are statistically analyzed according to their frequency of occurrence, and keywords with high frequency that can represent the overall semantic trend of the theme are selected as candidate theme words. Next, semantic similarity comparison is performed on the candidate theme words, and words with similar semantic meanings are merged and categorized to form several word groups with relatively concentrated semantic meanings. Then, representative words for each word group are selected, with the selection principle being that they can both highly summarize the meaning of the semantic word group and accurately indicate the overall semantic direction of the theme cluster. Next, multiple representative words are sorted from high to low semantic importance, so that the top-ranked representative words become the core expression of the theme cluster. Finally, the sorted representative words are combined into a concise semantic phrase according to a fixed format, so that the semantic phrase can accurately summarize the semantic content of the theme cluster, thus serving as the theme identifier of the theme cluster. Through the above steps, the obtained theme identifier is both representative and clearly reflects the semantic distribution characteristics within the theme cluster.

[0148] like Figure 2 The diagram shown is a functional block diagram of a multi-source data semantic fusion insight system based on a large language model provided in an embodiment of the present invention.

[0149] In this embodiment, the functions of each module / unit are as follows:

[0150] The Hybrid Index module is used to calculate the source ratio based on the customer voice record set, and to calculate the co-location hybrid index of the entry access point corresponding to the customer voice record set within the time slice based on the source ratio.

[0151] The sample counting module is used to calculate the number of samples based on the entry point identifier field and the data source type field of each customer voice record in the customer voice record collection;

[0152] The demixing semantic module is used to generate a global demixing semantic vector based on the number of samples and the co-address mixing index.

[0153] The topic clustering module is used to generate multiple topic clusters based on global dealiasing semantic vectors and calculate the sample proportion of topic clusters;

[0154] The insight generation module is used to generate structured insight entries for topic clusters based on the sample proportion of topic clusters.

[0155] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for global multi-source data semantic fusion insight based on a large language model, characterized in that, The method comprises the following steps: S1, calculating a source proportion based on a customer voice record set, and calculating a co-site mixing index of an entry access point corresponding to the customer voice record set in a time slice based on the source proportion; The specific steps of calculating the co-site mixing index are as follows: performing a product operation on the source proportion and the logarithmic value of the source proportion to obtain a source type information contribution amount; performing summation on the source type information contribution amount to obtain a total information contribution amount; performing a negative operation on the total information contribution amount to obtain a mixing entropy of the entry access point in the time slice; performing summation on all the mixing entropies to obtain a summation value of the mixing entropies; dividing the summation value of the mixing entropies by the number of the time slices to obtain the co-site mixing index of the entry access point; S2, calculating a sample number based on the entry access point identifier field and the data source type field of each customer voice record in the customer voice record set; S3, generating a global de-mixing semantic vector based on the sample number and the co-site mixing index; The specific steps of generating the global de-mixing semantic vector are as follows: adding 1 to the co-site mixing index to obtain a co-site mixing adjustment value; dividing the sample number by the co-site mixing adjustment value to obtain a de-mixing weight initial value; performing summation on all the de-mixing weight initial values to obtain a weight initial value total sum; dividing the de-mixing weight initial value by the weight initial value total sum to obtain a normalized de-mixing weight; performing an average operation on the semantic vectors in the semantic vector sub-set to obtain a local semantic center vector; performing a multiplication operation on the normalized de-mixing weight and the local semantic center vector to obtain a weighted semantic vector; performing vector summation operation on the weighted semantic vector to obtain the global de-mixing semantic vector; S4, generating a plurality of theme clusters based on the global de-mixing semantic vector, and calculating a sample proportion of the theme clusters; S5, generating a structured insight item of the theme cluster based on the sample proportion of the theme cluster.

2. The global multi-source data semantic fusion insight method based on a large language model according to claim 1, characterized in that, The method comprises the following steps: S1, calculating a source proportion based on a customer voice record set, and calculating a co-site mixing index of an entry access point corresponding to the customer voice record set in a time slice based on the source proportion; The specific steps of calculating the co-site mixing index are as follows: performing a product operation on the source proportion and the logarithmic value of the source proportion to obtain a source type information contribution amount; performing summation on the source type information contribution amount to obtain a total information contribution amount; 3. The global multi-source data semantic fusion insight method based on a large language model according to claim 2, characterized in that, performing a negative operation on the total information contribution amount to obtain a mixing entropy of the entry access point in the time slice; performing summation on all the mixing entropies to obtain a summation value of the mixing entropies; dividing the summation value of the mixing entropies by the number of the time slices to obtain the co-site mixing index of the entry access point; S2, calculating a sample number based on the entry access point identifier field and the data source type field of each customer voice record in the customer voice record set; 4. The global multi-source data semantic fusion insight method based on a large language model according to claim 3, characterized in that, S3, generating a global de-mixing semantic vector based on the sample number and the co-site mixing index; The specific steps of generating the global de-mixing semantic vector are as follows: adding 1 to the co-site mixing index to obtain a co-site mixing adjustment value; dividing the sample number by the co-site mixing adjustment value to obtain a de-mixing weight initial value; performing summation on all the de-mixing weight initial values to obtain a weight initial value total sum; dividing the de-mixing weight initial value by the weight initial value total sum to obtain a normalized de-mixing weight; performing an average operation on the semantic vectors in the semantic vector sub-set to obtain a local semantic center vector; performing a multiplication operation on the normalized de-mixing weight and the local semantic center vector to obtain a weighted semantic vector; performing vector summation operation on the weighted semantic vector to obtain the global de-mixing semantic vector; S4, generating a plurality of theme clusters based on the global de-mixing semantic vector, and calculating a sample proportion of the theme clusters; S5, generating a structured insight item of the theme cluster based on the sample proportion of the theme cluster. The method comprises the following steps: S1, calculating a source proportion based on a customer voice record set, and calculating a co-site mixing index of an entry access point corresponding to the customer voice record set in a time slice based on the source proportion; The semantic vector set is constructed based on the local semantic center vector and the global dealiasing semantic vector; The semantic vector set is clustered to obtain a plurality of topic clusters.

5. The global multi-source data semantic fusion insight method based on a large language model according to claim 1, characterized in that, The sample proportion of the topic cluster is calculated, including: The sample number of the sub-set of semantic vectors in the topic cluster is counted; The sample number of the sub-set of semantic vectors in the topic cluster is classified and summarized to obtain the sample number corresponding to each data source type field in the topic cluster; The sample proportion corresponding to each data source type field in the topic cluster is calculated based on the sample number corresponding to each data source type field in the topic cluster.

6. The global multi-source data semantic fusion insight method based on a large language model according to claim 1, characterized in that, The structured insight item of the topic cluster is generated based on the sample proportion of the topic cluster, including: The topic semantic description information of the topic cluster is generated based on the sample proportion of the topic cluster and the global dealiasing semantic vector; The structured insight item of the topic cluster is obtained by combining the topic identifier of the topic cluster, the topic semantic description information, and the sample proportion of the topic cluster.

7. The global multi-source data semantic fusion insight method based on a large language model according to claim 4, characterized in that, The semantic vector set is clustered to obtain a plurality of topic clusters, including: The semantic distance value is obtained by calculating the semantic distance between any two semantic vectors in the semantic vector set; The semantic vectors in the semantic vector set are aggregated based on the semantic distance value to obtain a plurality of semantic clustering groups; The semantic clustering group is defined as a topic cluster.

8. A global multi-source data semantic fusion insight system based on a large language model, characterized in that, The system includes: A mixing index module configured to calculate a source proportion based on the customer voice record set, and calculate a co-site mixing index of an entry access point of the customer voice record set in a time slice based on the source proportion; The specific steps of calculating the co-site mixing index are as follows: The product of the source proportion and the logarithmic value of the source proportion is calculated to obtain a source type information contribution amount; The sum of the source type information contribution amount is calculated to obtain a total information contribution amount; The negative of the total information contribution amount is calculated to obtain a mixing entropy of the entry access point in the time slice; The sum of all mixing entropies is calculated to obtain a sum of mixing entropies; The sum of mixing entropies is divided by the number of time slices to obtain the co-site mixing index of the entry access point; A sample counting module configured to calculate a sample number based on the entry access point identifier field and the data source type field of each customer voice record in the customer voice record set; A dealiasing semantic module configured to generate a global dealiasing semantic vector based on the sample number and the co-site mixing index; The specific steps of generating the global dealiasing semantic vector are as follows: The co-site mixing adjustment value is obtained by adding 1 to the co-site mixing index; The dealiasing weight initial value is obtained by dividing the sample number by the co-site mixing adjustment value; The sum of all dealiasing weight initial values is calculated to obtain a weight initial value sum; The normalized dealiasing weight is obtained by dividing the dealiasing weight initial value by the weight initial value sum; The local semantic center vector is obtained by averaging the semantic vectors in the sub-set of semantic vectors; The weighted semantic vector is obtained by multiplying the normalized dealiasing weight and the local semantic center vector; The global dealiasing semantic vector is obtained by performing vector sum operation on the weighted semantic vector; A topic clustering module configured to generate a plurality of topic clusters based on the global dealiasing semantic vector, and calculate the sample proportion of the topic cluster; An insight generation module configured to generate a structured insight item of the topic cluster based on the sample proportion of the topic cluster.

Images