Big model-based multi-modal government affair question and answer method and system

Abstract

The application discloses a large model-based multi-modal government affair question and answer method and system, and relates to the technical field of artificial intelligence. In the method, a plurality of historical government affair question data are acquired; a plurality of historical question feature vectors are constructed based on the plurality of historical government affair question data; in response to a government affair question and answer request initiated by a user, current government affair question data input by the user are acquired; a current question feature vector is constructed based on the current government affair question data; a first correlation degree of the current government affair question data and each historical government affair question data is calculated based on the plurality of historical question feature vectors and the current question feature vector; target historical government affair question data with a first correlation degree greater than a preset first correlation threshold are determined from the plurality of historical government affair question data; and current government affair answer data are generated based on the target historical government affair question data. The technical scheme of the application can effectively improve the matching degree of the generated government affair answer and the actual demand of the user.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a multimodal government affairs question-answering method and system based on a large model. Background Technology

[0002] Traditional government Q&A relies on human customer service, but with the development of artificial intelligence technology, intelligent government Q&A has been widely applied in government service Q&A systems. Current intelligent government Q&A methods typically match appropriate responses from a pre-set government Q&A database based on the user's input. However, this method cannot accurately analyze the user's actual intent, resulting in a low degree of relevance between the generated answers and the user's actual needs.

[0003] Therefore, improving the matching degree between generated government service answers and users' actual needs has become an urgent technical problem to be solved. Summary of the Invention

[0004] This application provides a multimodal government affairs question-answering method and system based on a large model, which can effectively improve the matching degree between the generated government affairs answers and the actual needs of users.

[0005] In a first aspect, this application provides a multimodal government affairs question-answering method based on a large model. The method includes: acquiring multiple historical government affairs question data; constructing multiple historical question feature vectors based on the multiple historical government affairs question data; in response to a government affairs question-answering request initiated by a user, acquiring current government affairs question data input by the user; constructing a current question feature vector based on the current government affairs question data; calculating a first correlation degree between the current government affairs question data and each of the historical government affairs question data based on the multiple historical question feature vectors and the current question feature vector; determining a target historical government affairs question data from the multiple historical government affairs question data; the target historical government affairs question data being historical government affairs question data whose first correlation degree is greater than a preset first correlation threshold; and generating current government affairs answer data based on the target historical government affairs question data.

[0006] By adopting the above technical solution, and constructing multiple historical question feature vectors based on a variety of historical government affairs question data, the accuracy of data processing is improved. By responding to user-initiated government affairs question-and-answer requests and acquiring the current government affairs question data input by the user, the specific needs and questions of the user can be directly captured, thus ensuring the relevance and relevance of the answer. By constructing a current question feature vector based on the current government affairs question data, the key content of the user's question can be accurately extracted. By calculating the first correlation degree between the current government affairs question data and each historical government affairs question data based on multiple historical question feature vectors and the current question feature vector, the solution can identify the historical cases most relevant to the current question, thereby improving the accuracy of the answer. By determining target historical government affairs question data from multiple historical government affairs question data with a first correlation degree greater than a preset first correlation threshold, and generating current government affairs answer data based on the target historical government affairs question data, the generated current government affairs answer data is highly relevant to the user's government affairs question, thereby effectively improving the matching degree between the generated government affairs answer and the user's actual needs.

[0007] Optionally, the historical government affairs inquiry data includes historical government affairs inquiry sub-data in multiple modalities; the step of constructing multiple historical inquiry feature vectors based on multiple historical government affairs inquiry data specifically includes: preprocessing the historical government affairs inquiry sub-data in multiple modalities based on the data types corresponding to the historical government affairs inquiry sub-data in multiple modalities to obtain historical preprocessed sub-data in multiple modalities; performing feature extraction operations on the historical preprocessed sub-data in multiple modalities to obtain multiple historical initial features; filtering all the historical initial features to obtain multiple historical inquiry features; and performing a fusion operation on the multiple historical inquiry features to obtain multiple historical inquiry feature vectors.

[0008] By employing the aforementioned technical solution, historical government question sub-data of various modalities is preprocessed based on the data types corresponding to these modalities, thereby standardizing and optimizing the original data and improving the accuracy of subsequent feature extraction. Feature extraction operations are performed on the preprocessed historical sub-data of various modalities, effectively extracting key information from different types of data. By screening all initial historical features, irrelevant or duplicate features are removed, ensuring the accuracy of the feature set. Finally, by fusing multiple historical question features, multiple historical question feature vectors are obtained, thereby improving the accuracy of the question-answering system.

[0009] Optionally, the step of filtering all the historical initial features to obtain multiple historical question features specifically includes: filtering out duplicate features from all the historical initial features to obtain multiple related historical features; calculating the feature importance coefficient value corresponding to each of the related historical features based on a feature selection algorithm; filtering multiple historical question features from the multiple related historical features; wherein the historical question features are related historical features whose feature importance coefficient value is greater than a preset feature importance coefficient threshold.

[0010] By adopting the above technical solution, redundant information can be reduced by filtering out duplicate features from all historical initial features, making the remaining historical question features more accurate. By calculating the feature importance coefficient value corresponding to each relevant historical feature, and selecting historical question features with feature importance coefficient values ​​greater than the preset feature importance coefficient threshold from multiple relevant historical features, the historical question features are highly relevant and representative for understanding the content of historical questions.

[0011] Optionally, the current government affairs inquiry data includes multiple modalities of current government affairs inquiry sub-data; the step of constructing a current inquiry feature vector based on the current government affairs inquiry data specifically includes: obtaining a contextual environment based on multiple historical government affairs inquiry data and the current government affairs inquiry data; preprocessing the multiple modalities of current government affairs inquiry sub-data based on the data types corresponding to the multiple modalities of current government affairs inquiry sub-data to obtain multiple modalities of current preprocessed sub-data; analyzing the multiple modalities of current preprocessed sub-data based on the contextual environment to obtain multiple current initial features; performing weighted processing and priority adjustment operations on each of the current initial features to obtain multiple current inquiry features; and performing a fusion operation on the multiple current inquiry features to obtain a current inquiry feature vector.

[0012] By adopting the above technical solution, and obtaining the contextual environment based on multiple historical and current government affairs question data, the system can understand the user's current question within a broader context, enhancing the accurate grasp of the question's intent and improving the relevance and relevance of the answer. Analyzing multiple modalities of current preprocessed sub-data based on the contextual environment yields multiple current initial features, ensuring that the extracted features comprehensively reflect all aspects of the current question and enhancing the representativeness of the features. By weighting and prioritizing each current initial feature, multiple current question features are obtained, ensuring that the most critical elements in the feature set receive greater attention, increasing the accuracy of the generated answer, and thus effectively improving the matching degree between the subsequently generated government affairs answers and the user's actual needs.

[0013] Optionally, the step of weighting and prioritizing each of the current initial features to obtain multiple current question features specifically includes: performing importance analysis on each of the current initial features to obtain weighting coefficients corresponding to each of the current initial features; determining initial feature values ​​corresponding to each of the current initial features; weighting each of the current initial features based on the weighting coefficients to obtain weighted current features; obtaining weighted feature values ​​corresponding to each of the weighted current features based on the initial feature values; and prioritizing the weighted current features based on the weighted feature values ​​to obtain a current question feature set; the current question feature set includes multiple sorted current question features.

[0014] Optionally, generating current government response data based on the target historical government question data specifically includes: obtaining historical government response data corresponding to each of the historical government question data; determining target historical government response data corresponding to the target historical government question data from the historical government response data based on the target historical government question data; and generating current government response data based on the target historical government response data.

[0015] Optionally, generating current government response data based on the target historical government response data specifically includes: constructing a target historical response feature vector based on the target historical government response data; calculating a second correlation degree between the current government question data and the target historical government response data based on the target historical response feature vector and the current question feature vector; determining reference historical government response data from multiple target historical government response data; the reference historical government response data being target historical government response data whose second correlation degree is greater than a preset second correlation threshold; and generating the current government response data based on the reference historical government response data.

[0016] By employing the aforementioned technical solution, target historical government affairs questions and answers are identified from historical government affairs answer data, ensuring that the selected historical answers are closely related to the current question and improving the adaptability and accuracy of the generated answers in terms of content and context. Furthermore, by calculating the second degree of association between the current government affairs question data and the target historical government affairs answer data based on the feature vectors of the target historical answers and the current question, the selected historical answers are highly matched to the current question in terms of content and theme. By identifying reference historical government affairs answer data from multiple target historical government affairs answer data sets, the most suitable reference can be selected from a series of highly relevant historical answers, further improving the relevance and quality of the answers. Finally, by identifying reference historical government affairs answer data with a second degree of association greater than a preset second association threshold from multiple target historical government affairs answer data sets, and generating the current government affairs answer data based on this reference historical government affairs answer data, the selected historical answers are highly relevant to the current question, thereby improving the relevance and accuracy of the final generated answer and ensuring that the generated government affairs answer highly matches the user's actual needs.

[0017] A second aspect of this application provides a multimodal government affairs question-and-answer system based on a large model. The system includes an acquisition module and a processing module. The acquisition module is used to acquire at least one historical government affairs question data. The processing module is used to construct multiple historical question feature vectors based on multiple historical government affairs question data. The acquisition module is also used to acquire current government affairs question data input by the user in response to a government affairs question-and-answer request initiated by the user. The processing module is also used to construct a current question feature vector based on the current government affairs question data. The processing module is also used to calculate a first correlation degree between the current government affairs question data and each of the historical government affairs question data based on the multiple historical question feature vectors and the current question feature vector. The processing module is also used to determine a target historical government affairs question data from the multiple historical government affairs question data. The target historical government affairs question data is historical government affairs question data whose first correlation degree is greater than a preset first correlation threshold. The processing module is also used to generate current government affairs answer data based on the target historical government affairs question data.

[0018] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of the first aspects of this application.

[0019] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that can be loaded by a processor and execute the method described in any of the first aspects of this application.

[0020] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. By constructing multiple historical question feature vectors based on various historical government affairs question data, the accuracy of data processing is improved. Responding to user-initiated government affairs question-and-answer requests and acquiring the current government affairs question data input by the user, the specific needs and questions of the user can be directly captured, ensuring the relevance and relevance of the answer. Constructing a current question feature vector based on the current government affairs question data allows for accurate extraction of the key content of the user's question. By calculating the first correlation degree between the current government affairs question data and each historical government affairs question data based on multiple historical question feature vectors and the current question feature vector, the solution can identify the historical cases most relevant to the current question, thereby improving the accuracy of the answer. By identifying target historical government affairs question data from multiple historical government affairs question data with a first correlation degree greater than a preset first correlation threshold, and generating current government affairs answer data based on the target historical government affairs question data, the generated current government affairs answer data is highly relevant to the user's government affairs question, thus effectively improving the matching degree between the generated government affairs answer and the user's actual needs.

[0021] 2. By obtaining the contextual environment based on multiple historical and current government affairs question data, the system can understand the user's current question within a broader context, enhancing the accurate grasp of the question's intent and improving the relevance and targeting of the answers. Analyzing multiple modalities of current preprocessed sub-data based on the contextual environment yields multiple current initial features, ensuring that the extracted features comprehensively reflect all aspects of the current question and enhancing feature representativeness. Weighting and prioritizing these current initial features results in multiple current question features, ensuring that the most critical elements in the feature set receive greater attention, increasing the accuracy of answer generation, and effectively improving the matching degree between subsequently generated government affairs answers and the user's actual needs.

[0022] 3. By identifying target historical government affairs questions and answers from historical government affairs response data, the system ensures that the selected historical answers are closely related to the current question, improving the adaptability and accuracy of the generated answers in terms of content and context. Furthermore, by calculating the second degree of association between the current government affairs question and the target historical government affairs response data based on the feature vectors of the target historical answers and the current question, the system ensures that the selected historical answers are highly matched to the current question in terms of content and theme. By identifying reference historical government affairs response data from multiple target historical government affairs response data sets, the system can select the most suitable reference from a series of highly relevant historical answers, further improving the relevance and quality of the answers. Finally, by identifying reference historical government affairs response data from multiple target historical government affairs response data sets with a second degree of association greater than a preset second association threshold, and generating the current government affairs response data based on this reference historical government affairs response data, the system ensures that the selected historical answers are highly relevant to the current question, thereby improving the relevance and accuracy of the final generated answer and ensuring that the generated government affairs answer highly matches the user's actual needs. Attached Figure Description

[0023] Figure 1 This is one of the flowcharts illustrating a multimodal government affairs question-answering method based on a large model provided in this application embodiment; Figure 2 This is the second flowchart illustrating a multimodal government affairs question-answering method based on a large model provided in this application embodiment; Figure 3 This is the third flowchart illustrating a multimodal government affairs question-answering method based on a large model provided in this application embodiment; Figure 4 This is a schematic diagram of the structure of a multimodal government affairs question-answering system based on a large model, provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application.

[0024] Explanation of reference numerals in the attached drawings: 1. Acquisition module; 2. Processing module; 500. Electronic device; 501. Processor; 502. Communication bus; 503. User interface; 504. Network interface; 505. Memory. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0026] In the description of the embodiments in this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a concrete manner.

[0027] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0028] This application provides a multimodal government affairs question-answering method based on a large model, applied in a server. (See reference...) Figure 1 This illustration shows one of the flowcharts of a multimodal government affairs question-answering method based on a large model provided in an embodiment of this application. The method includes steps S1-S7, as follows: Step S1: Obtain multiple historical government affairs inquiry data.

[0029] Specifically, in this technical solution, historical government affairs question data can be questions previously asked by the current user and other users on the government affairs question and answer platform. Historical government affairs question data includes sub-data in multiple modalities, including text-formatted, image-formatted, and audio-formatted historical government affairs question data.

[0030] Step S2: Construct multiple historical question feature vectors based on multiple historical government affairs question data.

[0031] Specifically, in this technical solution, the server constructs multiple historical question feature vectors based on multiple historical government affairs question data.

[0032] In one possible implementation, refer to Figure 2 This illustrates a second flowchart of a multimodal government affairs question-answering method based on a large model, provided in an embodiment of this application. Step S2 specifically includes steps S21-S24: Step S21: Preprocess the multi-modality historical government question sub-data based on the corresponding data types of the multi-modality historical government question sub-data, to obtain multi-modality historical preprocessed sub-data.

[0033] Specifically, in this technical solution, the multi-modality historical government question sub-data includes historical government question sub-data in text format, historical government question sub-data in image format, and historical government question sub-data in audio format. Therefore, the manner of preprocessing the multi-modality historical government question sub-data specifically includes text data preprocessing, image data preprocessing, and audio data preprocessing.

[0034] More specifically, the text data preprocessing includes the following steps: data cleaning, that is, removing unnecessary characters in the text, such as HTML tags, special symbols, incorrect formats, etc.; standardization processing, that is, unifying the used vocabulary or phrases through synonym replacement; word segmentation processing, that is, splitting the text into words or phrases according to language characteristics; stop word removal, that is, deleting common words in the text that have no actual analytical value, such as "de", "he", etc.; entity recognition, that is, identifying important entities in the text, such as locations, person names, organization names, etc.

[0035] The image data preprocessing includes the following steps: format conversion, that is, converting the image into a unified format, such as JPEG, PNG and other formats; size adjustment, that is, adjusting the resolution or size of the image to meet the requirements of subsequent processing; color adjustment, that is, converting the image into a grayscale image to reduce processing complexity; feature extraction, that is, using image processing technologies to extract key features such as edge detection, texture analysis, etc.; noise removal, that is, eliminating noise in the image to improve image quality.

[0036] The audio data preprocessing includes the following steps: format conversion, that is, converting the audio data into a standard format, such as WAV or MP3; noise reduction processing, that is, reducing or eliminating background noise; segmentation processing, that is, splitting the audio into smaller segments according to pauses; sound extraction, that is, extracting the main sound or voice content in the audio file; feature extraction, that is, extracting key features in the audio such as pitch, rhythm, speech speed, etc.

[0037] Step S22: Perform a feature extraction operation on the multi-modality historical preprocessed sub-data to obtain a plurality of historical initial features.

[0038] Specifically, in this technical solution, the server performs a feature extraction operation on the multi-modality historical preprocessed sub-data, that is, performing text data feature extraction on the text-type historical preprocessed sub-data; performing image data feature extraction on the image-type historical preprocessed sub-data; performing audio data feature extraction on the audio-type historical preprocessed sub-data, so as to obtain a plurality of historical initial features.

[0039] More specifically, text data feature extraction includes the following methods: keyword extraction, which identifies keywords or phrases in the text, such as words expressing core concepts or themes; topic modeling, which applies algorithms such as LDA (Latent Dirichlet Allocation) to extract the topic distribution of the text; sentiment analysis, which determines the sentiment tendency of the text, such as positive, negative, or neutral; and word embedding, which uses methods such as Word2Vec and GloVe to convert words in the text into vector form to capture the semantic relationships between words.

[0040] Image data feature extraction includes the following methods: Edge detection: identifying edges in an image to obtain the outlines of objects. Texture analysis: analyzing texture features such as smoothness and roughness. Color distribution: extracting the dominant colors and color distribution in an image. Deep learning feature extraction: using deep learning techniques such as convolutional neural networks (CNNs) to extract high-level features of an image.

[0041] Audio data feature extraction includes the following methods: frequency analysis, which analyzes the frequency characteristics of the audio signal, such as pitch and volume; rhythm extraction, which identifies rhythmic patterns in the audio, such as tempo and dynamics; voice feature extraction, which extracts features of speech, such as timbre and speech rate; and spectrogram analysis, which extracts key information by analyzing the spectrogram of the audio.

[0042] Step S23: Filter all historical initial features to obtain multiple historical question features.

[0043] Specifically, in this technical solution, the server will filter all historical initial features to obtain historical question features that play a key role in understanding and answering government affairs questions from multiple historical questions.

[0044] In one possible implementation, step S23 specifically includes the following steps: By filtering out duplicate features from all initial historical features, multiple relevant historical features are obtained.

[0045] Specifically, in this technical solution, duplicate features are filtered out from all historical initial features. That is, duplicate words or phrases are filtered out from the historical initial features of text types; duplicate image textures, color distributions, and edge features are filtered out from the historical initial features of image types; and frequency distributions, rhythm patterns, and sound features are filtered out from the historical initial features of audio types. This yields multiple relevant historical features.

[0046] Based on the feature selection algorithm, the feature importance coefficient value corresponding to each relevant historical feature is calculated.

[0047] Specifically, in this technical solution, the server will first select a suitable feature selection algorithm based on the data type being processed and the specific application. Feature selection algorithms include, but are not limited to, filtering, wrapping, and embedding methods.

[0048] Then, based on the selected feature selection algorithm, the feature importance coefficient values ​​corresponding to each relevant historical feature are calculated to reflect the contribution of each relevant historical feature to the prediction or classification task. For relevant historical features of text type, the corresponding feature importance coefficient values ​​are obtained by evaluating the information content of words or their role in document classification; for relevant historical features of image type, the corresponding feature importance coefficient values ​​are obtained by evaluating the discriminative power of a certain visual feature; and for relevant historical features of image type, the corresponding feature importance coefficient values ​​are obtained by evaluating the recognition importance of a specific audio feature.

[0049] Multiple historical question features are obtained by filtering from multiple relevant historical features; historical question features are relevant historical features whose feature importance coefficient values ​​are greater than the preset feature importance coefficient threshold.

[0050] Specifically, in this technical solution, a preset feature importance coefficient threshold needs to be determined based on statistical analysis of feature importance distribution or expert knowledge. This application does not specify the exact value of the preset feature importance coefficient threshold. The server then filters from multiple relevant historical features to obtain relevant historical features whose feature importance coefficient values ​​are greater than the preset feature importance coefficient threshold, thus obtaining multiple historical query features.

[0051] Step S24: Perform a fusion operation on multiple historical question features to obtain multiple historical question feature vectors.

[0052] Specifically, in this technical solution, the server first normalizes multiple historical question features to ensure that they can be compared and calculated on the same scale. Then, all selected historical question features corresponding to a single historical government question are integrated into a multi-dimensional feature vector to obtain the historical question feature vector corresponding to that single historical government question.

[0053] It should be noted that in step S24, the term "multiple" refers to different quantities. For example, if the current user and other users have made a total of n historical government affairs inquiries, and each historical government affairs inquiry corresponds to m historical inquiry features, then there are a total of n×m historical inquiry features. Finally, the n×m historical inquiry features are fused to obtain n historical inquiry feature vectors.

[0054] Step S3: In response to the user's government affairs Q&A request, obtain the current government affairs question data entered by the user.

[0055] Specifically, in this technical solution, the current government affairs question data can be the questions that users currently ask on the government affairs question and answer platform. The current government affairs question data includes multiple modalities of current government affairs question sub-data. These multiple modalities include text-formatted current government affairs question sub-data, image-formatted current government affairs question sub-data, and audio-formatted current government affairs question sub-data.

[0056] Step S4: Construct the current question feature vector based on the current government affairs question data.

[0057] Specifically, in this technical solution, the server constructs a feature vector for the current government affairs inquiry based on the current inquiry data.

[0058] In one possible implementation, refer to Figure 3 This illustrates the third flowchart of a multimodal government affairs question-answering method based on a large model provided in this application embodiment. Step S4 specifically includes steps S41-S45: Step S41: Obtain the context based on multiple historical government affairs inquiry data and current government affairs inquiry data.

[0059] Specifically, in this technical solution, the server establishes connections between multiple historical government affairs query data and current government affairs query data to identify similarities and differences. It analyzes whether the current government affairs query is related to certain government affairs questions in the historical data, or whether the current government affairs query represents a completely new type of government affairs query, thereby obtaining the contextual environment.

[0060] Step S42: Based on the data types corresponding to the current government affairs question sub-data of multiple modalities, preprocess the current government affairs question sub-data of multiple modalities to obtain the current preprocessed sub-data of multiple modalities.

[0061] Specifically, in this technical solution, the method of preprocessing the current government affairs question sub-data based on the data type corresponding to the current government affairs question sub-data of multiple modalities is the same as the method of preprocessing the historical government affairs question sub-data based on the data type corresponding to the historical government affairs question sub-data of multiple modalities in the aforementioned embodiment, so it will not be elaborated on here.

[0062] Step S43: Analyze the current preprocessed sub-data of multiple modalities based on the context environment to obtain multiple current initial features.

[0063] Specifically, in this technical solution, the server will first perform feature extraction on the current preprocessed sub-data of multiple modalities to obtain multiple features, and then obtain multiple current initial features from the multiple features based on the context environment.

[0064] The method by which the server performs feature extraction on the current preprocessed sub-data of multiple modalities is the same as the method for performing feature extraction on the historical preprocessed sub-data of multiple modalities in the aforementioned embodiments, so it will not be elaborated further here.

[0065] After obtaining multiple features, the server will consider these features in a broader context based on the context, analyzing how the current query relates to historical data, socioeconomic background, policy changes, and other factors, thereby determining multiple current initial features from the multiple features.

[0066] Step S44: Perform weighted processing and priority adjustment on each current initial feature to obtain multiple current query features.

[0067] Specifically, in this technical solution, the server performs weighted processing and priority adjustment operations on each current initial feature to obtain multiple current query features.

[0068] In one possible implementation, step S44 specifically includes the following steps: An importance analysis is performed on each current initial feature to obtain the weighting coefficients corresponding to each current initial feature.

[0069] Specifically, in this technical solution, the server will evaluate the importance of each current initial feature extracted from the current government inquiry to determine its importance in understanding the user's query. The importance evaluation method can be based on the frequency of the current initial feature in the question, its historical performance in similar questions, or its contribution to understanding the question's intent. The server will also select an appropriate importance evaluation method based on the data type (e.g., text, image, audio) corresponding to the current initial feature. For example, for text-type current initial features, the importance of keywords can be determined through term frequency-inverse document frequency (TF-IDF) analysis; for image-type current initial features, the salience of specific elements can be determined through image recognition technology. The server will then obtain a weighted coefficient corresponding to each current initial feature based on the importance evaluation results.

[0070] Determine the initial feature value corresponding to each current initial feature.

[0071] Specifically, in this technical solution, skilled personnel will determine the initial feature values ​​corresponding to each current initial feature based on the properties associated with that feature. Initial feature values ​​can be numerical (e.g., count, frequency, probability) or Boolean (e.g., presence / absence). More specifically, for text-type current initial features, initial feature values ​​can be quantified using methods such as word frequency and TF-IDF scores. For image-type current initial features, initial feature values ​​can be quantified using the number of recognized objects, the frequency of occurrence of specific colors or shapes, etc. For audio-type current initial features, initial feature values ​​can be quantified based on the duration, volume, and pitch of the sound.

[0072] Each initial feature is weighted based on its respective weighting coefficient to obtain the weighted current feature; and the weighted feature value corresponding to each weighted current feature is obtained based on each initial feature value.

[0073] Specifically, in this technical solution, the weighted eigenvalue = initial eigenvalue × weighting coefficient.

[0074] Based on each weighted feature value, the priority of the weighted current features is adjusted to obtain the current question feature set; the current question feature set includes multiple sorted current question features.

[0075] Specifically, in this technical solution, the server sorts the weighted feature values ​​corresponding to all weighted current features from high to low to determine the priority of each feature. The priority of a feature determines its importance in constructing an answer.

[0076] It should be noted that when adjusting feature priorities, the possible dependencies or complementarities between different features should be considered to ensure that priority adjustments are not only based on the importance of individual features, but also on the overall effect of feature combinations.

[0077] Step S45: Perform a fusion operation on multiple current question features to obtain the current question feature vector.

[0078] Specifically, in this technical solution, the method of fusing multiple current question features to obtain the current question feature vector is the same as the method of fusing multiple historical question features to obtain multiple historical question feature vectors in the aforementioned embodiment, so it will not be elaborated further here.

[0079] Step S5: Based on multiple historical question feature vectors and the current question feature vector, calculate the first correlation degree between the current government question data and each historical government question data.

[0080] Specifically, in this technical solution, the methods for calculating the first degree of correlation between the current government affairs query data and each historical government affairs query data include, but are not limited to, cosine similarity, Euclidean distance, and Jaccard similarity. The server then assigns a correlation score, i.e., the first degree of correlation, to each historical government affairs query based on the similarity calculation results. The first degree of correlation reflects the strength of the correlation between the historical query and the current query.

[0081] Step S6: Determine the target historical government affairs question data from multiple historical government affairs question data; the target historical government affairs question data is the historical government affairs question data with a first correlation degree greater than the preset first correlation threshold.

[0082] Specifically, in this technical solution, the server will determine the historical government affairs question data with a first correlation degree greater than a preset first correlation threshold from multiple historical government affairs question data, thus obtaining the target historical government affairs question data.

[0083] The preset first association threshold is determined based on statistical analysis of data importance distribution or expert knowledge. In this application, the specific value of the preset first association threshold is not limited.

[0084] Step S7: Generate current government response data based on the target historical government question data.

[0085] Specifically, in this technical solution, the server generates current government response data based on the target's historical government inquiry data.

[0086] In one possible implementation, step S7 specifically includes the following steps: Obtain historical government response data corresponding to each historical government question.

[0087] Specifically, in this technical solution, the server will search the database for historical question-and-answer pairs and obtain the historical government affairs answer data corresponding to each historical government affairs question data.

[0088] Based on the target historical government affairs question data, the target historical government affairs answer data corresponding to the target historical government affairs question data is determined from the historical government affairs answer data.

[0089] Specifically, in this technical solution, the correspondence between historical government affairs questions and answers on the server is used to determine the target historical government affairs answer data corresponding to the target historical government affairs question data.

[0090] Generate current government response data based on the target's historical government response data.

[0091] Specifically, in this technical solution, the server generates current government response data based on the target's historical government response data.

[0092] In one possible implementation, generating current government response data based on target historical government response data specifically includes the following steps: Based on the target's historical government affairs response data, construct the target's historical response feature vector.

[0093] Specifically, in this technical solution, the method of constructing a target historical answer feature vector based on the target historical government affairs answer data is the same as the method of constructing multiple historical question feature vectors based on multiple historical government affairs question data in step S2 of the aforementioned embodiment, so it will not be elaborated on here.

[0094] Based on the target historical answer feature vector and the current question feature vector, the second degree of correlation between the current government question data and the target historical government answer data is calculated.

[0095] Specifically, in this technical solution, the method by which the server calculates the second degree of correlation between the current government affairs question data and the target historical government affairs answer data based on the target historical answer feature vector and the current question feature vector is the same as the method in step S5 of the aforementioned embodiment for calculating the first degree of correlation between the current government affairs question data and each historical government affairs question data based on multiple historical question feature vectors and the current question feature vector. Therefore, it will not be elaborated further here.

[0096] Reference historical government response data is determined from multiple target historical government response data; the reference historical government response data is the target historical government response data with a second correlation degree greater than a preset second correlation threshold.

[0097] Specifically, in this technical solution, the server will determine the historical government question data with a second correlation degree greater than a preset second correlation threshold from multiple target historical government response data, thus obtaining the target historical government response data.

[0098] Similarly, the preset second association threshold is determined based on statistical analysis of data importance distribution or expert knowledge, and no specific value of the preset second association threshold is limited in this application.

[0099] The current government response data is generated based on historical government response data.

[0100] Specifically, in this technical solution, the server first conducts an in-depth analysis of historical government response data to understand its core content, structure, and response style. Based on the specific content and requirements of the current question, the information in the historical responses is adjusted to better adapt to the current context. This includes reorganizing the information structure, updating data or facts, and modifying the responses. The adjusted responses are ensured to maintain consistency with the original historical responses in language and style, while also fitting the background and needs of the current question. Natural Language Generation (NLG) technology is then used to convert the adjusted content into fluent, coherent, and easily understandable response text, thus obtaining the current government response data.

[0101] Reference Figure 4 This document illustrates a schematic diagram of a multimodal government affairs question-and-answer system based on a large model, provided in an embodiment of this application. The system includes an acquisition module 1 and a processing module 2. The acquisition module 1 is used to acquire at least one historical government affairs question data. The processing module 2 is used to construct multiple historical question feature vectors based on multiple historical government affairs question data. The acquisition module 1 is also used to acquire current government affairs question data input by the user in response to a government affairs question-and-answer request initiated by the user. The processing module 2 is also used to construct a current question feature vector based on the current government affairs question data. The processing module 2 is also used to calculate a first correlation degree between the current government affairs question data and each historical government affairs question data based on multiple historical question feature vectors and the current question feature vector. The processing module 2 is also used to determine a target historical government affairs question data from the multiple historical government affairs question data. The target historical government affairs question data is historical government affairs question data whose first correlation degree is greater than a preset first correlation threshold. The processing module 2 is also used to generate current government affairs answer data based on the target historical government affairs question data.

[0102] In one possible implementation, processing module 2 is further configured to preprocess the historical government inquiry sub-data of multiple modalities based on the data types corresponding to the historical government inquiry sub-data of multiple modalities, to obtain historical preprocessed sub-data of multiple modalities; processing module 2 is further configured to perform feature extraction operations on the historical preprocessed sub-data of multiple modalities to obtain multiple historical initial features; processing module 2 is further configured to filter all historical initial features to obtain multiple historical inquiry features; processing module 2 is further configured to perform fusion operations on the multiple historical inquiry features to obtain multiple historical inquiry feature vectors.

[0103] In one possible implementation, the processing module 2 is further configured to remove duplicate features from all historical initial features to obtain multiple related historical features; the processing module 2 is further configured to calculate the feature importance coefficient value corresponding to each related historical feature based on a feature selection algorithm; the processing module 2 is further configured to select multiple historical question features from multiple related historical features; the historical question features are related historical features whose feature importance coefficient value is greater than a preset feature importance coefficient threshold.

[0104] In one possible implementation, processing module 2 is further configured to obtain a contextual environment based on multiple historical government affairs inquiry data and current government affairs inquiry data; processing module 2 is further configured to preprocess the current government affairs inquiry sub-data of multiple modalities based on the data types corresponding to the current government affairs inquiry sub-data of multiple modalities to obtain current preprocessed sub-data of multiple modalities; processing module 2 is further configured to analyze the current preprocessed sub-data of multiple modalities based on the contextual environment to obtain multiple current initial features; processing module 2 is further configured to perform weighted processing and priority adjustment operations on each current initial feature to obtain multiple current inquiry features; processing module 2 is further configured to perform a fusion operation on multiple current inquiry features to obtain a current inquiry feature vector.

[0105] In one possible implementation, processing module 2 is further configured to perform importance analysis on each current initial feature to obtain a weighting coefficient corresponding to each current initial feature; processing module 2 is further configured to determine an initial feature value corresponding to each current initial feature; processing module 2 is further configured to perform weighting processing on each current initial feature based on each weighting coefficient to obtain a weighted current feature; and based on each initial feature value, obtain a weighted feature value corresponding to each weighted current feature; processing module 2 is further configured to perform priority adjustment on the weighted current feature based on each weighted feature value to obtain a current question feature set; the current question feature set includes multiple sorted current question features.

[0106] In one possible implementation, the acquisition module 1 is further configured to acquire historical government response data corresponding to each historical government question data; the processing module 2 is further configured to determine the target historical government response data corresponding to the target historical government question data from the historical government response data based on the target historical government question data; the processing module 2 is further configured to generate the current government response data based on the target historical government response data.

[0107] In one possible implementation, processing module 2 is further configured to construct a target historical government response feature vector based on the target historical government response data; processing module 2 is further configured to calculate a second correlation degree between the current government question data and the target historical government response data based on the target historical response feature vector and the current question feature vector; processing module 2 is further configured to determine reference historical government response data from multiple target historical government response data; the reference historical government response data is the target historical government response data whose second correlation degree is greater than a preset second correlation threshold; processing module 2 is further configured to generate current government response data based on the reference historical government response data.

[0108] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0109] This application also discloses an electronic device. (See reference...) Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. The electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0110] The communication bus 502 is used to enable communication between these components.

[0111] The user interface 503 may include a display screen and a camera. Optionally, the user interface 503 may also include a standard wired interface and a wireless interface.

[0112] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0113] The processor 501 may include one or more processing cores. The processor 501 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 505, and by calling data stored in memory 505. Optionally, the processor 501 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 501 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 501 and may be implemented as a separate chip.

[0114] The memory 505 may include random access memory (RAM) or read-only memory. Optionally, the memory 505 may include non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 505 may also be at least one storage device located remotely from the aforementioned processor 501. (Refer to...) Figure 5 The memory 505, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and an application program.

[0115] exist Figure 5In the illustrated electronic device 500, the user interface 503 is mainly used to provide an input interface for the user and to acquire user input data; while the processor 501 can be used to call an application stored in the memory 505. When executed by one or more processors 501, the electronic device 500 performs one or more methods as described in the above embodiments. It should be noted that, for the foregoing method embodiments, for the sake of simplicity, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0116] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0117] In the various embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0119] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0120] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0121] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will readily conceive of by those skilled in the art upon consideration of the specification and the disclosure of practical truths.

[0122] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A multimodal government affairs question-answering method based on a large model, characterized in that, The method includes: Obtain multiple historical government affairs inquiry data; Based on the aforementioned historical government affairs inquiry data, multiple historical inquiry feature vectors are constructed; In response to a user's request for government affairs Q&A, obtain the current government affairs question data entered by the user; Based on the current government affairs inquiry data, construct the current inquiry feature vector; Based on the multiple historical question feature vectors and the current question feature vector, calculate the first correlation degree between the current government affairs question data and each of the historical government affairs question data; The target historical government affairs question data is determined from a plurality of historical government affairs question data; the target historical government affairs question data is the historical government affairs question data whose first correlation degree is greater than a preset first correlation threshold. Based on the target's historical government affairs inquiry data, generate current government affairs response data; The historical government affairs inquiry data includes sub-data of historical government affairs inquiries in multiple modalities; the construction of multiple historical inquiry feature vectors based on multiple historical government affairs inquiry data specifically includes: Based on the data types corresponding to the historical government affairs question sub-data of the various modalities, the historical government affairs question sub-data of the various modalities is preprocessed to obtain historical preprocessed sub-data of the various modalities; Feature extraction is performed on the historical preprocessed sub-data of the various modalities to obtain multiple historical initial features; By filtering all the aforementioned initial historical features, multiple historical query features are obtained; A fusion operation is performed on multiple historical question features to obtain multiple historical question feature vectors; The current government affairs inquiry data includes multiple modalities of current government affairs inquiry sub-data; the construction of a current inquiry feature vector based on the current government affairs inquiry data specifically includes: The context is obtained based on multiple historical government affairs inquiry data and the current government affairs inquiry data; Based on the data types corresponding to the current government affairs question sub-data of the various modalities, the current government affairs question sub-data of the various modalities is preprocessed to obtain the current preprocessed sub-data of the various modalities; Based on the aforementioned context, the current preprocessed sub-data of the various modalities are analyzed to obtain multiple current initial features; The current initial features are weighted and prioritized to obtain multiple current query features. A fusion operation is performed on multiple current question features to obtain the current question feature vector; The process of generating current government response data based on the target historical government inquiry data specifically includes: Obtain historical government response data corresponding to each of the aforementioned historical government question data; Based on the target historical government affairs question data, determine the target historical government affairs answer data corresponding to the target historical government affairs question data from the historical government affairs answer data; Based on the target's historical government affairs response data, generate the current government affairs response data; The process of generating current government response data based on the target historical government response data specifically includes: Based on the target's historical government affairs response data, construct a target historical response feature vector; Based on the target historical answer feature vector and the current question feature vector, calculate the second degree of correlation between the current government question data and the target historical government answer data; Reference historical government response data is determined from multiple target historical government response data; the reference historical government response data is target historical government response data whose second correlation degree is greater than a preset second correlation threshold. The current government response data is generated based on the historical government response data.

2. The method according to claim 1, characterized in that, The process of filtering all the initial historical features yields multiple historical query features, specifically including: Duplicate features are removed from all the initial historical features to obtain multiple relevant historical features; Based on the feature selection algorithm, the feature importance coefficient value corresponding to each of the relevant historical features is calculated; Multiple historical question features are obtained by filtering from multiple related historical features; the historical question features are related historical features whose feature importance coefficient values ​​are greater than a preset feature importance coefficient threshold.

3. The method according to claim 1, characterized in that, The weighting and priority adjustment operation on each of the current initial features to obtain multiple current query features specifically includes: An importance analysis is performed on each of the current initial features to obtain the weighting coefficients corresponding to each of the current initial features; Determine the initial feature value corresponding to each of the current initial features; The current initial features are weighted based on the weighting coefficients to obtain the weighted current features; and the weighted feature values ​​corresponding to the weighted current features are obtained based on the initial feature values. Based on each of the weighted feature values, the priority of the weighted current features is adjusted to obtain the current question feature set; the current question feature set includes multiple sorted current question features.

4. A multimodal government affairs question-answering system based on a large model, characterized in that, The system includes an acquisition module and a processing module; The acquisition module is used to acquire at least one historical government affairs inquiry data; The processing module is used to construct multiple historical question feature vectors based on multiple historical government affairs question data. The acquisition module is also used to respond to a government affairs Q&A request initiated by a user and acquire the current government affairs question data input by the user; The processing module is also used to construct a current question feature vector based on the current government affairs question data; The processing module is further configured to calculate the first degree of correlation between the current government affairs question data and each of the historical government affairs question data based on the multiple historical question feature vectors and the current question feature vector; The processing module is further configured to determine target historical government affairs question data from the plurality of historical government affairs question data; the target historical government affairs question data is historical government affairs question data whose first correlation degree is greater than a preset first correlation threshold; The processing module is further configured to generate current government response data based on the target historical government question data; the historical government question data includes historical government question sub-data in multiple modalities; The acquisition module is also used to preprocess the historical government affairs inquiry sub-data of the multiple modalities based on the data types corresponding to the historical government affairs inquiry sub-data of the multiple modalities, so as to obtain historical preprocessed sub-data of the multiple modalities. Feature extraction is performed on the historical preprocessed sub-data of the various modalities to obtain multiple historical initial features; By filtering all the aforementioned initial historical features, multiple historical query features are obtained; A fusion operation is performed on multiple historical question features to obtain multiple historical question feature vectors; The processing module is further configured to include multiple modalities of current government affairs question sub-data in the current government affairs question data; the step of constructing a current question feature vector based on the current government affairs question data specifically includes: The context is obtained based on multiple historical government affairs inquiry data and the current government affairs inquiry data; Based on the data types corresponding to the current government affairs question sub-data of the various modalities, the current government affairs question sub-data of the various modalities is preprocessed to obtain the current preprocessed sub-data of the various modalities; Based on the aforementioned context, the current preprocessed sub-data of the various modalities are analyzed to obtain multiple current initial features; The current initial features are weighted and prioritized to obtain multiple current query features. A fusion operation is performed on multiple current question features to obtain the current question feature vector; Obtain historical government response data corresponding to each of the aforementioned historical government question data; Based on the target historical government affairs question data, determine the target historical government affairs answer data corresponding to the target historical government affairs question data from the historical government affairs answer data; Based on the target's historical government affairs response data, generate the current government affairs response data; Based on the target's historical government affairs response data, construct a target historical response feature vector; Based on the target historical answer feature vector and the current question feature vector, calculate the second degree of correlation between the current government question data and the target historical government answer data; Reference historical government response data is determined from multiple target historical government response data; the reference historical government response data is target historical government response data whose second correlation degree is greater than a preset second correlation threshold. The current government response data is generated based on the historical government response data.

5. An electronic device, characterized in that, The device includes a processor (501), a memory (505), a user interface (503), and a network interface (504). The memory (505) is used to store instructions. The user interface (503) and the network interface (504) are used to communicate with other devices. The processor (501) is used to execute the instructions stored in the memory (505) to cause the electronic device (500) to perform the method as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-3.

Images