LLM-based data interpretation method and system, electronic equipment and storage medium

Abstract

The invention provides an LLM-based data interpretation method and system, electronic equipment and a storage medium, and belongs to the field of data interpretation. The method comprises the steps that a to-be-interpreted data set is acquired, and the data set is a comprehensive data set containing text data, image data and video data; converting the image data and the video data into corresponding text data, and selecting a corresponding interpretation mode for analysis according to the type of the text data to obtain analysis data of texts, images and videos; and combining the analysis data of the text, the image and the video into prompts, inputting the prompts into the LLM large model, and outputting interpretation results of the text data, the image data and the video data. By optimizing the interpretation process of various types of data, including texts, images and videos, the interpretation efficiency, accuracy and depth are improved, and the limitation of an existing data interpretation method is broken through.

Description

Technical Field

[0001] The present invention belongs to the technical field of data interpretation, and specifically relates to a data interpretation method, system, electronic device and storage medium based on LLM. Background Art

[0002] With the rapid development of information technology, the speed of generating and accumulating text, image and video data is accelerating. These data contain rich information and knowledge. How to effectively extract valuable information from these data has become a common challenge faced by all walks of life. In many fields such as education, entertainment, security monitoring, market analysis, etc., the demand for automatic interpretation of text, image and video data is growing.

[0003] Traditional data interpretation methods often rely on manual operations. For example, text analysis requires professional linguists to analyze the content, and image and video analysis requires image processing experts to extract features and recognize patterns. These methods are not only time-consuming and labor-intensive, but also limited by the personal experience and subjective judgment of experts, making it difficult to adapt to the rapid growth of data volume and the diversification of data types.

[0004] In the field of text data, such as massive posts on social media, news reports and academic papers, traditional text analysis methods require manual keyword extraction, topic modeling and sentiment analysis, which are cumbersome and inefficient. In the field of image and video data, such as images captured by surveillance cameras, pictures and videos circulated on the Internet, traditional image processing technology requires experts to perform scene recognition, target tracking and event detection, which are complex tasks and require high professional knowledge.

[0005] In addition, existing data interpretation systems are mostly designed for specific types of data and are difficult to adapt to comprehensive datasets containing multiple data types such as text, images, and videos. Summary of the invention

[0006] The purpose of the embodiments of the present invention is to provide a data interpretation method, system, electronic device and storage medium based on LLM, which are used to fully or at least partially solve the technical problems existing in the above-mentioned prior art that it is incapable of adapting to comprehensive data sets containing multiple data types and has low efficiency.

[0007] In a first aspect, an embodiment of the present application provides a data interpretation method based on LLM, comprising: Acquire a data set to be interpreted, wherein the data set is a comprehensive data set including text data, image data, and video data; Convert the image data and the video data into corresponding text data, and select a corresponding interpretation method for analysis according to the type of the text data to obtain analysis data of the text, image and video; The analysis data of the text, image and video are combined into prompts and input into the LLM large model, and the interpretation results of the text data, image data and video data are output.

[0008] Optionally, converting the image data and the video data into corresponding text data includes: Performing data cleaning on the image data and the video data to obtain target image data and target video data; For image data or video data, AI technology is used to extract features from a single image or a single video to identify the target object in the single image or a single video; Use object detection algorithms to classify and locate the identified target objects, apply image description technology to provide text descriptions, and generate text data based on the obtained text descriptions.

[0009] Optionally, a corresponding interpretation method is selected according to the type of text data to be analyzed to obtain analysis data of text, image and video, including: Clean the text data to obtain the target text data; Classifying the target text data to obtain a single text document type and a multi-text document type, wherein the multi-text document type includes a multi-quantity text document type and a multi-category text document type; For single text document types, perform topic modeling, keyword extraction, word frequency analysis, sentiment analysis, and summary generation; For multiple text document types, keyword extraction and short summary generation are performed for each text, cluster analysis is performed based on keywords, and then sentiment analysis is performed within each cluster, and the cluster center document is treated as a single document type; For multiple types of text documents, topic modeling and sentiment analysis are performed within each text category.

[0010] Optionally, for the single text document type, a word frequency statistical analysis is performed, the word frequency-inverse document frequency is used to find the key words in the text, and the keyword extraction is performed to generate a summary, and then a sentiment analysis is performed on the document to determine the emotional tendency of the single text document, and finally the theme of the document is analyzed by topic modeling to obtain the single text document type analysis result; For the multiple text document types, use word frequency-inverse document frequency to find out the key words in the text, perform keyword extraction to generate a summary, perform cluster analysis based on the extracted keywords, obtain the keyword closest to the cluster center in each cluster as the cluster sample document, and perform topic modeling. For each cluster, perform word frequency-inverse document frequency analysis and sentiment analysis respectively to obtain the analysis results of multiple text document types; For the multiple categories of text document types, topic modeling and sentiment analysis are performed on the documents in each category respectively, and the overall topic modeling distribution and sentiment trend of the documents in the category are obtained based on statistical analysis of the topic modeling results and sentiment analysis results of each document.

[0011] Optionally, the analysis data of the text, image and video are combined into prompts and input into the LLM large model, and the interpretation results of the text data, image data and video data are output, including: The summaries, sentiment analysis results, and document topics of a single text document type, multiple text document types, and multiple types of text document types are input into multiple Transformer layers in the LLM large model. Based on the self-attention mechanism set in each Transformer layer, the summaries, sentiment analysis results, and document topics of a single text document type, multiple text document types, and multiple types of text document types are processed in parallel to obtain multiple interpretation results. Multiple interpretation results are fused to obtain interpretation results of text data, image data and video data.

[0012] In a second aspect, the embodiment of the present application further provides a data interpretation system based on LLM, comprising: An acquisition unit, used for acquiring a data set to be interpreted, wherein the data set is a comprehensive data set including text data, image data and video data; An analysis unit, used to convert the image data and the video data into corresponding text data, and select a corresponding interpretation method for analysis according to the type of the text data to obtain analysis data of the text, image and video; The interpretation unit is used to combine the analysis data of the text, image and video into prompts and input them into the LLM large model, and output the interpretation results of the text data, image data and video data.

[0013] Optionally, the analysis unit is specifically used for: Clean the text data to obtain the target text data; Classifying the target text data to obtain a single text document type and a multi-text document type, wherein the multi-text document type includes a multi-quantity text document type and a multi-category text document type; For single text document types, perform topic modeling, keyword extraction, word frequency analysis, sentiment analysis, and summary generation; For multiple text document types, keyword extraction and short summary generation are performed for each text, cluster analysis is performed based on keywords, and then sentiment analysis is performed within each cluster, and the cluster center document is treated as a single document type; For multiple types of text documents, topic modeling and sentiment analysis are performed within each text category.

[0014] Optionally, the analysis unit is specifically used for: For the single-text document type, a word frequency statistical analysis is performed, and the key words in the text are found using word frequency-inverse document frequency, and keyword extraction is performed to generate a summary. Then, a sentiment analysis is performed on the document to determine the emotional tendency of the single-text document. Finally, topic modeling is used to analyze the theme of the document to obtain the single-text document type analysis result; For the multiple text document types, use word frequency-inverse document frequency to find out the key words in the text, perform keyword extraction to generate a summary, perform cluster analysis based on the extracted keywords, obtain the keyword closest to the cluster center in each cluster as the cluster sample document, and perform topic modeling. For each cluster, perform word frequency-inverse document frequency analysis and sentiment analysis respectively to obtain the analysis results of multiple text document types; For the multiple categories of text document types, topic modeling and sentiment analysis are performed on the documents in each category respectively, and the overall topic modeling distribution and sentiment trend of the documents in the category are obtained based on statistical analysis of the topic modeling results and sentiment analysis results of each document.

[0015] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned LLM-based data interpretation method when executing the program.

[0016] In a fourth aspect, an embodiment of the present application further provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned LLM-based data interpretation method are implemented.

[0017] It can be seen from the above technical solutions that the present invention has the following advantages: In the LLM-based data interpretation method, system, electronic device and storage medium provided in the present application, images and videos in the comprehensive data containing text, images and videos are converted into text, the corresponding interpretation method is selected for analysis based on the type of text, and the interpretation result is automatically obtained in combination with the LLM large model, thereby improving the efficiency and accuracy of data interpretation, lowering the threshold for data interpretation, adapting to diverse data environments, and meeting the needs of different industries and fields for data interpretation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0019] Figure 1 A flowchart of a data interpretation method based on LLM provided by an embodiment of the present invention; Figure 2 A flow chart of a text data interpretation method provided by an embodiment of the present invention; Figure 3 A detailed implementation flow chart of a data interpretation method based on LLM provided in an embodiment of the present invention; Figure 4 A schematic diagram of the structure of a data interpretation system based on LLM provided by an embodiment of the present invention; Figure 5 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In the detailed description below, various embodiments of the present disclosure will be described more fully. The present disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but rather the present disclosure should be understood to cover all adjustments, equivalents and / or alternatives that fall within the spirit and scope of the various embodiments of the present disclosure.

[0021] Hereinafter, the terms "include" or "may include" that may be used in various embodiments of the present disclosure indicate the presence of the disclosed functions or operations, and do not limit the addition of one or more functions or operations. In addition, as used in various embodiments of the present disclosure, the terms "include", "have" and their cognates are intended only to indicate specific features, numbers, steps, operations, or combinations of the foregoing items, and should not be understood as first excluding the presence of one or more other features, numbers, steps, operations, or combinations of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, or combinations of the foregoing items.

[0022] In various embodiments of the present disclosure, the expression "or" or "at least one of A or / and B" includes any combination or all combinations of the words listed at the same time. For example, the expression "A or B" or "at least one of A or / and B" may include A, may include B, or may include both A and B.

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] See also Figure 1 The figure is a flowchart of a data interpretation method based on LLM in a specific embodiment, including the following execution steps: Step 100: Obtain a data set to be interpreted, wherein the data set is a comprehensive data set including text data, image data, and video data.

[0025] Step 101: convert the image data and the video data into corresponding text data, and select a corresponding interpretation method for analysis according to the type of the text data to obtain analysis data of the text, image and video.

[0026] Specifically, converting the image data and the video data into corresponding text data includes the following steps: performing data cleaning on the image data and the video data to obtain target image data and target video data; using AI technology to extract features of a single image or a single video for the image data or the video data, and identifying the target object in the single image or the single video; using an object detection algorithm to classify and locate the identified target object, and applying image description technology to perform text description, and generating text data based on the obtained text description.

[0027] Exemplarily, feature extraction of image data includes converting image files into a format that can be processed by a computer, removing noise from the image, and improving image quality. Find stable and highly repeatable key points in the image, which are usually located at locations with significant geometric and texture features in the image, such as edges, corners, spots, or line intersections. Generate a numerical vector for each detected feature point to describe the feature information of the neighborhood around the point in detail. Feature extraction of video data includes converting video files into a format that can be processed by a computer, such as decoding compressed video files into image frames, splitting the video into a series of static images (frames), and these image frames can be used as input data for subsequent processing. The optical flow method is used to extract features such as motion information, edge information, and texture information in the video. Use deep learning models such as convolutional neural networks (CNNs) to automatically extract high-level features from video frames. For each video frame, one or more feature vectors are extracted. These feature vectors can be used for subsequent video analysis tasks, such as action recognition, object tracking, scene classification, etc.

[0028] In some embodiments, for image and video data, data cleaning is first performed, such as denoising (filters remove noise in images, such as median filtering, Gaussian filtering, etc.), adjusting brightness and contrast, adjusting size and orientation, and ensuring data consistency. For video data, inter-frame noise reduction can also be performed to reduce noise between consecutive frames. And resolution adjustment to meet different analysis needs. Data interpretation is performed on a single image and a single video, and AI technology is used to extract features from the image / video, identify the main objects in the image / video, and use object detection algorithms to classify and locate the identified main objects. Image description technology is used for text description. Prompts are generated based on the obtained text description and input into the large model to obtain the deep meaning contained in the image / video. For example, an image text description is a group of children making a snowman together in the snow, but there is a child watching not far away. This text description is input into the large model, hoping that the large model will analyze that the child in the distance is afraid of the cold or isolated.

[0029] In some implementations, for text data (pdf, doc, docx, html, xml files, etc.), the Agent calls a text cleaning operator to clean the text data, including removing irrelevant characters and stop words. For English text, operations such as case conversion, part of speech restoration, and word segmentation are performed.

[0030] Step 102: Combine the analysis data of the text, image and video into prompts and input them into the LLM large model, and output the interpretation results of the text data, image data and video data.

[0031] Specifically, the summaries, sentiment analysis results and document topics of a single text document type, multiple text document types and multiple types of text document types are input into multiple Transformer layers in the LLM large model, and the summaries, sentiment analysis results and document topics of a single text document type, multiple text document types and multiple types of text document types are processed in parallel based on the self-attention mechanism set in each Transformer layer to obtain multiple interpretation results; the multiple interpretation results are fused to obtain interpretation results of text data, image data and video data.

[0032] In this embodiment, text, image and video data can be automatically processed and interpreted, the efficiency and accuracy of data interpretation can be improved, the threshold for data interpretation can be lowered, it can adapt to diverse data environments, and meet the needs of different industries and fields for data interpretation.

[0033] In one embodiment of the present invention, referring to Figure 2 As shown, based on step 101, a possible example will be given below to illustrate its specific implementation scheme in a non-limiting manner.

[0034] S1: Clean the text data to obtain target text data.

[0035] S2: Classifying the target text data to obtain a single text document type and a multi-text document type, wherein the multi-text document type includes a plurality of quantity text document types and a plurality of types text document types.

[0036] S3: For single text document types, perform topic modeling, keyword extraction, word frequency analysis, sentiment analysis, and summary generation.

[0037] Specifically, for the single-text document type, a word frequency statistical analysis is performed, and the key words in the text are found using word frequency-inverse document frequency, and keyword extraction is performed to generate a summary. Then, a sentiment analysis is performed on the document to determine the emotional tendency of the single-text document. Finally, topic modeling is used to analyze the subject of the document to obtain the single-text document type analysis result.

[0038] In some implementations, the designed single text document prompt template is as follows: “The following is the statistical information of a single text data: 1. Text Subject: {Enter the subject or title of the document here}; 2. Keyword extraction results: {[keyword 1], [keyword 2], ...}; 3. Term frequency-inverse document frequency results: {[keyword 1]: TF-IDF value, [keyword 2]: TF-IDF value, ...}; 4. Text summary results: {a brief summary of the document is entered here}; 5. Sentiment analysis results: {positive sentiment score: [], negative sentiment score: [], neutral sentiment score: [], overall sentiment tendency: []}; Please provide your interpretation of the document based on the above findings." S4: For multiple text document types, perform keyword extraction and short summary generation for each text, perform cluster analysis based on keywords, then perform sentiment analysis within each cluster, and treat the cluster center document as a single document type.

[0039] Specifically, for the multiple text document types, the word frequency-inverse document frequency is used to find the key words in the text, and keyword extraction is performed to generate a summary. Cluster analysis is performed based on the extracted keywords to obtain the keywords closest to the cluster center in each cluster as the cluster example document, and topic modeling is performed. For each cluster, word frequency-inverse document frequency analysis and sentiment analysis are performed to obtain analysis results of multiple text document types.

[0040] In some implementations, the designed prompt template for a large number of text documents is as follows: “The following is the statistical information of a large number of text data: 1. Cluster analysis: - Cluster 1 {[Number of documents in the cluster]: {Enter the number of documents here}, sentiment analysis results within the cluster: [Positive: {Enter the number of positive sentiment documents in the cluster here}, Negative: {Enter the number of negative sentiment documents in the cluster here}, Neutral: {Enter the number of neutral sentiment documents in the cluster here}]}; -Cluster center document: {Keywords: {[keyword 1], [keyword 2],,,,}; Term frequency-inverse document frequency: {[keyword 1]: TF-IDF value, [keyword 2]: TF-IDF value, ...}; Text topic: {enter the topic or title of the cluster center document here}; Text summary result: {enter a short summary of the cluster center document here}}; - Cluster 2 {repeating cluster 1 structure}; -cluster....

[0041] 2. Overall sentiment analysis results: Number of positive sentiment documents: {enter the total number of positive sentiment documents here}, Number of negative sentiment documents: {enter the total number of negative sentiment documents here}, Number of neutral sentiment documents: {enter the total number of neutral sentiment documents here}; Please provide your interpretation of the document based on the above findings." S5: For multiple types of text documents, topic modeling and sentiment analysis are performed within each text category.

[0042] Specifically, for the multiple categories of text document types, topic modeling and sentiment analysis are performed on the documents in each category, and the overall topic modeling distribution and sentiment trend of the documents in the category are obtained based on statistical analysis of the topic modeling results and sentiment analysis results of each document.

[0043] In some implementations, the designed multiple-type text document prompt template is as follows: “The following is the statistical information of multiple-type text data: 1. Category 1: {enter category name here}; Intra-class sentiment analysis results: {Positive: [enter the number of documents with positive sentiment within the class here], Negative: [enter the number of documents with negative sentiment within the class here], Neutral: [enter the number of documents with neutral sentiment within the class here]; In-category topic modeling results: {Enter the overall topic modeling distribution results within the category here}; 2. Category 2 {repeating Category 1 structure}; Please provide your interpretation of the document based on the above findings." Of course, for a large number of text documents and multiple types of text documents, you can take out the individual documents of interest one by one to interpret the single document text data.

[0044] In some embodiments, the image / video prompt template is as follows: "The image / video text description is: {enter the obtained text description content here}; please use your imagination to speculate what may happen to the object according to the above text description." For a large number of image / video data and multiple types of image / video data, the processing method for a single image / video data is adopted to obtain the corresponding text description, which is then converted into the processing of a large number of documents and multiple types of documents in the text data. Subsequently, corresponding processing can be performed according to the above text data processing method.

[0045] In one embodiment, Figure 3 This is a detailed flow chart of a data interpretation method based on LLM provided according to an embodiment of the present invention. This embodiment is further optimized and expanded on the basis of the above embodiments.

[0046] For text data, text cleaning is performed to remove invalid or irrelevant information, and the cleaned text is classified to obtain single documents, multiple quantities and multiple categories of documents. For single documents, topic modeling, keyword extraction, word frequency analysis, summary generation and sentiment analysis are performed; for multiple quantities of documents, topic modeling, keyword extraction, word frequency analysis, summary generation, sentiment analysis and cluster analysis are performed; for multiple categories of documents, topic modeling and sentiment analysis are performed to obtain analysis results, and the analysis results are combined into appropriate prompts and passed to the large model to obtain the data and obtain the interpretation results; for image or video data, text description is performed, and for single data, the text description is directly passed to the large model to obtain the interpretation result, and for multiple quantities and multiple categories of data, the above processing operations are used for interpretation.

[0047] The technical effects achieved by this application are as follows: 1. Lowering the threshold for data interpretation: non-professionals can easily interpret data without deep data processing background or experience. 2. Improving the efficiency of data interpretation: automated data processing and analysis reduces the time and effort of data scientists to manually process data. 3. Enhancing the accuracy of data interpretation: improving the accuracy of interpretation results by automatically identifying data features and applying the most appropriate analysis methods. 4. Supporting rapid decision-making: providing fast and accurate data interpretation results for content management, market analysis, education, entertainment, and security monitoring, and supporting rapid decision-making.

[0048] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0049] like Figure 4 As shown, the following is an embodiment of the LLM-based data interpretation system provided in an embodiment of the present disclosure, which belongs to the same inventive concept as the LLM-based data interpretation method of the above-mentioned embodiments. For details not described in detail in the embodiment of the LLM-based data interpretation system, please refer to the embodiment of the above-mentioned LLM-based data interpretation method.

[0050] An acquisition unit 40 is used to acquire a data set to be interpreted, wherein the data set is a comprehensive data set including text data, image data and video data; An analysis unit 41 is used to convert the image data and the video data into corresponding text data, and select a corresponding interpretation method for analysis according to the type of the text data to obtain analysis data of the text, image and video; The interpretation unit 42 is used to combine the analysis data of the text, image and video into prompts to input into the LLM large model, and output the interpretation results of the text data, image data and video data.

[0051] In some implementations, the analysis unit 41 is specifically used for: Clean the text data to obtain the target text data; Classifying the target text data to obtain a single text document type and a multi-text document type, wherein the multi-text document type includes a multi-quantity text document type and a multi-category text document type; For single text document types, perform topic modeling, keyword extraction, word frequency analysis, sentiment analysis, and summary generation; For multiple text document types, keyword extraction and short summary generation are performed for each text, cluster analysis is performed based on keywords, and then sentiment analysis is performed within each cluster, and the cluster center document is treated as a single document type; For multiple types of text documents, topic modeling and sentiment analysis are performed within each text category.

[0052] In some implementations, the analysis unit 41 is specifically used for: For the single-text document type, a word frequency statistical analysis is performed, and the key words in the text are found using word frequency-inverse document frequency, and keyword extraction is performed to generate a summary. Then, a sentiment analysis is performed on the document to determine the emotional tendency of the single-text document. Finally, topic modeling is used to analyze the theme of the document to obtain the single-text document type analysis result; For the multiple text document types, use word frequency-inverse document frequency to find out the key words in the text, perform keyword extraction to generate a summary, perform cluster analysis based on the extracted keywords, obtain the keyword closest to the cluster center in each cluster as the cluster sample document, and perform topic modeling. For each cluster, perform word frequency-inverse document frequency analysis and sentiment analysis respectively to obtain the analysis results of multiple text document types; For the multiple categories of text document types, topic modeling and sentiment analysis are performed on the documents in each category respectively, and the overall topic modeling distribution and sentiment trend of the documents in the category are obtained based on statistical analysis of the topic modeling results and sentiment analysis results of each document.

[0053] Figure 5 It is a schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0054] The data interpretation method based on LLM provided in the embodiment of the present application can be applied to electronic devices. Those skilled in the art will appreciate that the electronic device structure involved in the embodiment of the present invention does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown, or combine certain components, or arrange different components. In the embodiment of the present invention, the electronic device includes but is not limited to a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described and / or required herein.

[0055] The electronic device may include a processor, an external memory interface, an internal memory, a universal serial bus (USB) interface, a charging management module, a power management module, a battery, a wireless communication module, an audio module, a speaker, a microphone, a sensor module, buttons, a camera, a display, and a SIM card interface, etc.

[0056] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0057] The processor may include one or more processing units, for example, the processor may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated into one or more processors.

[0058] The processor can be the nerve center and command center of the electronic device. The controller can generate an operation control signal according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0059] A memory may also be provided in the processor for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. The memory may store instructions or data that the processor has just used or is cyclically used. If the processor needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor, and thus improves system efficiency.

[0060] The external memory interface can be used to connect an external memory card, such as a MicroSD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor through the external memory interface to implement data storage functions. For example, files such as music and videos can be saved in the external memory card.

[0061] The internal memory can be used to store computer executable program codes, which include instructions. The processor executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory. The internal memory may include a program storage area and a data storage area. The internal memory may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0062] The wireless communication function of an electronic device can be realized through an antenna, a wireless communication module, a modem processor, and a baseband processor.

[0063] The wireless communication module can provide wireless communication solutions for electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc.

[0064] Electronic devices can implement audio functions, etc. through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0065] Electronic devices can achieve shooting functions through ISP, camera, video codec, GPU, display and application processor.

[0066] Electronic devices can achieve display functions through GPU, display screen and application processor.

[0067] The GPU is a microprocessor for image processing that connects the display screen and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor may include one or more GPUs that execute program instructions to generate or change display information.

[0068] The display screen is used to display images, videos, etc. The display screen includes a display panel.

[0069] The storage medium provided in the present application stores a program product that can implement the data interpretation method based on LLM.

[0070] The data interpretation method based on LLM includes: obtaining a data set to be interpreted, wherein the data set is a comprehensive data set including text data, image data and video data; converting the image data and the video data into corresponding text data, and selecting a corresponding interpretation method for analysis according to the type of the text data to obtain analysis data of the text, image and video; combining the analysis data of the text, image and video into prompts and inputting them into the LLM large model, and outputting the interpretation results of the text data, image data and video data.

[0071] In some possible implementations, the subject matter of the present disclosure, namely, the data interpretation method and system based on LLM, can be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary implementations of the present disclosure described in the above “Exemplary Method” section of this specification.

[0072] The storage medium of the present disclosure can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0073] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data interpretation method based on LLM, characterized in that: include: Acquire a data set to be interpreted, wherein the data set is a comprehensive data set including text data, image data, and video data; Convert the image data and the video data into corresponding text data, and select a corresponding interpretation method for analysis according to the type of the text data to obtain analysis data of the text, image and video; The analysis data of the text, image and video are combined into prompts and input into the LLM large model, and the interpretation results of the text data, image data and video data are output.

2. The data interpretation method based on LLM according to claim 1, characterized in that: Converting the image data and the video data into corresponding text data includes: Performing data cleaning on the image data and the video data to obtain target image data and target video data; For image data or video data, AI technology is used to extract features from a single image or a single video to identify the target object in the single image or a single video; Use object detection algorithms to classify and locate the identified target objects, apply image description technology to provide text descriptions, and generate text data based on the obtained text descriptions.

3. The data interpretation method based on LLM according to claim 1, characterized in that: According to the type of text data, the corresponding interpretation method is selected for analysis to obtain analysis data of text, images and videos, including: Clean the text data to obtain the target text data; Classifying the target text data to obtain a single text document type and a multi-text document type, wherein the multi-text document type includes a multi-quantity text document type and a multi-category text document type; For single text document types, perform topic modeling, keyword extraction, word frequency analysis, sentiment analysis, and summary generation; For multiple text document types, keyword extraction and short summary generation are performed for each text, cluster analysis is performed based on keywords, and then sentiment analysis is performed within each cluster, and the cluster center document is treated as a single document type; For multiple types of text documents, topic modeling and sentiment analysis are performed within each text category.

4. The data interpretation method based on LLM according to claim 3, characterized in that: For the single-text document type, a word frequency statistical analysis is performed, and the key words in the text are found using word frequency-inverse document frequency, and keyword extraction is performed to generate a summary. Then, a sentiment analysis is performed on the document to determine the emotional tendency of the single-text document. Finally, topic modeling is used to analyze the theme of the document to obtain the single-text document type analysis result; For the multiple text document types, use word frequency-inverse document frequency to find out the key words in the text, perform keyword extraction to generate a summary, perform cluster analysis based on the extracted keywords, obtain the keyword closest to the cluster center in each cluster as the cluster sample document, and perform topic modeling. For each cluster, perform word frequency-inverse document frequency analysis and sentiment analysis respectively to obtain the analysis results of multiple text document types; For the multiple categories of text document types, topic modeling and sentiment analysis are performed on the documents in each category respectively, and the overall topic modeling distribution and sentiment trend of the documents in the category are obtained based on statistical analysis of the topic modeling results and sentiment analysis results of each document.

5. The data interpretation method based on LLM according to claim 4, characterized in that: The analysis data of the text, image and video are combined into prompts and input into the LLM large model, and the interpretation results of the text data, image data and video data are output, including: The summaries, sentiment analysis results, and document topics of a single text document type, multiple text document types, and multiple types of text document types are input into multiple Transformer layers in the LLM large model. Based on the self-attention mechanism set in each Transformer layer, the summaries, sentiment analysis results, and document topics of a single text document type, multiple text document types, and multiple types of text document types are processed in parallel to obtain multiple interpretation results. Multiple interpretation results are fused to obtain interpretation results of text data, image data and video data.

6. A data interpretation system based on LLM, characterized in that: include: An acquisition unit, used for acquiring a data set to be interpreted, wherein the data set is a comprehensive data set including text data, image data and video data; An analysis unit, used to convert the image data and the video data into corresponding text data, and select a corresponding interpretation method for analysis according to the type of the text data to obtain analysis data of the text, image and video; The interpretation unit is used to combine the analysis data of the text, image and video into prompts and input them into the LLM large model, and output the interpretation results of the text data, image data and video data.

7. The LLM-based data interpretation system according to claim 6, characterized in that: The analysis unit is specifically used for: Clean the text data to obtain the target text data; Classifying the target text data to obtain a single text document type and a multi-text document type, wherein the multi-text document type includes a multi-quantity text document type and a multi-category text document type; For single text document types, perform topic modeling, keyword extraction, word frequency analysis, sentiment analysis, and summary generation; For multiple text document types, keyword extraction and short summary generation are performed for each text, cluster analysis is performed based on keywords, and then sentiment analysis is performed within each cluster, and the cluster center document is treated as a single document type; For multiple types of text documents, topic modeling and sentiment analysis are performed within each text category.

8. The LLM-based data interpretation system according to claim 7, characterized in that: The analysis unit is specifically used for: For the single-text document type, a word frequency statistical analysis is performed, and the key words in the text are found using word frequency-inverse document frequency, and keyword extraction is performed to generate a summary. Then, a sentiment analysis is performed on the document to determine the emotional tendency of the single-text document. Finally, topic modeling is used to analyze the theme of the document to obtain the single-text document type analysis result; For the multiple text document types, use word frequency-inverse document frequency to find out the key words in the text, perform keyword extraction to generate a summary, perform cluster analysis based on the extracted keywords, obtain the keyword closest to the cluster center in each cluster as the cluster sample document, and perform topic modeling. For each cluster, perform word frequency-inverse document frequency analysis and sentiment analysis respectively to obtain the analysis results of multiple text document types; For the multiple categories of text document types, topic modeling and sentiment analysis are performed on the documents in each category respectively, and the overall topic modeling distribution and sentiment trend of the documents in the category are obtained based on statistical analysis of the topic modeling results and sentiment analysis results of each document.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the LLM-based data interpretation method according to any one of claims 1 to 5 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the LLM-based data interpretation method according to any one of claims 1 to 5 are implemented.

Images