Multimodal data analysis chart generation method based on artificial intelligence

Abstract

The present invention relates to the technical field of analysis chart generation, and in particular to a method for generating multimodal data analysis charts based on artificial intelligence. The method comprises: obtaining a multimodal data analysis chart sample set, wherein a single analysis chart sample element in the multimodal data analysis chart sample set contains four modal data, specifically text data, image data, audio data, and structured data; performing preprocessing and multimodal feature fusion operations on the single analysis chart sample element, and outputting a feature matrix of the multimodal data analysis chart sample set; constructing an importance value prediction model to predict the importance value of the analysis chart sample element and calculate the importance coefficient of the single analysis chart sample element; constructing a communication power prediction model to predict the communication power of the analysis chart sample element and calculate the communication power coefficient of the single analysis chart sample element; and calculating the priority coefficient for analysis chart generation to ensure that high-priority analysis charts are generated first.

Description

Technical Field

[0001] The present invention relates to the technical field of analysis chart generation, and in particular to a method for generating multimodal data analysis charts based on artificial intelligence. Background Art

[0002] With the explosive growth of data, data modalities are becoming increasingly diverse, encompassing a wide range of types, including text, images, audio, and structured data. In the field of data analysis, efficiently processing and analyzing this multimodal data and presenting the results in intuitive charts has become a key requirement. Multimodal data analysis charts can integrate different types of data information to provide users with more comprehensive and in-depth insights, and are of great value in many fields, including business decision-making, scientific research, and information dissemination. For example, in business analysis, combining multimodal data such as text descriptions, sales data charts, market trend images, and consumer feedback audio can more accurately grasp market dynamics and consumer needs.

[0003] When processing multimodal data, existing technologies often process different modal data separately, failing to fully explore the potential connections and complementary information between modal data, resulting in an incomplete and in-depth understanding and analysis of the data. In the chart generation process, factors such as the importance and dissemination of data are usually not fully considered. The generated charts may not highlight key information, nor can they meet the chart needs of different users in different scenarios, reducing the practicality and value of the charts. Without dynamically allocating computing resources based on data priority, computing resources may be wasted on generating low-value charts, while important charts are generated slowly due to insufficient resources, affecting the efficiency and timeliness of data analysis. Summary of the Invention

[0004] The main purpose of the present invention is to provide a multimodal data analysis chart generation method based on artificial intelligence. By obtaining a set of multimodal data analysis chart samples containing text, images, audio and structured data, and performing preprocessing and multimodal feature fusion, it can fully explore the intrinsic connections between different modal data, comprehensively reflect the data characteristics, and lay the foundation for more accurate analysis and chart generation; construct an important value prediction model and a communication power prediction model to predict the importance value and communication power of the analysis chart sample elements respectively, and calculate the corresponding coefficients. Based on these coefficients, priority coefficients are generated so that the generated charts can better highlight key information and meet the user's attention needs for important data. At the same time, the influence of the charts in the communication process is also taken into account to improve the practicality and communication value of the charts; computing resources are dynamically allocated according to the priority coefficient to ensure that high-priority analysis charts are generated first, thereby optimizing the utilization efficiency of computing resources, avoiding resource waste, speeding up the generation speed of important charts, improving the overall data analysis efficiency, and providing support for decision-making more quickly.

[0005] The technical solutions of the present invention are as follows:

[0006] First, a method for generating multimodal data analysis charts based on artificial intelligence is proposed, which includes the following steps:

[0007] S1. Acquire a multimodal data analysis chart sample set, wherein a single analysis chart sample element in the multimodal data analysis chart sample set includes four modal data, specifically text data, image data, audio data, and structured data;

[0008] S2. performing preprocessing and multimodal feature fusion operations on individual analysis chart sample elements in the multimodal data analysis chart sample set, and outputting a feature matrix of the multimodal data analysis chart sample set;

[0009] S3. Construct an importance value prediction model to predict the importance values ​​of sample elements in the analysis chart, and calculate the importance coefficient of each sample element in the analysis chart based on the importance value prediction results;

[0010] S4. Construct a communication force prediction model to predict the communication force of the sample elements of the analysis chart, and calculate the communication force coefficient of a single analysis chart sample element;

[0011] S5. Based on the importance coefficient and communication coefficient of the analysis chart sample elements, calculate the priority coefficient of the analysis chart generation, and dynamically allocate computing resources to ensure that high-priority analysis charts are generated first.

[0012] A further improvement of the present invention is that S2 comprises the following specific steps:

[0013] S21, extracting a sample set M of multimodal data analysis charts, ;in, Represents the i-th analysis chart sample element; for a single analysis chart sample element , use natural language processing technology to preprocess text data and generate word frequency vectors ; Extract visual features from image data through convolutional neural network and generate visual feature vectors ; Generate spectral feature vectors in audio data through Fourier transform ; Standardize structured data to generate numerical vectors ;

[0014] S22. Extracting a single analysis chart sample element The word frequency vector , visual feature vector , spectral feature vector and numerical vectors Splice and output a single analysis chart sample element The eigenvector of , ; Output the feature matrix X of the multimodal data analysis chart sample set M, , T represents the transpose of the matrix.

[0015] A further improvement of the present invention is that S3 includes the following specific steps:

[0016] S31. Label the important value label for each analysis chart sample element in the multimodal data analysis chart sample set to obtain an important value label vector ,in, Sample element for a single analysis chart The important value label value of

[0017] S32. Construct a feature matrix X whose input is a set of multimodal data analysis chart samples M, and whose output is an important value prediction model of the important value of a single analysis chart sample element. The important value prediction model uses the important value corresponding to a single analysis chart sample element as the prediction target, and minimizes the sum of the squares of the errors between the predicted important value and the important value label value as the training target. The important value prediction model is trained until the sum of the squares of the errors between the predicted important value and the important value label value converges, and the training is stopped. The important value prediction model is output to obtain the important value of a single analysis chart sample element. The important value prediction model is any one of a naive Bayes algorithm, a random forest model, a support vector machine, or a neural network model.

[0018] A further improvement of the present invention is that S3 further includes:

[0019] S33. Calculate the similarity coefficient between the sample elements of the analysis chart. The calculation formula of the similarity coefficient is:

[0020] ;

[0021] in, Represents an analysis chart sample element Sample elements for infographics with analytics The similarity coefficient between is the smoothing factor and , Represents an analysis chart sample element Sample elements for infographics with analytics KL divergence of the data distribution between; Sample elements for analysis charts The important value of

[0022] S34. Calculate the importance coefficient of sample element i of the analysis chart. The calculation formula of the importance coefficient is:

[0023] ;

[0024] in, Represents an analysis chart sample element The importance coefficient of .

[0025] A further improvement of the present invention is that S4 includes the following specific steps:

[0026] S41. Label a communication force label for a single analysis chart sample element in the multimodal data analysis chart sample set to obtain a communication force label vector ,in, Sample element for a single analysis chart The spreadability label value of

[0027] S42. Construct a feature matrix X whose input is a set of multimodal data analysis chart samples M, and whose output is a communication force prediction model of the communication force of a single analysis chart sample element. The communication force prediction model uses the communication force corresponding to a single analysis chart sample element as the prediction target, and minimizes the sum of the squares of the errors between the predicted communication force and the communication force label value as the training target. The communication force prediction model is trained until the sum of the squares of the errors between the predicted communication force and the communication force label value converges, and the training is stopped. The communication force prediction model is output to obtain the communication force of a single analysis chart sample element.

[0028] A further improvement of the present invention is that the step S4 further comprises:

[0029] S43. Calculate the propagation coefficient of a single analysis chart sample element. The calculation formula of the propagation coefficient is: ;in, Represents an analysis chart sample element The transmission coefficient of Analysis chart sample elements The spread of Represents an analysis chart sample element Number of citations, represents the weight of the spread breadth, and , represents the time decay coefficient, and , Sample elements of a display analysis chart The number of views on day t, where T represents the sample element of the analysis chart Total number of days of public access.

[0030] A further improvement of the present invention is that the communication power prediction model in S42 is any one of a deep neural network model and a deep belief network model.

[0031] A further improvement of the present invention is that the calculation formula of the priority coefficient in S5 is:

[0032] ;

[0033] in, The data update interval, 、 、 are weight factors respectively, and there are .

[0034] In a second aspect, a computer-readable storage medium is proposed, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned method for generating multimodal data analysis charts based on artificial intelligence is implemented.

[0035] In a third aspect, an electronic device is proposed, comprising a memory for storing instructions; and a processor for executing the instructions, so that the device implements the above-mentioned method for generating multimodal data analysis charts based on artificial intelligence.

[0036] The technical effects of the present invention are as follows:

[0037] A multimodal data analysis chart generation method based on artificial intelligence was constructed. By obtaining a set of multimodal data analysis chart samples containing text, images, audio and structured data, and performing preprocessing and multimodal feature fusion, it can fully explore the intrinsic connections between different modal data, comprehensively reflect data characteristics, and lay the foundation for more accurate analysis and chart generation; construct an important value prediction model and a communication power prediction model to respectively predict the importance value and communication power of the analysis chart sample elements, and calculate the corresponding coefficients. Based on these coefficients, priority coefficients are generated, so that the generated charts can better highlight key information and meet users' attention needs for important data. At the same time, the influence of charts in the communication process is also taken into account to improve the practicality and communication value of charts; computing resources are dynamically allocated according to the priority coefficient to ensure that high-priority analysis charts are generated first, which optimizes the utilization efficiency of computing resources, avoids resource waste, speeds up the generation of important charts, improves overall data analysis efficiency, and can provide support for decision-making more quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0039] Figure 1This is a flow chart of a method for generating multimodal data analysis charts based on artificial intelligence according to Example 1 of the present invention. DETAILED DESCRIPTION

[0040] Example 1

[0041] This embodiment proposes a method for generating multimodal data analysis charts based on artificial intelligence. By obtaining a set of multimodal data analysis chart samples containing text, images, audio and structured data, and performing preprocessing and multimodal feature fusion, it can fully explore the intrinsic connections between different modal data, comprehensively reflect data characteristics, and lay the foundation for more accurate analysis and chart generation; construct an important value prediction model and a communication power prediction model to respectively predict the importance value and communication power of the analysis chart sample elements, and calculate the corresponding coefficients. Based on these coefficients, priority coefficients are calculated to make the generated charts more able to highlight key information and meet the user's attention needs for important data. At the same time, the influence of the charts in the communication process is also taken into account to improve the practicality and communication value of the charts; computing resources are dynamically allocated according to the priority coefficient to ensure that high-priority analysis charts are generated first, thereby optimizing the utilization efficiency of computing resources, avoiding resource waste, speeding up the generation of important charts, improving the overall data analysis efficiency, and providing support for decision-making more quickly.

[0042] Specifically, such as Figure 1 As shown, the method for generating a multimodal data analysis chart based on artificial intelligence proposed in this embodiment includes the following specific steps:

[0043] S1. Acquire a multimodal data analysis chart sample set, wherein a single analysis chart sample element in the multimodal data analysis chart sample set includes four modal data, specifically text data, image data, audio data, and structured data;

[0044] S2. performing preprocessing and multimodal feature fusion operations on individual analysis chart sample elements in the multimodal data analysis chart sample set, and outputting a feature matrix of the multimodal data analysis chart sample set;

[0045] S3. Construct an importance value prediction model to predict the importance values ​​of sample elements in the analysis chart, and calculate the importance coefficient of each sample element in the analysis chart based on the importance value prediction results;

[0046] S4. Construct a communication force prediction model to predict the communication force of the sample elements of the analysis chart, and calculate the communication force coefficient of a single analysis chart sample element;

[0047] S5. Based on the importance coefficient and communication coefficient of the analysis chart sample elements, calculate the priority coefficient of the analysis chart generation, and dynamically allocate computing resources to ensure that high-priority analysis charts are generated first.

[0048] In this embodiment, S2 includes the following specific steps:

[0049] S21, extracting a sample set M of multimodal data analysis charts, ;in, Represents the i-th analysis chart sample element; for a single analysis chart sample element , use natural language processing technology to preprocess text data and generate word frequency vectors ; Extract visual features from image data through convolutional neural network and generate visual feature vectors ; Generate spectral feature vectors in audio data through Fourier transform ; Standardize structured data to generate numerical vectors ;

[0050] S22. Extracting a single analysis chart sample element The word frequency vector , visual feature vector , spectral feature vector and numerical vectors Splice and output a single analysis chart sample element The eigenvector of , ; Output the feature matrix X of the multimodal data analysis chart sample set M, , T represents the transpose of the matrix.

[0051] In this embodiment, S3 includes the following specific steps:

[0052] S31. Label the important value label for each analysis chart sample element in the multimodal data analysis chart sample set to obtain an important value label vector ,in, Sample element for a single analysis chart The important value label value of

[0053] S32. Construct a feature matrix X whose input is a set of multimodal data analysis chart samples M, and whose output is an important value prediction model of the important value of a single analysis chart sample element. The important value prediction model uses the important value corresponding to a single analysis chart sample element as the prediction target, and minimizes the sum of the squares of the errors between the predicted important value and the important value label value as the training target. The important value prediction model is trained until the sum of the squares of the errors between the predicted important value and the important value label value converges, and the training is stopped. The important value prediction model is output to obtain the important value of a single analysis chart sample element. The important value prediction model is any one of a naive Bayes algorithm, a random forest model, a support vector machine, or a neural network model.

[0054] In this embodiment, S3 further includes:

[0055] S33. Calculate the similarity coefficient between the sample elements of the analysis chart. The calculation formula of the similarity coefficient is:

[0056] ;

[0057] in, Represents an analysis chart sample element Sample elements for infographics with analytics The similarity coefficient between is the smoothing factor and , Represents an analysis chart sample element Sample elements for infographics with analytics KL divergence of the data distribution between; Sample elements for analysis charts The important value of

[0058] S34. Calculate the importance coefficient of sample element i of the analysis chart. The calculation formula of the importance coefficient is:

[0059] ;

[0060] in, Represents an analysis chart sample element The importance coefficient of .

[0061] In this embodiment, S4 includes the following specific steps:

[0062] S41. Label a communication force label for a single analysis chart sample element in the multimodal data analysis chart sample set to obtain a communication force label vector ,in, Sample element for a single analysis chart The spreadability label value of

[0063] S42. Construct a feature matrix X whose input is a set of multimodal data analysis chart samples M, and whose output is a communication force prediction model of the communication force of a single analysis chart sample element. The communication force prediction model uses the communication force corresponding to a single analysis chart sample element as the prediction target, and minimizes the sum of the squares of the errors between the predicted communication force and the communication force label value as the training target. The communication force prediction model is trained until the sum of the squares of the errors between the predicted communication force and the communication force label value converges, and the training is stopped. The communication force prediction model is output to obtain the communication force of a single analysis chart sample element.

[0064] In this embodiment, the S4 further includes:

[0065] S43. Calculate the propagation coefficient of a single analysis chart sample element. The calculation formula of the propagation coefficient is: ;in, Represents an analysis chart sample element The transmission coefficient of Analysis chart sample elements The spread of Represents an analysis chart sample element Number of citations, represents the weight of the spread breadth, and , represents the time decay coefficient, and , Sample elements of a display analysis chart The number of views on day t, where T represents the sample element of the analysis chart Total number of days of public access.

[0066] In this embodiment, the communication power prediction model in S42 is any one of a deep neural network model and a deep belief network model.

[0067] In this embodiment, the calculation formula of the priority coefficient in S5 is:

[0068] ;

[0069] in, The data update interval, 、 、 are weight factors respectively, and there are .

[0070] Example 2

[0071] This embodiment provides an electronic device, including: a processor and a memory, wherein the memory stores a computer program that can be called by the processor; the processor executes the above-mentioned artificial intelligence-based multimodal data analysis chart generation method by calling the computer program stored in the memory.

[0072] The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) and one or more memories, wherein the memories store at least one computer program, which is loaded and executed by the processor to implement the artificial intelligence-based multimodal data analysis chart generation method provided by the above method embodiment. The electronic device may also include other components for realizing the functions of the device. For example, the electronic device may also have components such as a wired or wireless network interface and an input / output interface for data input and output. This embodiment will not be described in detail here.

[0073] Those skilled in the art will appreciate that the present invention may be implemented as a system, method, or computer program product. Therefore, the present disclosure may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the present invention may be implemented as a computer program product embodied in one or more computer-readable media containing computer-readable program code.

[0074] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device.

[0075] The present invention is described with reference to flowcharts and block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process or block in the flowcharts and block diagrams, as well as combinations of processes and blocks in the flowcharts or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts. Figure 1 A process or multiple processes and boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and boxes Figure 1 A step that specifies a function in one or more boxes.

[0077] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which are all protected by the present invention.

Claims

1. A method for generating multimodal data analysis charts based on artificial intelligence, characterized by: The specific steps include: S1. Acquire a multimodal data analysis chart sample set, wherein a single analysis chart sample element in the multimodal data analysis chart sample set includes four modal data, specifically text data, image data, audio data, and structured data; S2. performing preprocessing and multimodal feature fusion operations on individual analysis chart sample elements in the multimodal data analysis chart sample set, and outputting a feature matrix of the multimodal data analysis chart sample set; S3. Construct an importance value prediction model to predict the importance values ​​of sample elements in the analysis chart, and calculate the importance coefficient of each sample element in the analysis chart based on the importance value prediction results; S4. Construct a communication force prediction model to predict the communication force of the sample elements of the analysis chart, and calculate the communication force coefficient of a single analysis chart sample element; S5. Based on the importance coefficient and communication coefficient of the analysis chart sample elements, calculate the priority coefficient for analysis chart generation and dynamically allocate computing resources to ensure that high-priority analysis charts are generated first; The S3 includes the following specific steps: S31. Label the important value label for each analysis chart sample element in the multimodal data analysis chart sample set to obtain an important value label vector ,in, Sample element for a single analysis chart The important value label value of S32. Constructing an important value prediction model whose input is a feature matrix X of a multimodal data analysis chart sample set M, and whose output is an important value prediction model of an important value of a single analysis chart sample element, wherein the important value prediction model uses the important value corresponding to the single analysis chart sample element as a prediction target, and minimizes the sum of squares of errors between the predicted important value and the important value label value as a training target, and trains the important value prediction model until the sum of squares of errors between the predicted important value and the important value label value reaches convergence, and outputs the important value prediction model to obtain the important value of the single analysis chart sample element; the important value prediction model is any one of a naive Bayes algorithm, a random forest model, a support vector machine, or a neural network model; S33. Calculate the similarity coefficient between the sample elements of the analysis chart. The calculation formula of the similarity coefficient is: ； in, Represents an analysis chart sample element Sample elements for infographics with analytics The similarity coefficient between is the smoothing factor and , Represents an analysis chart sample element Sample elements of infographics with analytics KL divergence of the data distribution between; Sample elements for analysis charts The important value of S34. Calculate the importance coefficient of sample element i of the analysis chart. The calculation formula of the importance coefficient is: ； in, Represents an analysis chart sample element The importance coefficient of The S4 includes the following specific steps: S41. Label a communication force label for a single analysis chart sample element in the multimodal data analysis chart sample set to obtain a communication force label vector ,in, Sample element for a single analysis chart The spreadability label value of S42. Construct a feature matrix X whose input is a set of multimodal data analysis chart samples M, and output a communication force prediction model of the communication force of a single analysis chart sample element. The communication force prediction model uses the communication force corresponding to a single analysis chart sample element as the prediction target, and takes minimizing the sum of the squares of the errors between the predicted communication force and the communication force label value as the training target. The communication force prediction model is trained until the sum of the squares of the errors between the predicted communication force and the communication force label value reaches convergence, and the training is stopped. The communication force prediction model is output to obtain the communication force of a single analysis chart sample element. S43. Calculate the propagation coefficient of a single analysis chart sample element. The calculation formula of the propagation coefficient is: ;in, Represents an analysis chart sample element The transmission coefficient of Analysis chart sample elements The spread of Represents an analysis chart sample element Number of citations, represents the weight of the spread breadth, and , represents the time decay coefficient, and , Sample elements of a display analysis chart The number of views on day t, where T represents the sample element of the analysis chart Total number of days of disclosure; The calculation formula of the priority coefficient in S5 is: ； in, The data update interval, 、 、 are weight factors respectively, and there are .

2. The method for generating multimodal data analysis charts based on artificial intelligence according to claim 1, characterized in that: The S2 includes the following specific steps: S21, extracting a sample set M of multimodal data analysis charts, ;in, Represents the i-th analysis chart sample element; for a single analysis chart sample element , use natural language processing technology to preprocess text data and generate word frequency vectors ; Extract visual features from image data through convolutional neural network and generate visual feature vectors ; Generate spectral feature vectors in audio data through Fourier transform ; Standardize structured data to generate numerical vectors ; S22. Extracting a single analysis chart sample element The word frequency vector , visual feature vector , spectral feature vector and numerical vectors Splice and output a single analysis chart sample element The eigenvector of , ; Output the feature matrix X of the multimodal data analysis chart sample set M, , T represents the transpose of the matrix.

3. The method for generating multimodal data analysis charts based on artificial intelligence according to claim 2, characterized in that: The communication power prediction model in S42 is any one of a deep neural network model and a deep belief network model.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the artificial intelligence-based multimodal data analysis chart generation method as described in any one of claims 1 to 3.

5. An electronic device, characterized in that: It includes a memory for storing instructions; a processor for executing the instructions, so that the device implements the artificial intelligence-based multimodal data analysis chart generation method as described in any one of claims 1 to 3.

Images