Large-scale data processing method and system based on multi-modal data fusion

Abstract

The invention provides a large-scale data processing method and system based on multi-modal data fusion, and relates to the technical field of data process.The method comprises the following steps that multi-modal data are obtained and preprocessed; performing feature extraction and feature fusion on the preprocessed multi-modal data by adopting a mode of combining a deep learning technology and a multi-modal attention mechanism so as to generate a comprehensive feature vector; and performing data analysis and data updating on the comprehensive feature vector by adopting a mode of combining a deep learning technology and an online learning technology to obtain an analysis result of the large-scale data. Data fusion and data analysis are carried out based on deep learning and a multi-modal attention mechanism, deep semantic information can be fully utilized in the feature extraction and fusion process, and the problems that an existing data processing technology lacks a unified fusion mechanism, so that the data processing flow is complex, the data consistency is poor, and the data processing efficiency is high are solved. And real-time analysis is difficult to realize.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a large-scale data processing method and system based on multimodal data fusion. Background Art

[0002] With the rapid development of information technology, the scale and complexity of data in various fields of society have increased dramatically, and the demand for large-scale data processing technology has also increased dramatically. Most of the existing big data processing technologies rely on distributed computing frameworks (such as Hadoop, Spark, etc.), which perform very well in processing traditional structured and semi-structured data. However, for the current large amount of unstructured data (such as text, images, audio, video, etc.) and multimodal data, traditional big data processing technologies are stretched. For example, in the fields of intelligent manufacturing, medical health, social media, etc. Data mainly comes from images, videos, text, voice, etc. Therefore, how to efficiently fuse and process data of different modalities has become an urgent problem to be solved.

[0003] Existing data processing technologies use independent processing modules to process data of different modes separately and fuse the data. There is a lack of a unified fusion mechanism, which leads to complex data processing procedures, poor data consistency, and difficulty in achieving real-time analysis. Summary of the invention

[0004] The present invention provides a large-scale data processing method and system based on multimodal data fusion, which solves the problem that the existing data processing technology lacks a unified fusion mechanism, resulting in complex data processing flow, poor data consistency, and difficulty in achieving real-time analysis.

[0005] In a first aspect, an embodiment of the present invention provides a large-scale data processing method based on multimodal data fusion, the method comprising the following process:

[0006] Acquiring multimodal data by means of an adapter connection configuration, and preprocessing the multimodal data; wherein the multimodal data includes structured data, semi-structured data, unstructured data, and multimedia data;

[0007] The pre-processed multimodal data is extracted and fused by combining deep learning technology and multimodal attention mechanism to generate a comprehensive feature vector.

[0008] The deep learning technology and online learning technology are combined to analyze and update the comprehensive feature vector to obtain the analysis results of large-scale data;

[0009] The analysis results and comprehensive feature vectors are stored asynchronously and in parallel, and displayed interactively and visually.

[0010] In the above embodiments, the present invention performs feature extraction and feature fusion on multimodal data based on deep learning technology and multimodal attention mechanism, and can make full use of deep semantic information in the feature extraction and fusion process, so that the fused comprehensive feature vector is more representative and predictive; and on the basis of feature extraction and feature fusion, deep learning technology and online learning technology are used to perform data analysis and data update on the comprehensive feature vector, which can meet the real-time requirements of data analysis.

[0011] As some optional implementations of the present application, the preprocessing includes data cleaning, data standardization and data format conversion of the multimodal data.

[0012] In the above embodiment, the present invention performs data cleaning, data standardization and data format conversion on the data before feature extraction and feature fusion, can quickly process noise and abnormal data, and convert data of different modes into a unified format, providing guarantee for subsequent real-time analysis.

[0013] As some optional implementations of the present application, the process of extracting and fusing features of pre-processed multimodal data by combining deep learning technology and multimodal attention mechanism to generate a comprehensive feature vector is as follows:

[0014] Convolutional neural networks are used to extract features from preprocessed unstructured data and multimedia data, autoencoders are used to extract features from preprocessed semi-structured data, and recurrent neural networks are used to extract features from preprocessed structured data to obtain multimodal features;

[0015] In the feature extraction process, a multimodal attention mechanism is used to perform feature weighted fusion processing and feature enhancement processing on the features of multimodal data to obtain a comprehensive feature vector.

[0016] In the above embodiment, the present invention introduces a multimodal attention mechanism based on deep learning, which can automatically focus on the key parts of different modal data and weightedly fuse this information. The multimodal attention mechanism can dynamically adjust the weights of different modal features, so that the fused comprehensive feature vector is more representative and predictive.

[0017] As some optional implementation modes of the present application, the data analysis and data update are implemented based on an asynchronous processing architecture.

[0018] In the above embodiment, the asynchronous processing architecture of the present invention performs data analysis and data update, so that data analysis and data update can be performed in parallel, and can provide accurate data analysis results within a millisecond response time, which is suitable for application scenarios that require fast response.

[0019] As some optional implementation methods of the present application, the process of performing data analysis and data updating on the comprehensive feature vector by combining deep learning technology and online learning technology is as follows:

[0020] The deep learning model and online learning model are used to simultaneously analyze the comprehensive feature vector;

[0021] An asynchronous processing architecture is used to synchronize multimodal data preprocessing, feature extraction, data analysis, and data update.

[0022] As some optional implementations of the present application, the process of asynchronously and parallelly storing and visually interactively displaying the analysis results and the comprehensive feature vectors is as follows:

[0023] Use distributed database technology and data lake technology to store analysis results and comprehensive feature vectors in data shards;

[0024] The analysis results are visualized in the form of graphics and charts, and the data scheduling and request tasks of the analysis results and comprehensive feature vectors are supported at the same time.

[0025] In the above embodiments, the present invention adopts distributed database technology to automatically distribute different data to different nodes, and supports efficient storage and access of massive data; it adopts data lake technology to simultaneously store structured data, semi-structured data, unstructured data and multimedia data, and provides a unified storage and management interface, supports unified management of multiple data types, and improves the flexibility of data storage.

[0026] In a second aspect, the present invention provides a large-scale data processing system based on multimodal data fusion, the system comprising:

[0027] A data acquisition unit, the data acquisition unit is used to acquire multimodal data by adopting an adapter connection configuration; wherein the multimodal data includes structured data, semi-structured data, unstructured data and multimedia data;

[0028] A data preprocessing unit, wherein the data preprocessing unit is used to preprocess the multimodal data;

[0029] A feature fusion unit, wherein the feature fusion unit is used to extract and fuse features of the pre-processed multimodal data by combining deep learning technology with a multimodal attention mechanism to generate a comprehensive feature vector;

[0030] A data analysis unit, wherein the data analysis unit is used to perform data analysis and data update on the comprehensive feature vector by combining deep learning technology and online learning technology to obtain analysis results of large-scale data;

[0031] A data storage unit, the data storage unit is used to store the analysis results and the comprehensive feature vector;

[0032] A data interaction unit is used to interactively display the analysis results and the comprehensive feature vectors in a visual manner.

[0033] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the large-scale data processing method based on multimodal data fusion when executing the computer program.

[0034] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the large-scale data processing method based on multimodal data fusion is implemented.

[0035] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the large-scale data processing method based on multimodal data fusion.

[0036] The beneficial effects of the present invention are as follows:

[0037] 1. The present invention performs feature extraction and feature fusion on multimodal data based on deep learning technology and multimodal attention mechanism, which can make full use of deep semantic information in the feature extraction and fusion process, making the fused comprehensive feature vector more representative and predictive.

[0038] 2. The present invention performs data cleaning, data standardization and data format conversion on the data before feature extraction and feature fusion, can quickly process noise and abnormal data, and convert data of different modes into a unified format, providing guarantee for subsequent real-time analysis.

[0039] 3. The present invention adopts distributed database technology to automatically distribute different data to different nodes, and supports efficient storage and access of massive data; it adopts data lake technology to simultaneously store structured data, semi-structured data, unstructured data and multimedia data, and provides a unified storage and management interface, supports unified management of multiple data types, and improves the flexibility of data storage.

[0040] 4. The present invention adopts an asynchronous processing architecture to synchronize multimodal data preprocessing, feature extraction, data analysis and data update. The asynchronous processing architecture enables multiple data processing processes to be performed in parallel, reducing overall delays and improving the real-time performance of data processing.

[0041] 5. The present invention directly performs data enhancement during the feature extraction process, which not only improves the feature representation capability, but also improves the computational efficiency of data analysis, and is suitable for large-scale data processing environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 is a schematic diagram of the computer device structure of the hardware operating environment described in the embodiment of the present invention;

[0044] Figure 2 is a flow chart of a large-scale data processing method according to an embodiment of the present invention;

[0045] Figure 3 It is a structural block diagram of the large-scale data processing system described in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] In order to solve the problem that the existing data processing technology lacks a unified fusion mechanism, resulting in complex data processing procedures, poor data consistency, and difficulty in achieving real-time analysis. The embodiment of the present invention provides a large-scale data processing method and system based on multimodal data fusion. Before introducing the specific technical solution of the present application, the hardware operating environment involved in the embodiment of the present application is first introduced.

[0048] See also Figure 1 , Figure 1 The figure is a schematic diagram of the computer device structure of the hardware operating environment involved in the embodiment of the present invention.

[0049] like Figure 1As shown, the computer device may include: a processor, such as a central processing unit (CPU), a communication bus, a user interface, a network interface, and a memory. Among them, the communication bus is used to realize the connection and communication between these components. The user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard), and optionally, the user interface may also include a standard wired interface and a wireless interface. The network interface may be a wired interface or a wireless interface (such as a wireless fidelity interface). The memory may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory may optionally be a storage device independent of the aforementioned processor.

[0050] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the computer device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0051] like Figure 1 As shown, the memory as a storage medium may include an operating system module, a network communication module, a user interface module and a data storage module.

[0052] exist Figure 1 In the computer device shown, the network interface is mainly used for data communication with the network server; the user interface is mainly used for data interaction with the user; the processor and memory in the computer device of the present application can be set in the computer device, and the computer device calls the computer program product stored in the data storage module through the processor, and executes the large-scale data processing method based on multimodal data fusion provided in the embodiment of the present application.

[0053] Based on the hardware environment of the aforementioned embodiment, the embodiment of the present application provides a large-scale data processing method based on multimodal data fusion, please refer to Figure 2 , Figure 2 The flowchart of the large-scale data processing method is as follows:

[0054] (1) Using adapter connection configuration to acquire multimodal data in real time; wherein the multimodal data includes structured data, semi-structured data, unstructured data and multimedia data. For example, connecting sensors, databases, etc. through JDBC / IOT gateways, connecting social media and monitoring equipment through API interfaces, etc., receiving data streams or batch data files in a standardized manner, and assigning timestamps to the data streams or batch data files.

[0055] In an embodiment of the present invention, after acquiring the multimodal data, it is necessary to perform data analysis and data filtering on the multimodal data, and perform data verification on the multimodal data after data analysis and data filtering to ensure the integrity and validity of the data.

[0056] (2) Preprocessing the multimodal data, wherein the preprocessing includes data cleaning, data standardization, and data format conversion of the multimodal data.

[0057] In an embodiment of the present invention, the multimodal data is first cleaned to process the noise and redundant information in the data to ensure the integrity and validity of the data again; then the multimodal data is standardized according to the data type, which can effectively shorten the later data fusion time and provide guarantee for subsequent real-time analysis, such as converting text data into word vectors and converting image data into grayscale images; finally, the multimodal data is converted into a unified intermediate format, such as a vector or tensor, to facilitate subsequent feature extraction and feature fusion.

[0058] (3) A combination of deep learning technology and multimodal attention mechanism is used to extract and fuse features of the preprocessed multimodal data to generate a comprehensive feature vector.

[0059] In an embodiment of the present invention, a process of extracting and fusing features of pre-processed multimodal data by combining deep learning technology with a multimodal attention mechanism to generate a comprehensive feature vector is as follows:

[0060] (3.1) A convolutional neural network (CNN) is used to extract features from pre-processed unstructured data (such as image data) and multimedia data (such as audio data and video data). An autoencoder (Transformer network) is used to extract features from pre-processed semi-structured data (such as log files). A recurrent neural network (RNN) is used to extract features from pre-processed structured data. That is, deep learning technology can be used to capture the fine-grained information of each modal data at the feature level to obtain multi-modal features.

[0061] (3.2) In the feature extraction process, a multimodal attention mechanism or a graph convolutional network (GCN) is used to perform feature weighted fusion processing and feature enhancement processing on the features of the multimodal data to obtain a comprehensive feature vector, wherein the comprehensive feature vector is represented in the form of four elements:

[0062] Vector=[Element1,Element2,Element3,Element4]

[0063] Among them, Vector represents a comprehensive feature vector, Element1 represents the characteristics of structured data after weighted processing, Element2 represents the characteristics of semi-structured data after weighted processing, Element3 represents the characteristics of unstructured data after weighted processing, and Element4 represents the characteristics of multimedia data after weighted processing.

[0064] Specifically, the comprehensive feature vector is assigned a timestamp in a manner of combining a specific time period, an adjacent time period and a specific moment within a specific time period; and a priority is assigned to the specific time period, the adjacent time period and the specific moment within the specific time period, that is, the priority of the specific time period is higher than the priority of the specific moment within the specific time period, and the priority of the specific moment within the specific time period is higher than the priority of the adjacent time period.

[0065] In the embodiment of the present invention, in the process of fusion processing of multimodal data features, a multimodal attention mechanism is introduced, which can automatically focus on the key parts of different modal data and weightedly fuse this information. This mechanism can dynamically adjust the weights of different modal features, making the fused comprehensive feature vector more representative and predictive.

[0066] Specifically, data enhancement, such as feature selection and feature dimensionality reduction, is performed directly during feature extraction, which not only improves the representation capability of features but also improves the efficiency of data analysis and calculation, and is particularly suitable for large-scale data environments.

[0067] (4) Using a combination of deep learning technology and online learning technology to perform data analysis and data update on the comprehensive feature vector to obtain analysis results of large-scale data; wherein the data analysis and data update are implemented based on an asynchronous processing architecture.

[0068] In the embodiment of the present invention, the process of performing data analysis and data updating on the comprehensive feature vector by combining deep learning technology and online learning technology is as follows:

[0069] (4.1) Use lightweight deep learning models and online learning models to simultaneously analyze the comprehensive feature vectors. For example, use lightweight time series prediction models and online random forest models to simultaneously analyze the comprehensive feature vectors, which can instantly update and generate analysis results when new data arrives, meeting real-time analysis needs.

[0070] Specifically, in the process of data analysis of the comprehensive feature vector, the comprehensive feature vector containing information of the same time period is first analyzed. Since the comprehensive feature vector corresponds to the diversity of features, that is, there are features of time series data and time period data, etc., directly using the features of time series data will result in feature missing. Therefore, a lightweight deep learning model and an online learning model are used to analyze the comprehensive feature vector of the feature composition of the same time period, and based on the comprehensive feature vector of the feature combination of the same time period, an auxiliary analysis is performed on the comprehensive feature vector of the feature composition of a specific moment, which can more quickly and accurately obtain the analysis results of large-scale data; in this process, if the feature composition of a certain time period is missing, the features of the adjacent time period are used to supplement it.

[0071] (4.2) An asynchronous processing architecture is used to synchronize multimodal data preprocessing, feature extraction, data analysis, and data update. The asynchronous processing architecture allows multiple data processing processes to be performed in parallel, reducing overall latency. At the same time, a memory cache mechanism is used to further improve real-time performance.

[0072] (5) Asynchronous parallel storage and interactive visualization of analysis results and comprehensive feature vectors.

[0073] In the embodiment of the present invention, the process of asynchronously and parallelly storing and visually interactively displaying the analysis results and the comprehensive feature vectors is as follows:

[0074] (5.1) Distributed database technology and data lake technology are used to store analysis results and comprehensive feature vectors in data shards. Distributed database technology can automatically distribute different data to different nodes and support efficient storage and access of massive data; data lake technology can simultaneously store structured data, semi-structured data, unstructured data, and multimedia data, and provide a unified storage and management interface, support unified management of multiple data types, and improve the flexibility of data storage. At the same time, by building an index and cache mechanism, efficient data retrieval capabilities can be achieved to meet the needs of on-demand query and analysis of stored data and improve data utilization efficiency.

[0075] (5.2) The analysis results are visualized in 2D / 3D in the form of graphics and charts, and the data scheduling and request tasks of the analysis results and comprehensive feature vectors are supported at the same time.

[0076] Specifically, the analysis results are visualized in the form of charts and graphs, and interactive charts are supported, such as clicking to view detailed data, zooming in and out of charts, etc., which improves the readability and experience of the data; and structured report documents are generated based on the analysis results, including charts, data tables, text descriptions, etc., which are convenient for downloading or printing reports for subsequent analysis or decision-making; at the same time, security permission configuration and user feedback acquisition are also supported.

[0077] In summary, the present invention performs feature extraction and feature fusion on multimodal data based on deep learning technology and multimodal attention mechanism, and can make full use of deep semantic information in the feature extraction and fusion process, so that the fused comprehensive feature vector is more representative and predictive; and on the basis of feature extraction and feature fusion, deep learning technology and online learning technology are used to perform data analysis and data update on the comprehensive feature vector, which solves the problem that the existing data processing technology lacks a unified fusion mechanism, resulting in complex data processing flow, poor data consistency, and difficulty in achieving real-time analysis.

[0078] In addition, in one embodiment, based on the same inventive concept as the above embodiment, the embodiment of the present invention provides a large-scale data processing system based on multimodal data fusion, and the system corresponds to the method of embodiment 1 one by one, please refer to Figure 3 , Figure 3 is a structural block diagram of the large-scale data processing system, the system comprising:

[0079] A data acquisition unit, the data acquisition unit is used to acquire multimodal data by adopting an adapter connection configuration; wherein the multimodal data includes structured data, semi-structured data, unstructured data and multimedia data;

[0080] A data preprocessing unit, wherein the data preprocessing unit is used to preprocess the multimodal data;

[0081] A feature fusion unit, wherein the feature fusion unit is used to extract and fuse features of the pre-processed multimodal data by combining deep learning technology with a multimodal attention mechanism to generate a comprehensive feature vector;

[0082] A data analysis unit, wherein the data analysis unit is used to perform data analysis and data update on the comprehensive feature vector by combining deep learning technology and online learning technology to obtain analysis results of large-scale data;

[0083] A data storage unit, the data storage unit is used to store the analysis results and the comprehensive feature vector;

[0084] A data interaction unit is used to interactively display the analysis results and the comprehensive feature vectors in a visual manner.

[0085] It should be noted that each unit in the large-scale data processing system based on multimodal data fusion in this embodiment corresponds one-to-one to each step in the large-scale data processing method based on multimodal data fusion in the aforementioned embodiment. Therefore, the specific implementation method and technical effects achieved by this embodiment can refer to the implementation method of the aforementioned large-scale data processing method based on multimodal data fusion, which will not be repeated here.

[0086] In addition, in one embodiment, the present application also provides a computer device, which includes a processor, a memory, and a computer program stored in the memory, and the computer program implements the method in the aforementioned embodiment when executed by the processor.

[0087] In addition, in one embodiment, the present application further provides a computer storage medium, on which a computer program is stored, and the computer program implements the method in the aforementioned embodiment when executed by a processor.

[0088] In some embodiments, the computer readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. The computer may be various computing devices including intelligent terminals and servers.

[0089] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0090] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0091] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0092] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0093] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0094] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a multimedia terminal device (which can be a mobile phone, a computer, a television receiver, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0095] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A large-scale data processing method based on multimodal data fusion, characterized in that: The method comprises the following steps: Acquiring multimodal data by means of an adapter connection configuration, and preprocessing the multimodal data; wherein the multimodal data includes structured data, semi-structured data, unstructured data, and multimedia data; The pre-processed multimodal data is extracted and fused by combining deep learning technology and multimodal attention mechanism to generate a comprehensive feature vector. The deep learning technology and online learning technology are combined to analyze and update the comprehensive feature vector to obtain the analysis results of large-scale data; The analysis results and comprehensive feature vectors are stored asynchronously and in parallel, and displayed interactively and visually.

2. The large-scale data processing method based on multimodal data fusion according to claim 1 is characterized in that: The preprocessing includes data cleaning, data standardization and data format conversion of multimodal data.

3. The large-scale data processing method based on multimodal data fusion according to claim 1 is characterized in that: The process of extracting and fusing features from preprocessed multimodal data by combining deep learning technology and multimodal attention mechanism to generate a comprehensive feature vector is as follows: Convolutional neural networks are used to extract features from preprocessed unstructured data and multimedia data, autoencoders are used to extract features from preprocessed semi-structured data, and recurrent neural networks are used to extract features from preprocessed structured data to obtain multimodal features; In the feature extraction process, a multimodal attention mechanism is used to perform feature weighted fusion processing and feature enhancement processing on the features of multimodal data to obtain a comprehensive feature vector.

4. The large-scale data processing method based on multimodal data fusion according to claim 1 is characterized in that: The data analysis and data update are implemented based on an asynchronous processing framework.

5. The large-scale data processing method based on multimodal data fusion according to claim 1 is characterized in that: The process of combining deep learning technology and online learning technology to analyze and update the comprehensive feature vector is as follows: The deep learning model and online learning model are used to simultaneously analyze the comprehensive feature vector; An asynchronous processing architecture is used to synchronize multimodal data preprocessing, feature extraction, data analysis, and data update.

6. The large-scale data processing method based on multimodal data fusion according to claim 1 is characterized in that: The process of asynchronous parallel storage and interactive visualization of analysis results and comprehensive feature vectors is as follows: Use distributed database technology and data lake technology to store analysis results and comprehensive feature vectors in data shards; The analysis results are visualized in the form of graphics and charts, and the data scheduling and request tasks of the analysis results and comprehensive feature vectors are supported at the same time.

7. A large-scale data processing system based on multimodal data fusion, characterized in that: The system comprises: A data acquisition unit, the data acquisition unit is used to acquire multimodal data by adopting an adapter connection configuration; wherein the multimodal data includes structured data, semi-structured data, unstructured data and multimedia data; A data preprocessing unit, wherein the data preprocessing unit is used to preprocess the multimodal data; A feature fusion unit, wherein the feature fusion unit is used to extract and fuse features of the pre-processed multimodal data by combining deep learning technology with a multimodal attention mechanism to generate a comprehensive feature vector; A data analysis unit, wherein the data analysis unit is used to perform data analysis and data update on the comprehensive feature vector by combining deep learning technology and online learning technology to obtain analysis results of large-scale data; A data storage unit, the data storage unit is used to store the analysis results and the comprehensive feature vector; A data interaction unit is used to interactively display the analysis results and the comprehensive feature vectors in a visual manner.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the large-scale data processing method based on multimodal data fusion as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements a large-scale data processing method based on multimodal data fusion as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements a large-scale data processing method based on multimodal data fusion as described in any one of claims 1 to 6.

Images