A data management method and system for multimodal database

Abstract

The present invention discloses a data management method and system for a multimodal database, and relates to the technical field of data management. A data management system for a multimodal database includes: a multimodal data analysis module, a multimodal data storage module, and a multimodal data update module. The present invention improves the accuracy and depth of data recognition by extracting deep-level features using a multimodal data feature analysis model; optimizes the data storage structure and improves query efficiency by performing database storage allocation using feature matching technology; introduces a feature adaptive threshold factor Z and a CNN model improvement based on a knowledge graph to enhance the generalization ability and adaptability of the model; and realizes dynamic updates of database partitions through a multimodal database management update model, responds to changes in user operations, maintains the timeliness and relevance of the database, and enhances user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and in particular to a data management method and system for a multimodal database. Background Art

[0002] The data management technology of multimodal databases stems from the increase in multiple data types and the improvement in complexity. These data types include text, images, video, audio, and various sensor data. Especially when processing complex, multi-source, and heterogeneous data, the traditional single data model database has been difficult to meet the needs of modern applications due to its limitations. General data management refers to a series of strategies and technologies for effective data collection, storage, security management, verification, and processing. However, with the development of technology, ordinary data management methods cannot meet the needs of most users. It is necessary to conduct in-depth analysis of the data, explore internal features, provide intelligent and personalized management modes for diversified multimodal data, adjust and optimize the internal storage structure, and optimize the data storage strategy according to the characteristics of multimodal data. Summary of the invention

[0003] The present invention aims to provide a data management method and system for a multimodal database, which optimizes the data storage strategy according to the characteristics of multimodal data.

[0004] A data management method for a multimodal database comprises the following steps:

[0005] Get the multimodal data set to be managed, which contains N multimodal data D to be managed. n , multimodal data to be managed D n is text data or image data; based on the multimodal data set to be managed and the multimodal data feature analysis model, the fusion feature R of the multimodal data to be managed is obtained n , n=1,2,…,N;

[0006] The multimodal data feature analysis model includes a data preprocessing layer, a feature extraction layer, a comprehensive analysis layer, and a fusion feature output layer. The basic CNN model is improved based on the knowledge graph algorithm, and a feature adaptive threshold factor Z is introduced to extract the multimodal data D to be managed. n The deep-level characteristics of

[0007] Based on the fusion feature R of multimodal data to be managed n and multimodal data management allocation model, treat the management of multimodal data D nPerform database storage allocation and store the multimodal data set to be managed in the multimodal database; wherein the multimodal database contains M multimodal database partitions Pm, m=1, 2, ..., M; each multimodal database partition Pm corresponds to a database access implicit value Y m ;

[0008] The multimodal data management allocation model includes a feature matching layer, a partition storage layer, and a result output layer. The feature matching technology is used to manage multimodal data. n Perform database storage allocation;

[0009] Obtain the target user's operation records; based on the target user's operation records and the multimodal database management update model, partition the multimodal database P m Perform dynamic updates to obtain a new multimodal database partition P m , and assign a new multimodal database partition P m New database access implicit value Y m ;

[0010] The multimodal database management update model includes an operation feature quantification layer, an implicit value change layer and a database storage update layer, and is optimized based on a heuristic improved swarm optimization algorithm.

[0011] As a preferred technical solution of the present invention, the multimodal data feature analysis model includes a data preprocessing layer, a feature extraction layer, a comprehensive analysis layer and a fusion feature output layer;

[0012] The data preprocessing layer is used to manage multimodal data D n Perform preprocessing to obtain preprocessed multimodal data D to be managed n ';

[0013] The feature extraction layer is used to use the improved knowledge graph algorithm and CNN model to preprocess the multimodal data D n 'Perform feature extraction to obtain the multimodal data features T to be managed n ;

[0014] The comprehensive analysis layer is used to integrate all pre-processed multimodal data D n 'Perform fusion feature analysis to obtain multimodal data set features;

[0015] The fusion feature output layer is used to transform the multimodal data features T to be managed n Perform feature fusion with the multimodal data set features to obtain the fusion feature R of the multimodal data to be managed n .

[0016] As a preferred technical solution of the present invention, the specific steps of performing feature extraction in the feature extraction layer include:

[0017] The feature extraction layer consists of I convolutional layers, i=1, 2, ..., I; the feature adaptive threshold factor Z is set, , σ is the multimodal data convolution feature J 3i-2 The standard deviation of represents the fluctuation range of the feature; μ is the convolution feature J of multimodal data 3i-2 The mean of the features, indicating the average level of the features; H(F) is the convolution feature J of the multimodal data 3i-2 The entropy of represents the complexity of the feature; α, β and ϵ are adjustment coefficients;

[0018] In the 3i-2th convolutional layer, the preprocessed multimodal data D n ' or multimodal data convolution feature J 3i Perform feature recognition to obtain multimodal data convolution feature J 3i-2 ; Calculate the current feature adaptive threshold factor Z; when the feature adaptive threshold factor Z is greater than the preset feature adjustment threshold, reduce the pooling strength of the 3i-1th convolution layer; otherwise, remain unchanged;

[0019] In the 3i-1th convolutional layer, the multimodal data convolution feature J 3i-2 Perform pooling operation to obtain multimodal data convolution feature J 3i-1 ;

[0020] In the 3i convolutional layer, the multimodal data convolution feature J 3i-1 Perform entity recognition to obtain multimodal data convolution feature J 3i ;

[0021] Repeat the above operation, and in the I convolution layer, obtain the multimodal data features T to be managed n .

[0022] As a preferred technical solution of the present invention, the multimodal data management allocation model includes a feature matching layer, a partition storage layer and a result output layer;

[0023] The feature matching layer is used to manage multimodal data fusion features R n Perform feature matching to obtain the database access matching prediction result G n ;

[0024] The partition storage layer is used to access the matching prediction results G based on the database n Matches the closest database access implicit value Y m , get the database management allocation result Q n ;

[0025] The result output layer is used to manage the distribution results Q based on all databases n The multimodal data to be managed Dn Stored to the corresponding multimodal database partition P m middle.

[0026] As a preferred technical solution of the present invention, the specific steps of training the feature matching layer include:

[0027] Collect several groups of data feature matching samples with label values; each group of data feature matching samples contains data fusion features and label values ​​corresponding to verified data, and the label value represents the data access label; several groups of data feature matching samples with label values ​​are combined to obtain a data feature matching training set;

[0028] The data feature matching training set is input into the multimodal data management allocation model to train the feature matching layer using the label value as the target to obtain the initial feature matching layer; the initial feature matching layer is model evaluated to obtain the initial feature matching layer model evaluation result; if the initial feature matching layer model evaluation result is passed, the initial feature matching layer is used as the feature matching layer in the multimodal data management allocation model; otherwise, the model training is continued using the data feature matching training set.

[0029] As a preferred technical solution of the present invention, the multimodal database management update model includes an operation feature quantification layer, an implicit value change layer and a database storage update layer;

[0030] The operation feature quantization layer is used to quantify the target user's operation records and obtain the target user's operation feature quantization value L m ;

[0031] The implicit value change layer is used to quantify the value L according to the target user's operation characteristics. m Analyze with the database implicit value change function to get the new database access implicit value Y m ;The database implicit value change function is determined by the heuristic improved group optimization algorithm;

[0032] The database storage update layer is used to access the implicit value Y based on the new database m Redistribute multimodal database partitions P m The multimodal data in the new multimodal database partition P m .

[0033] As a preferred technical solution of the present invention, the specific steps of determining the database implicit value change function include:

[0034] Construct 2K simulated implicit value change function individuals A k , k=1, 2, ..., 2K; each simulated implicit value change function individual A kContains the solution to the implicit value change function of the database; the first K simulated implicit value change function individuals A k As the population for exploring the implicit value change function, the next K simulated implicit value change function individuals A k As the development implicit value change function population; set the number of iterations b, b=1, 2, ..., B; record the value with the largest fitness in the development implicit value change function population as the sentinel implicit value change function individual;

[0035] The specific steps for calculating fitness include:

[0036] Collecting several groups of verified initial feature quantization values ​​and corresponding database allocation results as a group of database implicit value change training samples;

[0037] Change function individual A according to the simulated implicit value k Perform simulation calculation on the database implicit value change training samples to obtain the simulated change implicit value, and perform database partition matching according to the simulated feature quantization value to obtain the simulated database partition result; calculate the accuracy of the simulated database partition result and the corresponding database allocation result to obtain the simulated calculation accuracy; use all the simulated calculation accuracy as the simulated implicit value change function individual A k The fitness S k ;

[0038] When b%2=0, the exploration implicit value change function population is iterated to obtain an updated exploration implicit value change function population; at the same time, the development implicit value change function population is iterated to obtain an updated development implicit value change function population, and a new sentinel implicit value change function individual is selected;

[0039] When b%2≠0, the exploration implicit value change function population does not perform population iteration and no operation is performed; the development implicit value change function population performs population iteration around the sentinel implicit value change function individuals to obtain an updated development implicit value change function population;

[0040] When b=B, the simulated implicit value change function individual with the largest fitness in the exploration implicit value change function population and the development implicit value change function population is output, which is the optimal simulated implicit value change function individual; the solution of the database implicit value change function in the optimal simulated implicit value change function individual is used as the database implicit value change function.

[0041] A data management system for a multimodal database, comprising:

[0042] The multimodal data analysis module includes a data acquisition unit and a data analysis unit; the data acquisition unit is used to acquire a multimodal data set to be managed, and the multimodal data set to be managed includes N multimodal data D to be managed. n , multimodal data to be managed Dn is text data or image data; the data analysis unit is used to analyze based on the multimodal data set to be managed and the multimodal data feature analysis model to obtain the fusion feature R of the multimodal data to be managed n , n=1,2,…,N;

[0043] The multimodal data storage module includes a data storage unit; the data storage unit is used to fuse the feature R of the multimodal data to be managed. n and multimodal data management allocation model, treat the management of multimodal data D n Perform database storage allocation and store the multimodal data set to be managed in the multimodal database; wherein the multimodal database contains M multimodal database partitions P m , m=1,2,…,M; each multimodal database partition P m Corresponding to a database access implicit value Y m ;

[0044] The multimodal data update module includes a data storage update unit; the data storage update unit is used to obtain the target user operation record; based on the target user operation record and the multimodal database management update model, the multimodal database partition P m Perform dynamic updates to obtain a new multimodal database partition P m , and assign a new multimodal database partition P m New database access implicit value Y m .

[0045] The present invention has the following advantages:

[0046] 1. The present invention improves the accuracy and depth of data recognition by extracting deep-level features using a multimodal data feature analysis model; optimizes the data storage structure and improves query efficiency by performing database storage allocation through feature matching technology; introduces a feature adaptive threshold factor Z and a CNN model improvement based on a knowledge graph to enhance the generalization ability and adaptability of the model; and implements a dynamic update of database partitions through a multimodal database management update model, responds to changes in user operations, maintains the timeliness and relevance of the database, and jointly improves the intelligent level of multimodal data processing, reduces management costs, and enhances user experience.

[0047] 2. By fusing multimodal information such as text data and image data, the present invention can more comprehensively mine the potential features of the data to be managed and improve the richness and accuracy of data representation; the knowledge graph algorithm is introduced to improve the CNN model, so that the deep relationship between data can be better understood and captured during the feature extraction process, especially through the adaptive adjustment of feature extraction by the adaptive threshold factor Z, which effectively improves the depth and accuracy of feature mining; through the multimodal data management allocation model, it can intelligently allocate database storage to the multimodal data to be managed according to its fused features. This method optimizes the storage process, so that data can be efficiently allocated to appropriate database partitions, avoiding data storage conflicts and redundancy, and improving database access efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of the structure of a data management system for a multimodal database used in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to enable persons skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0050] Embodiment 1, a data management method for a multimodal database, comprises the following steps:

[0051] Get the multimodal data set to be managed, which contains N multimodal data D to be managed. n , multimodal data to be managed D n is text data or image data; based on the multimodal data set to be managed and the multimodal data feature analysis model, the fusion feature R of the multimodal data to be managed is obtained n , n=1,2,…,N;

[0052] The multimodal data feature analysis model includes a data preprocessing layer, a feature extraction layer, a comprehensive analysis layer, and a fusion feature output layer. The basic CNN model is improved based on the knowledge graph algorithm, and a feature adaptive threshold factor Z is introduced to extract the multimodal data D to be managed. n The deep-level characteristics of

[0053] The data preprocessing layer is used to manage multimodal data D n Perform preprocessing to obtain preprocessed multimodal data D to be managed n ';The data preprocessing layer can normalize and clean multimodal data, remove interference factors such as noise and outliers, provide higher quality input for subsequent feature extraction, and improve the overall performance of the model;

[0054] The feature extraction layer is used to use the improved knowledge graph algorithm and CNN model to preprocess the multimodal data D n 'Perform feature extraction to obtain the multimodal data features T to be managed n ; By combining the knowledge graph algorithm, it is possible to mine and utilize the semantic associations in multimodal data, improve the accuracy and adaptability of feature extraction, and effectively improve the model's interpretability and reasoning capabilities, especially when facing complex data;

[0055] The comprehensive analysis layer is used to integrate all pre-processed multimodal data D n 'Perform fusion feature analysis to obtain multimodal data set features;

[0056] The fusion feature output layer is used to transform the multimodal data features T to be managed n Perform feature fusion with the multimodal data set features to obtain the fusion feature R of the multimodal data to be managed n ; Through the comprehensive analysis layer and the fusion feature output layer, the data features from different modalities can be effectively fused to provide more accurate multimodal data set features, which helps to better handle the correlation and complementarity between different modalities;

[0057] By integrating multimodal information such as text data and image data, the potential features of the data to be managed can be more comprehensively mined, and the richness and accuracy of data representation can be improved. The knowledge graph algorithm is introduced to improve the CNN model, so that the deep relationship between data can be better understood and captured during the feature extraction process. In particular, the adaptive adjustment of feature extraction through the adaptive threshold factor Z effectively improves the depth and accuracy of feature mining.

[0058] The specific steps of feature extraction in the feature extraction layer include:

[0059] The feature extraction layer consists of I convolutional layers, i=1, 2, ..., I; the feature adaptive threshold factor Z is set, , σ is the multimodal data convolution feature J 3i-2 The standard deviation of represents the fluctuation range of the feature; μ is the convolution feature J of multimodal data 3i-2 The mean of the features, indicating the average level of the features; H(F) is the convolution feature J of the multimodal data 3i-2 The entropy of represents the complexity of the feature; α, β and ϵ are adjustment coefficients;

[0060] In the 3i-2th convolutional layer, the preprocessed multimodal data D n ' or multimodal data convolution feature J 3i Perform feature recognition to obtain multimodal data convolution feature J 3i-2; Calculate the current feature adaptive threshold factor Z; when the feature adaptive threshold factor Z is greater than the preset feature adjustment threshold, reduce the pooling strength of the 3i-1th convolution layer; otherwise, remain unchanged;

[0061] In the 3i-1th convolutional layer, the multimodal data convolution feature J 3i-2 Perform pooling operation to obtain multimodal data convolution feature J 3i-1 ;

[0062] In the 3i convolutional layer, the multimodal data convolution feature J 3i-1 Perform entity recognition to obtain multimodal data convolution feature J 3i ;

[0063] Repeat the above operation, and in the I convolution layer, obtain the multimodal data features T to be managed n ;

[0064] The feature extraction layer dynamically adjusts the strength of feature extraction according to the standard deviation (σ), mean (μ) and entropy (H(F)) of the convolution feature by introducing a feature adaptive threshold factor Z. This adaptive adjustment mechanism makes the feature extraction process more flexible and effectively copes with changes in different data distributions and complexities, thereby improving the accuracy and reliability of feature extraction. In the feature extraction process, the pooling strength is dynamically adjusted according to the Z value and the preset threshold to avoid over-pooling or under-pooling, which helps to retain the key features in multimodal data and prevent the loss of important information, thereby enhancing the performance of the model. The ability to express; each convolutional layer performs feature recognition and entity recognition, so that feature extraction can be gradually refined at multiple levels, and gradually capture and refine features from primary to deep. In this way, feature extraction can not only capture local features, but also understand deeper complex patterns and improve the accuracy of feature expression; through multi-level convolution feature extraction and entity recognition operations, the information in multimodal data can be fused and enhanced at multiple levels. The feature extraction of each layer will be further refined based on the features of the previous layer, ensuring that important information in different modal data is fully utilized and avoiding information loss or confusion;

[0065] Based on the fusion feature R of multimodal data to be managed n and multimodal data management allocation model, treat the management of multimodal data D n Perform database storage allocation and store the multimodal data set to be managed in the multimodal database; wherein the multimodal database contains M multimodal database partitions P m , m=1,2,…,M; each multimodal database partition P m Corresponding to a database access implicit value Y m ;

[0066] The multimodal data management allocation model includes a feature matching layer, a partition storage layer, and a result output layer. The feature matching technology is used to manage multimodal data. n Perform database storage allocation;

[0067] The feature matching layer is used to manage multimodal data fusion features R n Perform feature matching to obtain the database access matching prediction result G n ;

[0068] The partition storage layer is used to access the matching prediction results G based on the database n Matches the closest database access implicit value Y m , get the database management allocation result Q n ;

[0069] The result output layer is used to manage the distribution results Q based on all databases n The multimodal data to be managed D n Stored to the corresponding multimodal database partition P m middle;

[0070] Through the multimodal data management allocation model, it is possible to intelligently allocate database storage to the multimodal data to be managed based on its fusion features. This method optimizes the storage process, enables data to be efficiently allocated to appropriate database partitions, avoids conflicts and redundancies in data storage, and improves the access efficiency of the database; the feature matching layer generates database access matching prediction results by matching the fusion features of multimodal data, ensuring that the multimodal data to be managed can be stored in the most suitable database partition. Such a precise matching mechanism reduces mismatches in the storage process and improves the efficiency of data storage and retrieval; the partition storage layer automatically selects the closest database access implicit value based on the database access matching prediction results, realizing dynamic adjustment of data storage. This adaptive feature enables the model to flexibly adapt to the ever-changing multimodal data and optimize the storage strategy based on the characteristics of the data;

[0071] The specific steps for training the feature matching layer include:

[0072] Collect several groups of data feature matching samples with label values; each group of data feature matching samples contains data fusion features and label values ​​corresponding to verified data, and the label value represents the data access label; several groups of data feature matching samples with label values ​​are combined to obtain a data feature matching training set;

[0073] Input the data feature matching training set into the multimodal data management allocation model to train the feature matching layer using the label value as the target to obtain an initial feature matching layer; perform model evaluation on the initial feature matching layer to obtain an initial feature matching layer model evaluation result; if the initial feature matching layer model evaluation result is passed, use the initial feature matching layer as the feature matching layer in the multimodal data management allocation model; otherwise, continue model training using the data feature matching training set;

[0074] Through training and evaluation, we ensure that the feature matching layer can accurately identify and match data features and improve the accuracy of data management. Through training with a large number of samples, the model has better generalization ability and can adapt to different data environments and conditions. Through continuous model iteration and optimization, we reduce the feature matching error rate and reduce data loss or processing errors caused by incorrect matching.

[0075] Obtain the target user's operation records; based on the target user's operation records and the multimodal database management update model, partition the multimodal database P m Perform dynamic updates to obtain a new multimodal database partition P m , and assign a new multimodal database partition P m New database access implicit value Y m ;

[0076] The multimodal database management update model includes an operation feature quantification layer, an implicit value change layer, and a database storage update layer, and is optimized based on a heuristic improved group optimization algorithm.

[0077] The operation feature quantization layer is used to quantify the target user's operation records and obtain the target user's operation feature quantization value L m ;

[0078] The implicit value change layer is used to quantify the value L according to the target user's operation characteristics. m Analyze with the database implicit value change function to get the new database access implicit value Y m ;The database implicit value change function is determined by the heuristic improved group optimization algorithm;

[0079] The database storage update layer is used to access the implicit value Y based on the new database m Redistribute multimodal database partitions P m The multimodal data in the new multimodal database partition P m ;

[0080] By quantifying the user's operation characteristics, the user's behavior patterns and preferences can be accurately captured to ensure that the access to the database partition is more targeted and efficient, which means that in the updated database partition, the data storage and access path can better meet the user's needs, thereby improving access efficiency; through the implicit value change layer, the model can dynamically adjust the database access implicit value to reflect the target user's latest behavior pattern, making database access more intelligent and personalized, reducing redundant data and irrelevant access requests, and thus improving the performance of the system; by intelligently partitioning and adjusting the storage of the database based on user behavior and access patterns, the database can store and manage multimodal data more efficiently according to demand, thereby improving the overall efficiency of data access; as more user data is continuously input, the update model can automatically adjust the strategy to adapt to new needs and environments, avoiding the tediousness of manual intervention, and can maintain good adaptive capabilities in long-term use;

[0081] The specific steps for determining the database implicit value change function include:

[0082] Construct 2K simulated implicit value change function individuals A k , k=1, 2, ..., 2K; each simulated implicit value change function individual A k Contains the solution to the implicit value change function of the database; the first K simulated implicit value change function individuals A k As the population for exploring the implicit value change function, the next K simulated implicit value change function individuals A k As the population of the development implicit value change function; set the number of iterations b, b=1, 2, ..., B; record the value with the largest fitness in the population of the development implicit value change function as the sentinel implicit value change function individual;

[0083] The specific steps for calculating fitness include:

[0084] Collecting several groups of verified initial feature quantization values ​​and corresponding database allocation results as a group of database implicit value change training samples;

[0085] Change function individual A according to the simulated implicit value k Perform simulation calculation on the database implicit value change training samples to obtain the simulated change implicit value, and perform database partition matching according to the simulated feature quantization value to obtain the simulated database partition result; calculate the accuracy of the simulated database partition result and the corresponding database allocation result to obtain the simulated calculation accuracy; use all the simulated calculation accuracy as the simulated implicit value change function individual A k The fitness S k ;

[0086] When b%2=0, the exploration implicit value change function population is iterated to obtain an updated exploration implicit value change function population; at the same time, the development implicit value change function population is iterated to obtain an updated development implicit value change function population, and a new sentinel implicit value change function individual is selected;

[0087] When b%2≠0, the exploration implicit value change function population does not perform population iteration and no operation is performed; the development implicit value change function population performs population iteration around the sentinel implicit value change function individuals to obtain an updated development implicit value change function population;

[0088] When b=B, the simulated implicit value change function individual with the largest fitness in the exploration implicit value change function population and the development implicit value change function population is output, which is the optimal simulated implicit value change function individual; the solution of the database implicit value change function in the optimal simulated implicit value change function individual is used as the database implicit value change function;

[0089] % represents the modulo operation. By constructing multiple simulated implicit value change function individuals and performing iterative optimization, the system can select the optimal solution from multiple candidate implicit value change functions. This process can effectively reduce the errors that may occur when the implicit value of the database is changed, ensuring that the database update is more accurate and reliable; the fitness calculation evaluates the effect of the implicit value change function by matching the simulated database partition results with the actual database allocation results, further improving the accuracy and effect of the database implicit value change function; exploring the implicit value change function population focuses on a wide search for potential implicit value change solutions, while developing the implicit value change function population focuses on the current optimal solution. The combination of the two can avoid falling into the local optimal solution too early, improve the global search ability of the model, and better adapt to the changing user needs and data environment; by controlling the number of iterations b, especially the collaborative iteration of the exploration population and the development population when b%2 is an even number, and only iterating the development population when b%2 is an odd number, the search strategy can be flexibly adjusted to enhance the model's ability to adapt to the implicit value change process of the database; through the competition and evolution of multiple populations, the optimal implicit value change function finally output can ensure that the implicit value change process in the database management process is more accurate and efficient, thereby improving the overall performance and stability of the system.

[0090] Example 2, a data management system for a multimodal database, see Figure 1 As shown, including:

[0091] The multimodal data analysis module includes a data acquisition unit and a data analysis unit; the data acquisition unit is used to acquire a multimodal data set to be managed, and the multimodal data set to be managed includes N multimodal data D to be managed. n , multimodal data to be managed D nis text data or image data; the data analysis unit is used to analyze based on the multimodal data set to be managed and the multimodal data feature analysis model to obtain the fusion feature R of the multimodal data to be managed n , n=1,2,…,N;

[0092] The multimodal data storage module includes a data storage unit; the data storage unit is used to fuse the feature R of the multimodal data to be managed. n and multimodal data management allocation model, treat the management of multimodal data D n Perform database storage allocation and store the multimodal data set to be managed in the multimodal database; wherein the multimodal database contains M multimodal database partitions P m , m=1,2,…,M; each multimodal database partition P m Corresponding to a database access implicit value Y m ;

[0093] The multimodal data update module includes a data storage update unit; the data storage update unit is used to obtain the target user operation record; based on the target user operation record and the multimodal database management update model, the multimodal database partition P m Perform dynamic updates to obtain a new multimodal database partition P m , and assign a new multimodal database partition P m New database access implicit value Y m .

[0094] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention. Parts not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A data management method for a multimodal database, characterized in that: The following steps are involved: Get the multimodal data set to be managed, which contains N multimodal data D to be managed. n , multimodal data to be managed D n is text data or image data; based on the multimodal data set to be managed and the multimodal data feature analysis model, the fusion feature R of the multimodal data to be managed is obtained n , n=1,2,…,N; The multimodal data feature analysis model includes a data preprocessing layer, a feature extraction layer, a comprehensive analysis layer, and a fusion feature output layer. The basic CNN model is improved based on the knowledge graph algorithm, and a feature adaptive threshold factor Z is introduced to extract the multimodal data D to be managed. n The deep-level characteristics of Based on the fusion feature R of multimodal data to be managed n and multimodal data management allocation model, treat the management of multimodal data D n Perform database storage allocation and store the multimodal data set to be managed in the multimodal database; wherein the multimodal database contains M multimodal database partitions P m , m=1,2,…,M; each multimodal database partition P m Corresponding to a database access implicit value Y m ; The multimodal data management allocation model includes a feature matching layer, a partition storage layer, and a result output layer. The feature matching technology is used to manage multimodal data. n Perform database storage allocation; Obtain the target user's operation records; based on the target user's operation records and the multimodal database management update model, partition the multimodal database P m Perform dynamic updates to obtain a new multimodal database partition P m , and assign a new multimodal database partition P m New database access implicit value Y m ; The multimodal database management update model includes an operation feature quantification layer, an implicit value change layer, and a database storage update layer, and is optimized based on a heuristic improved group optimization algorithm. The multimodal data feature analysis model includes a data preprocessing layer, a feature extraction layer, a comprehensive analysis layer, and a fusion feature output layer; The data preprocessing layer is used to manage multimodal data D n Perform preprocessing to obtain preprocessed multimodal data D to be managed n '; The feature extraction layer is used to use the improved knowledge graph algorithm and CNN model to preprocess the multimodal data D n 'Perform feature extraction to obtain the multimodal data features T to be managed n ; The comprehensive analysis layer is used to integrate all pre-processed multimodal data D n 'Perform fusion feature analysis to obtain multimodal data set features; The fusion feature output layer is used to transform the multimodal data features T to be managed n Perform feature fusion with the multimodal data set features to obtain the fusion feature R of the multimodal data to be managed n ; The specific steps of feature extraction in the feature extraction layer include: The feature extraction layer consists of I convolutional layers, i=1, 2, ..., I; the feature adaptive threshold factor Z is set, , σ is the multimodal data convolution feature J 3i-2 The standard deviation of represents the fluctuation range of the feature; μ is the convolution feature J of multimodal data 3i-2 The mean of the features, indicating the average level of the features; H(F) is the convolution feature J of the multimodal data 3i-2 The entropy of represents the complexity of the feature; α, β and ϵ are adjustment coefficients; In the 3i-2th convolutional layer, the preprocessed multimodal data D n ' or multimodal data convolution feature J 3i Perform feature recognition to obtain multimodal data convolution feature J 3i-2 ; Calculate the current feature adaptive threshold factor Z; when the feature adaptive threshold factor Z is greater than the preset feature adjustment threshold, reduce the pooling strength of the 3i-1th convolution layer; otherwise, remain unchanged; In the 3i-1th convolutional layer, the multimodal data convolution feature J 3i-2 Perform pooling operation to obtain multimodal data convolution feature J 3i-1 ; In the 3i convolutional layer, the multimodal data convolution feature J 3i-1 Perform entity recognition to obtain multimodal data convolution feature J 3i ; Repeat the above operation, and in the I convolution layer, obtain the multimodal data features T to be managed n ; The multimodal data management allocation model includes a feature matching layer, a partition storage layer and a result output layer; The feature matching layer is used to manage multimodal data fusion features R n Perform feature matching to obtain the database access matching prediction result G n ; The partition storage layer is used to access the matching prediction results G based on the database n Matches the closest database access implicit value Y m , get the database management allocation result Q n ; The result output layer is used to manage the distribution results Q based on all databases n The multimodal data to be managed D n Stored to the corresponding multimodal database partition P m middle; The multimodal database management update model includes an operation feature quantification layer, an implicit value change layer and a database storage update layer; The operation feature quantization layer is used to quantify the target user's operation records and obtain the target user's operation feature quantization value L m ; The implicit value change layer is used to quantify the value L according to the target user's operation characteristics. m Analyze with the database implicit value change function to get the new database access implicit value Y m ;The database implicit value change function is determined by the heuristic improved group optimization algorithm; The database storage update layer is used to access the implicit value Y based on the new database m Redistribute multimodal database partitions P m The multimodal data in the new multimodal database partition P m ; The specific steps for determining the database implicit value change function include: Construct 2K simulated implicit value change function individuals A k , k=1, 2, ..., 2K; each simulated implicit value change function individual A k Contains the solution to the implicit value change function of the database; the first K simulated implicit value change function individuals A k As the population for exploring the implicit value change function, the next K simulated implicit value change function individuals A k As the population of the development implicit value change function; set the number of iterations b, b=1, 2, ..., B; record the value with the largest fitness in the population of the development implicit value change function as the sentinel implicit value change function individual; The specific steps for calculating fitness include: Collecting several groups of verified initial feature quantization values ​​and corresponding database allocation results as a group of database implicit value change training samples; Change function individual A according to the simulated implicit value k Perform simulation calculation on the database implicit value change training samples to obtain the simulated change implicit value, and perform database partition matching according to the simulated feature quantization value to obtain the simulated database partition result; calculate the accuracy of the simulated database partition result and the corresponding database allocation result to obtain the simulated calculation accuracy; use all the simulated calculation accuracy as the simulated implicit value change function individual A k The fitness S k ; When b%2=0, the exploration implicit value change function population is iterated to obtain an updated exploration implicit value change function population; at the same time, the development implicit value change function population is iterated to obtain an updated development implicit value change function population, and a new sentinel implicit value change function individual is selected; When b%2≠0, the exploration implicit value change function population does not perform population iteration and no operation is performed; the development implicit value change function population performs population iteration around the sentinel implicit value change function individuals to obtain an updated development implicit value change function population; When b=B, the simulated implicit value change function individual with the largest fitness in the exploration implicit value change function population and the development implicit value change function population is output, which is the optimal simulated implicit value change function individual; the solution of the database implicit value change function in the optimal simulated implicit value change function individual is used as the database implicit value change function.

2. The data management method of a multimodal database according to claim 1, characterized in that: The specific steps for training the feature matching layer include: Collect several groups of data feature matching samples with label values; each group of data feature matching samples contains data fusion features and label values ​​corresponding to verified data, and the label value represents the data access label; several groups of data feature matching samples with label values ​​are combined to obtain a data feature matching training set; The data feature matching training set is input into the multimodal data management allocation model to train the feature matching layer using the label value as the target to obtain the initial feature matching layer; the initial feature matching layer is model evaluated to obtain the initial feature matching layer model evaluation result; if the initial feature matching layer model evaluation result is passed, the initial feature matching layer is used as the feature matching layer in the multimodal data management allocation model; otherwise, the model training is continued using the data feature matching training set.

3. A data management system for a multimodal database, characterized in that: The system applies a data management method for a multimodal database as described in any one of claims 1 to 2, including: The multimodal data analysis module includes a data acquisition unit and a data analysis unit; the data acquisition unit is used to acquire a multimodal data set to be managed, and the multimodal data set to be managed includes N multimodal data D to be managed. n , multimodal data to be managed D n is text data or image data; the data analysis unit is used to analyze based on the multimodal data set to be managed and the multimodal data feature analysis model to obtain the fusion feature R of the multimodal data to be managed n , n=1,2,…,N; The multimodal data storage module includes a data storage unit; the data storage unit is used to fuse the feature R of the multimodal data to be managed. n and multimodal data management allocation model, treat the management of multimodal data D n Perform database storage allocation and store the multimodal data set to be managed in the multimodal database; wherein the multimodal database contains M multimodal database partitions P m , m=1,2,…,M; each multimodal database partition P m Corresponding to a database access implicit value Y m ; The multimodal data update module includes a data storage update unit; the data storage update unit is used to obtain the target user operation record; based on the target user operation record and the multimodal database management update model, the multimodal database partition P m Perform dynamic updates to obtain a new multimodal database partition P m , and assign a new multimodal database partition P m New database access implicit value Y m .

Images