Market management platform based on multi-modal data fusion

Abstract

The invention discloses a market management platform based on multi-modal data fusion, and relates to the technical field of market data processing, the management platform comprises a task planning module, a data acquisition module, a multi-modal identification module, a data analysis module and a feedback module, the task planning module is used for planning different feature data, and the data acquisition module is used for acquiring different feature data; the data acquisition module is used for acquiring multi-modal data and classifying the multi-modal data according to the feature data, and the multi-modal identification module identifies the classified multi-modal data, normalizes the identified multi-modal data, and sends the normalized multi-modal data to the data processing module; the system has the advantages that the application fields of different image and video data in the market are analyzed through the multi-modal identification module, and then the multi-modal data models in the different application fields are called to be connected in series, so that the multi-modal data models process single market data; and the analysis accuracy of the fused multi-modal data is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of market data processing, and specifically to a market management platform based on multi-modal data fusion. Background Technique

[0002] A market management platform is a digital tool system used by enterprises to optimize and manage market activities. Its main purpose is to help enterprises and certain institutions achieve real-time monitoring of market data, planning of market activities, customer management, and marketing effect analysis, etc., and then assist in improving the efficiency and accuracy of enterprise decision-making and maintaining enterprise competitiveness by integrating various aspects of information in market operations.

[0003] With the development of technology, various types of data from different data sources are increasingly integrated and analyzed by artificial intelligence in today's market management platforms. Most of them are based on various data parameters publicly disclosed on official or civilian websites, and the development trend of the market is quantitatively analyzed through algorithms and a large number of data parameters;

[0004] In the existing market management data, due to different application fields in the same type of data, the accuracy of data analysis may be affected. For this reason, we propose a market management platform based on multi-modal data fusion. Summary of the Invention

[0005] The purpose of the present invention is to provide a market management platform based on multi-modal data fusion.

[0006] To solve the problems raised in the above background technique, the present invention provides the following technical solution: A market management platform based on multi-modal data fusion, the management platform includes a task planning module, a data collection module, a multi-modal recognition module, a data analysis module, and a feedback module;

[0007] The task planning module is used to plan different feature data;

[0008] The data collection module is used to collect multi-modal data and classify the multi-modal data according to the feature data;

[0009] The multi-modal recognition module identifies the classified multi-modal data, performs normalization processing on the identified multi-modal data, and performs evaluation processing on the normalized multi-modal data, and sets the normalized multi-modal data as X i , that is, X i =f i1 +f i2 +……+f in to obtain the application proportion of multi-modal data in different fields, where f i1 , f i2 , ……, f inrespectively represent the application proportion of the i-th multi-modal data in different fields, and the data analysis module corresponding to the application proportion ≤ 20β ia % is removed, and they are sorted in ascending order according to the application proportion data, where β i1 represents the weight coefficient of the i-th multi-modal data in the a-th application field;

[0010] In the data analysis module, multi-modal data models for different fields are set, and the multi-modal data models are connected in series according to the sorting of the application proportion. Subsequently, the normalized multi-modal data is imported into the multi-modal data models in sequence;

[0011] The feedback module generates a visualization chart according to the output result of the multi-modal data model.

[0012] As a further solution of the present invention: a data fusion unit and a management feature storage unit are set in the task planning module. The data fusion unit is used to obtain the application field of the multi-modal data collected by the data acquisition module, plan according to the market images and video data in the application field of the multi-modal data, and the task planning module retrieves the corresponding feature data in the management feature storage unit according to different application fields, and analyzes the feature correlation data according to the temporal and spatial relationships in the market images and video data.

[0013] As a further solution of the present invention: the data acquisition module includes a data quality analysis module, an image classification unit, and a temporary storage unit. After the data acquisition module matches the feature correlation data with the collected market images and video data, the temporary storage unit temporarily stores the matched data. The data quality analysis module evaluates the data quality of the pixel sizes of the market images and videos, and the image classification unit analyzes the feature data in the images that are not recognized by the feature correlation data, and simultaneously obtains the application field feature data different from the feature correlation data.

[0014] As a further solution of the present invention: the multi-modal recognition module evaluates the market images and video data, and obtains the accuracy of the classification of the multi-modal data collected by the data acquisition module by analyzing the texture and color in the market images and video data. When the accuracy of the multi-modal data classification is lower than 90%, the cross-validation evaluation method is introduced.

[0015] As a further solution of the present invention: the multi-modal recognition module normalizes the image data through a formula, and the specific formula is as follows:

[0016]

[0017] Among them, X 总Denote the normalized data of multimodal data, \(T\) represents the process of normalizing multimodal data, \(N\) represents the maximum number of application fields, and \(X\). a·min Denote the minimum threshold in the \(a\)-th application field, \(X\). a·max Denote the maximum threshold in the \(a\)-th application field.

[0018] As a further solution of the present invention: A personalized unit and a data management unit are provided in the multimodal recognition module. The personalized unit retrieves the historical multimodal data in the data management unit, analyzes the personalized differences according to the multimodal data. After the analysis by the personalized unit, the normalized multimodal data is also transmitted to the data management unit. The personalized unit analyzes the personalized differences through a formula, and the specific formula is as follows:

[0019]

[0020] Among them, \(Q\). r Denote the analysis data of personalized differences, \(Q\) represents the difference data, \(W\) represents the judgment of differences, and \(W=(0,1)\), \(Z\). 阈 Denote the quantity threshold of difference data.

[0021] As a further solution of the present invention: A permission management unit is provided in the data analysis module. The permission management unit analyzes the permission level according to the personalized difference data, and sets different permission levels as \(E1\) and \(E2\) in sequence according to the size of the differences. When \(Q\). r \(<E1\), the data analysis module retrieves 0%-20% of the computing power of the multimodal data model related to the current application field for processing. When \(E1\leq Q\). r \(<E2\), the data analysis module retrieves 20%-50% of the computing power of the multimodal data model related to the current application field for processing. When \(Q\). r \(\geq E2\), the data analysis module retrieves more than 50% of the computing power of the multimodal data model related to the current application field for processing.

[0022] As a further solution of the present invention: After the data analysis module exports the chart data, the feedback module optimizes the chart data, and at the same time imports the optimized data into the simulation model, and generates a feedback report according to the simulation model, and transmits the feedback report data to the cloud and the management platform.

[0023] Adopting the above technical solutions, compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] 1. The present invention analyzes the application fields of different images and video data in the market through a multimodal recognition module, and then retrieves and concatenates multimodal data models in different application fields, enabling the multimodal data models to process individual market data, thereby improving the accuracy of the fused multimodal data analysis.

[0025] 2. The present invention pre-plans and processes multimodal data in advance through a task planning module, facilitating the data acquisition module to analyze and process the acquired multimodal data according to the pre-planning process, thereby improving the operating speed of the management platform.

[0026] 3. The present invention analyzes market images or video data to ensure the accuracy of the data collected by the data acquisition module, and then normalizes the data using a formula, facilitating the management platform to analyze based on the normalized data.

[0027] 4. The present invention adjusts the computing power of the multimodal data model according to different difference levels, reducing the computing power when the difference is smaller, enabling the multimodal data model to use the excess computing power to process other resources, thereby improving the resource utilization rate of the overall management platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flowchart of the management platform in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0030] Embodiment 1:

[0031] Please refer to the attached Figure 1 , a market management platform based on multimodal data fusion of the present invention, the management platform includes a task planning module, a data acquisition module, a multimodal recognition module, a data analysis module, and a feedback module;

[0032] The task planning module is used to plan different feature data;

[0033] The data acquisition module is used to collect multimodal data and classify the multimodal data according to the feature data;

[0034] The multimodal recognition module recognizes the classified multimodal data, normalizes the recognized multimodal data, and evaluates the normalized multimodal data, and sets the normalized multimodal data as X i , that is, X i =f i1 +f i2 +……+f in Obtain the application proportion of multimodal data in different fields. Among them, f i1 , f i2 , ……, f in respectively represent the application proportion of the i-th multimodal data in different fields. Eliminate the corresponding data analysis modules with the application proportion ≤ 20β ia %, and sort them in ascending order according to the application proportion data. Among them, β i1 represents the weight coefficient of the i-th multimodal data in the a-th application field;

[0035] Set multimodal data models for different fields in the data analysis module, and connect the multimodal data models in series according to the sorting of the application proportion, and then import the normalized multimodal data into the multimodal data models in turn;

[0036] The feedback module generates a visualization chart according to the output result of the multimodal data model.

[0037] Specific working process: Obtain visual information such as the staying time and expressions of consumers in front of the shelves through in-store surveillance cameras. For example, if a consumer stays in front of a certain product shelf for a long time and has a concentrated expression, it may indicate a high interest in the product. Combine the consumer expressions and staying time in the image data, the comment content in the text data, and the purchase frequency in the behavior data to construct a model of the degree of consumer preference for the product. For example, it is found that consumers who stay in front of a certain product for a long time, have a high purchase frequency, and have good comments are loyal users of the product, and targeted marketing can be carried out. Combine geographical, demographic, and consumer psychology data to divide the market into different target markets, set multimodal data models for different fields, connect the multimodal data models in series according to the sorting of the application proportion, import the normalized multimodal data into the multimodal data models in turn, and generate a visualization chart according to the output result of the multimodal data model.

[0038] Furthermore, analyze the application fields of different images and video data in the market through the multimodal recognition module, and then retrieve and connect the multimodal data models in different application fields, so that the multimodal data model processes the single market data, thereby improving the accuracy of the fused multimodal data analysis.

[0039] Embodiment 2:

[0040] Based on Example 1, please refer to the appendix Figure 1 As shown, a data fusion unit and a management feature storage unit are set in the task planning module. The data fusion unit is used to obtain the application fields of the multi-modal data collected by the data acquisition module, plan according to the market images and video data in the application fields of the multi-modal data, and the task planning module retrieves the corresponding feature data in the management feature storage unit according to different application fields, and analyzes the feature correlation data according to the temporal and spatial relationships in the market images and video data. The data acquisition module includes a data quality analysis module, an image classification unit and a temporary storage unit. After the data acquisition module matches the feature correlation data with the collected market images and video data, the temporary storage unit temporarily stores the matched data. The data quality analysis module evaluates the data quality of the pixel sizes of the market images and videos, and the image classification unit analyzes the feature data in the images that are not recognized by the feature correlation data, and at the same time obtains the application field feature data different from the feature correlation data.

[0041] Specific working process: Set up a data fusion unit and a management feature storage unit, obtain the application fields of the multi-modal data collected by the data acquisition module through the data fusion unit, plan according to the market images and video data in the application fields of the multi-modal data, retrieve the corresponding feature data in the management feature storage unit according to different application fields, analyze the feature correlation data according to the temporal and spatial relationships of the market images and video data, match the feature correlation data with the collected market images and video data, the temporary storage unit temporarily stores the matched data, the data quality analysis module evaluates the data quality of the pixel sizes of the market images and videos, the image classification unit analyzes the feature data in the images that are not recognized by the feature correlation data, and at the same time obtains the application field feature data different from the feature correlation data;

[0042] The multi-modal data is not limited to images and videos, but may also include sounds, texts, etc.

[0043] Furthermore, the task planning module pre-plans and processes the multi-modal data in advance, so that it is convenient for the data acquisition module to analyze and process the collected multi-modal data according to the pre-planning process, thereby improving the operation speed of the management platform.

[0044] Example 3:

[0045] Based on Example 2, please refer to the appendix Figure 1As shown, the multi-modal recognition module evaluates market images and video data. By analyzing the texture and color in the market images and video data, it obtains the accuracy of classifying the multi-modal data collected by the data acquisition module. When the accuracy of classifying the multi-modal data is lower than 90%, the cross-validation evaluation method is imported. The multi-modal recognition module normalizes the image data through a formula, and the specific formula is as follows:

[0046]

[0047] Among them, X 总 represents the normalized data of the multi-modal data, T represents the process of normalizing the multi-modal data, N represents the maximum number of application fields, and X a·min represents the minimum threshold in the a-th application field, and X a·max represents the maximum threshold in the a-th application field.

[0048] Furthermore, by analyzing the market image or video data, the accuracy of the data collected by the data acquisition module is ensured, and then the data is normalized using the formula, which facilitates the management platform to analyze based on the normalized data.

[0049] Example 4:

[0050] Based on Example 3, please refer to the attached Figure 1 As shown, a personalized unit and a data management unit are set in the multi-modal recognition module. The personalized unit retrieves the historical multi-modal data in the data management unit, analyzes the personalized differences based on the multi-modal data. After the analysis by the personalized unit, the normalized multi-modal data is also transmitted to the data management unit. The personalized unit analyzes the personalized differences through a formula, and the specific formula is as follows:

[0051]

[0052] Among them, Q r represents the analysis data of the personalized differences, Q represents the difference data, W represents the judgment of the difference, and W = (0, 1), Z 阈 represents the quantity threshold of the difference data. A permission management unit is set in the data analysis module. The permission management unit determines the permission level according to the personalized difference data analysis, and sets different permission levels as E1 and E2 in sequence according to the size of the difference. When Q r < E1, the data analysis module retrieves 0% - 20% of the computing power of the multi-modal data model related to the current application field for processing. When E1 ≤ Q r < E2, the data analysis module retrieves 20% - 50% of the computing power of the multi-modal data model related to the current application field for processing. When Q rWhen it is ≥E2, the data analysis module retrieves the computing power of the multimodal data model that is more than 50% relevant to the current application field for processing. After the data analysis module exports the chart data, the feedback module optimizes the chart data, and at the same time imports the optimized data into the simulation model, and generates a feedback report according to the simulation model, and transmits the feedback report data to the cloud and the management platform.

[0053] Furthermore, the computing power of the multimodal data model is retrieved through different difference levels, so that when the difference is smaller, the computing power is reduced, enabling the multimodal data model to use the redundant computing power to process other resources, thereby improving the resource utilization rate of the overall management platform.

[0054] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification only illustrates the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A market management platform based on multi-modal data fusion, characterized in that: The management platform includes a task planning module, a data acquisition module, a multi-modal recognition module, a data analysis module, and a feedback module; The task planning module is used to plan different feature data; The data acquisition module is used to acquire multi-modal data and classify the multi-modal data according to the feature data; The multi-modal recognition module recognizes the classified multi-modal data, normalizes the recognized multi-modal data, and performs an evaluation process on the normalized multi-modal data, and sets the normalized multi-modal data as X i , that is, X i = f i1 + f i2 +……+ f in Obtain the application proportion of the multi-modal data in different fields. Among them, f i1 , f i2 , ……, f in respectively represent the application proportion of the i-th multi-modal data in different fields. Eliminate the corresponding data analysis modules with an application proportion ≤ 20β ia %, and sort them in ascending order according to the application proportion data. Among them, β i1 represents the weight coefficient of the i-th multi-modal data in the a-th application field; In the data analysis module, multi-modal data models in different fields are set, and the multi-modal data models are connected in series according to the sorting of the application ratios. Subsequently, the normalized multi-modal data is sequentially imported into the multi-modal data models; The feedback module generates a visualization chart according to the output result of the multi-modal data model.

2. The market management platform based on multi-modal data fusion according to claim 1, wherein: A data fusion unit and a management feature storage unit are set in the task planning module. The data fusion unit is used to obtain the application field of the multi-modal data acquired by the data acquisition module, plan according to the market images and video data in the application field of the multi-modal data, and the task planning module retrieves the corresponding feature data in the management feature storage unit according to different application fields, and analyzes the feature correlation data according to the temporal and spatial relationships in the market images and video data.

3. The market management platform based on multi-modal data fusion according to claim 2, characterized in that: The data acquisition module includes a data quality analysis module, an image classification unit, and a temporary storage unit. After the data acquisition module matches the feature correlation data with the acquired market images and video data, the temporary storage unit temporarily stores the matched data. The data quality analysis module evaluates the data quality of the pixel sizes of the market images and videos, and the image classification unit analyzes the feature data in the images that have not been recognized by the feature correlation data, and simultaneously obtains the application field feature data different from the feature correlation data.

4. A market management platform based on multi-modal data fusion according to claim 3, characterized in that: The multi-modal recognition module evaluates the market images and video data, and obtains the accuracy of the classification of the multi-modal data acquired by the data acquisition module by analyzing the texture and color in the market images and video data. When the accuracy of the multi-modal data classification is lower than 90%, a cross-validation evaluation method is imported.

5. The market management platform based on multi-modal data fusion according to claim 3, characterized in that: The multi-modal recognition module normalizes the image data through a formula. The specific formula is as follows: Among them, X 总 represents the normalized data of multimodal data, T represents the process of normalizing multimodal data, N represents the maximum number of application fields, and X a·min represents the minimum threshold in the a-th application field, and X a·max represents the maximum threshold in the a-th application field.

6. A market management platform based on multi-modal data fusion according to claim 1, characterized in that: A personalized unit and a data management unit are set in the multi-modal recognition module. The personalized unit retrieves the historical multi-modal data in the data management unit, analyzes the personalized differences according to the multi-modal data. After the analysis by the personalized unit, the normalized multi-modal data is also transmitted to the data management unit. The personalized unit analyzes the personalized differences through a formula. The specific formula is as follows: Among them, Q r represents the analysis data of personalized differences, Q represents the difference data, W represents the judgment of differences, and W = (0, 1), Z 阈 represents the quantity threshold of the difference data.

7. The market management platform based on multi-modal data fusion according to claim 6, characterized in that: A permission management unit is set in the data analysis module. The permission management unit analyzes the permission levels based on personalized difference data, and sequentially sets different permission levels as E1 and E2 according to the size of the differences. When Q r < E1, the data analysis module retrieves 0%-20% of the computing power of the multimodal data model related to the current application field for processing. When E1 ≤ Q r < E2, the data analysis module retrieves 20%-50% of the computing power of the multimodal data model related to the current application field for processing. When Q r ≥ E2, the data analysis module retrieves more than 50% of the computing power of the multimodal data model related to the current application field for processing.

8. A market management platform based on multi-modal data fusion according to claim 1, characterized in that: After the data analysis module exports the chart data, the feedback module optimizes the chart data, and at the same time imports the optimized data into the simulation model, generates a feedback report according to the simulation model, and transmits the feedback report data to the cloud and the management platform.

Images