Scene management and intelligent recommendation method and system based on data space

Abstract

The invention provides a scene management and intelligent recommendation method and system based on a data space, and the method comprises the steps: selecting a system function in a preset database, and carrying out the matching of a corresponding data type according to the selected system function, making a corresponding data acquisition plan according to system functions and data types; collecting data based on the formulated data collection plan and selecting a data processing model matched with the collected data according to the collected data; processing the collected data to form a feature fusion layer, and introducing an attention mechanism and an inter-channel information transmission mechanism into the selected data processing model to optimize the selected data processing model; and training the optimized data processing model to obtain an attention mechanism weight matrix, and adjusting the obtained attention weight matrix to realize personalized recommendation. The technical problem that the heterogeneous data processing and recommendation ability is insufficient in the related technology is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of heterogeneous data processing, and in particular to a scene management and intelligent recommendation method and system based on data space. Background Art

[0002] With the development of data space technology, data generated by multiple devices and in different environments is becoming increasingly complex. In the field of scene management and intelligent recommendation in data space, such as in application scenarios such as smart homes, the heterogeneity of data and information fragmentation pose challenges to the effective recommendation and scene management of the system. Since there are various types of data forms in the data space, such as pictures, text, audio, etc., we need to take completely different approaches to process different types of data. These data of different modalities are difficult to be effectively learned by artificial intelligence models. In the existing technology, data space scene management and intelligent recommendation face problems such as the integration of heterogeneous data, data association learning in different scenarios, and comprehensive consideration of user personalized needs. The current solution has relatively limited processing and recommendation capabilities when facing different data sources, forms and scenarios.

[0003] Therefore, the prior art needs to be further developed. Summary of the invention

[0004] The purpose of the present invention is to overcome the above-mentioned technical deficiencies and provide a scene management and intelligent recommendation method and system based on data space to solve the technical problem of insufficient processing and recommendation capabilities for heterogeneous data in related technologies.

[0005] In order to achieve the above technical objectives, the present invention adopts the following technical solutions: a scene management and intelligent recommendation method based on data space is provided, including: selecting system functions in a preset database, matching corresponding data types according to the selected system functions, and formulating corresponding data collection plans according to the system functions and data types; collecting data based on the formulated data collection plan and selecting a data processing model that matches the collected data according to the collected data; processing the collected data to form a feature fusion layer, and introducing an attention mechanism and an information transmission mechanism between channels into the selected data processing model to optimize the selected data processing model; training the optimized data processing model to obtain an attention mechanism weight matrix, and adjusting the obtained attention weight matrix to achieve personalized recommendations.

[0006] Furthermore, the method for formulating a data collection plan includes: selecting a data source according to a data type, and determining a start time for data collection, a frequency for data collection, and a duration for data collection according to a system function.

[0007] Furthermore, the method of selecting a data processing model that matches the collected data includes: performing data preprocessing on the collected data to convert the collected data into structured data; cleaning and deduplicating the structured data; establishing a data repository to store the collected data metadata and data documents of the cleaned and deduplicated structured data; performing data analysis on the structured data to determine data correlation, and determining the data processing model based on the data correlation and model performance.

[0008] Furthermore, the method for cleaning and deduplicating structured data includes: processing missing values, outliers and duplicate data of structured data; wherein, the method for processing missing values ​​includes deleting missing data, filling missing values ​​and filling using interpolation methods; the method for processing outliers includes identifying and processing using statistical methods or methods based on business logic; the method for processing duplicate data includes identifying duplicate data and deleting or merging the duplicate data.

[0009] Furthermore, metadata includes dataset name, description, creator, field type and field length, and metadata is used to describe data; data documents include data dictionary, data collection log, and data cleaning record, and data documents are used to record key information in the data collection and processing process.

[0010] Furthermore, the method for processing the collected data includes convolution, pooling, and multi-scale feature extraction on the collected data, and splicing or superimposing feature maps from different convolutional layers.

[0011] Furthermore, the method for optimizing the selected data processing model also includes introducing regularization technology and activation function into the selected data processing model to enhance the performance of the data processing model.

[0012] Furthermore, the method for realizing personalized recommendation includes: analyzing the obtained attention weight matrix, and obtaining the decision basis of the data processing model based on the analysis results; adjusting the attention weight matrix according to user behavior feedback to obtain the attention weight priority table; determining the processing order of the optimized data according to the obtained attention weight matrix priority table to realize personalized recommendation.

[0013] A computer-readable storage medium stores computer-readable instructions. When the computer-readable instructions are run on a communication device, the above method is executed.

[0014] A scene management and intelligent recommendation system based on data space, comprising: a plan making unit, the plan making unit is used to select a system function in a preset database, match a corresponding data type according to the selected system function, and make a corresponding data collection plan according to the system function and the data type; a data collection unit, the data collection unit is used to collect data based on the formulated data collection plan and select a data processing model matching the collected data according to the collected data; a data processing unit, the data processing unit is used to process the collected data to form a feature fusion layer, and introduce an attention mechanism and an information transmission mechanism between channels into the selected data processing model to optimize the selected data processing model; a data output unit, the data output unit is used to train the optimized data processing model, obtain an attention mechanism weight matrix, and adjust the obtained attention weight matrix to achieve personalized recommendation.

[0015] Beneficial effects:

[0016] 1. The scene management and intelligent recommendation method based on data space of the present invention can realize the intelligent integration and management of diversified data in different scenes, improve the organization and utilization of data flow, and enhance the system's adaptability to complex environments.

[0017] 2. The data space-based scenario management and intelligent recommendation method of the present invention can realize more accurate personalized intelligent recommendation services. By conducting comprehensive analysis of multi-source data, it can more comprehensively understand user needs, thereby providing personalized recommendation content that is more in line with user interests and preferences, thereby improving user satisfaction.

[0018] 3. The scene management and intelligent recommendation method based on data space of the present invention helps to solve the problem of data collaboration between different devices, different models and different environments. Through effective data space management, the system can realize collaborative learning and interaction of data between devices, and promote more collaborative and intelligent operation of smart devices in home scenes.

[0019] 4. The scene management and intelligent recommendation method based on data space of the present invention can realize the perception and intelligent response of scenes. By deeply understanding the scenes in the data space, it can intelligently adjust the working state of the equipment and improve the efficiency and intelligence level in specific scenes.

[0020] 5. The data space-based scenario management and intelligent recommendation method of the present invention can improve the interpretability and explainability of the intelligent recommendation model, so that users can more clearly understand how the model makes recommendation decisions, thereby establishing a sense of trust and encouraging users to participate more actively in system operations.

[0021] 6. The data space-based scenario management and intelligent recommendation method of the present invention can enhance the robustness of the system to different data sources and environmental changes. Through more intelligent data processing and learning mechanisms, the system can better adapt to different operating scenarios and improve the stability and adaptability of the system.

[0022] 7. The scene management and intelligent recommendation method based on data space of the present invention can utilize energy more effectively and reduce energy consumption through intelligent management of data space. For example, in the smart home scene, the system can intelligently adjust the operating status of the device according to real-time demand, thereby realizing the rational use of energy.

[0023] 8. The data space-based scenario management and intelligent recommendation method of the present invention can promote scientific research progress in related fields and play an important role in commercial applications. It provides advanced scenario management and intelligent recommendation technologies, provides innovative solutions for enterprises and research institutions, and promotes the development of the industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flow chart of a scenario management and intelligent recommendation method based on data space adopted in the first embodiment of the present invention;

[0025] Figure 2 It is a flow chart of the scene management and intelligent recommendation method based on data space adopted in the second embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] Embodiment 1:

[0028] like Figure 1 A scene management and intelligent recommendation method based on data space shown includes:

[0029] S110 selects a system function in a preset database, matches a corresponding data type according to the selected system function, and formulates a corresponding data collection plan according to the system function and the data type.

[0030] In specific practice, the method of formulating a data collection plan includes: selecting a data source based on the data type, and determining the start time, frequency, and duration of data collection based on system functions.

[0031] It should be noted that effective data collection in the data space is the core step in building an intelligent recommendation and scenario management system. This includes clearly defining data requirements and goals, selecting appropriate data sources, formulating a reasonable data collection plan, collecting and processing structured, semi-structured and unstructured data, cleaning and deduplication, protecting privacy and security, establishing data storage systems and monitoring systems, and establishing documents and metadata. This comprehensive data collection process aims to ensure that the collected data is of high quality and strong consistency, providing a reliable foundation for the system, thereby achieving accurate and personalized services for intelligent recommendation and scenario management.

[0032] The method of defining data requirements and goals is as follows: To determine what type of data the system needs, first you need to clarify the system's functions and goals, and then determine what type of data the system needs. Structured data usually exists in the form of tables, which can include user information, transaction records, etc., and is suitable for scenarios that require database queries, statistical analysis, etc. Semi-structured data, such as data in XML and JSON formats, has a certain organizational structure and is often used for data exchange and storage. Unstructured data includes text, images, audio, etc., which need to be analyzed and processed through methods such as deep learning. For example, for an e-commerce platform, the system may need structured user purchase records, semi-structured product information and comments, and unstructured user messages and picture uploads. It includes structured data (such as table data), semi-structured data (such as data in XML and JSON formats), and unstructured data (such as text, images, audio). At the same time, clarify the frequency and time span of data collection.

[0033] The method of selecting a data source is: select an appropriate data source, which may include sensors, devices, databases, APIs (application programming interfaces), web crawlers, etc. Ensure that the data source can provide the data required by the system, and consider data quality and reliability. Sensors can provide real-time environmental data, such as temperature, humidity, etc., which are suitable for scenarios that require real-time monitoring; device data can provide device status and operation information, which is suitable for device health status monitoring; a large amount of structured data is stored in the database, which is suitable for scenarios that require statistical analysis and query; APIs can access data from external services, such as weather forecasts, geographic location information, etc., which are suitable for scenarios that require additional external information; web crawlers can obtain data on the Internet, which is suitable for scenarios that require access to public information. According to system requirements, select an appropriate data source to ensure data quality and reliability.

[0034] The method of formulating a data collection plan is to formulate a data collection plan, including the time point, frequency, and duration of data collection. This helps to ensure that the system can obtain data on time and on demand. Formulating a data collection plan is a key step to ensure that the system can obtain data on time and on demand. The time point, frequency, and duration of data collection need to be determined. For example, for a real-time monitoring system, data collection may need to be performed once a minute for 24 hours; for a periodic analysis system, data collection can be performed at a fixed time every day to obtain data from the previous day. When formulating a plan, it is also necessary to consider the timeliness and update frequency of the data to ensure that the system can obtain the latest data. At the same time, it is also necessary to consider the cost and resource limitations of data collection, and reasonably arrange data collection tasks to ensure the stability and efficiency of system operation.

[0035] The method of installing a data collection plan is as follows: First, according to the data collection plan, set the time point, frequency and duration of data collection. Ensure that the system can obtain data on time and as needed, and take into account the timeliness and update frequency of the data. At the same time, it is necessary to evaluate and prepare the corresponding collection tools, equipment and resources to smoothly perform the data collection task. Special attention should also be paid to privacy and security issues.

[0036] It should be noted that protecting user privacy and data security is an important factor that must be considered during the data collection process. Protecting user privacy and data security specifically includes:

[0037] Assess privacy risks: Before formulating a collection plan, you first need to conduct a privacy risk assessment on the data types, sources, and processing methods involved. Understand whether the collected data involves sensitive information, and the risk of data leakage that may exist during the collection process. Develop privacy protection measures: Based on the results of the privacy risk assessment, formulate corresponding privacy protection measures. This includes using encryption technology to protect data, taking data desensitization or anonymization measures, and restricting data access rights. Comply with laws and regulations: Ensure that the formulation and implementation of the data collection plan complies with relevant laws and regulations, especially data collection involving user personal information must comply with privacy protection laws. Conduct privacy risk assessment and monitoring: Regularly conduct privacy risk assessment and monitoring of the data collection process to promptly identify and resolve potential privacy and security issues and ensure the security and privacy of user data.

[0038] S120 collects data based on the formulated data collection plan and selects a data processing model that matches the collected data.

[0039] In specific practice, methods for selecting a data processing model that matches the collected data include:

[0040] Perform data preprocessing on the collected data to convert the collected data into structured data;

[0041] Clean and remove duplicate structured data.

[0042] Specifically, the method for cleaning and deduplicating structured data includes: processing missing values, outliers and duplicate data of structured data; wherein, the method for processing missing values ​​includes deleting missing data, filling missing values ​​and filling using interpolation methods; the method for processing outliers includes identifying and processing using statistical methods or methods based on business logic; the method for processing duplicate data includes identifying duplicate data and deleting or merging the duplicate data.

[0043] A data repository is established to store the collected data metadata and data documents of the cleaned and deduplicated structured data.

[0044] It can be understood that the metadata includes the name, description, creator, field type and field length of the data set, and the metadata is used to describe the data;

[0045] It can be understood that the data document includes a data dictionary, a data collection log, and a data cleaning record, and the data document is used to record key information in the data collection and processing process.

[0046] In specific practice, for structured data sources (such as database tables), use appropriate tools or database query languages ​​to extract the required information from the data source. Ensure that the format of the collected data conforms to the data model of the system, and perform necessary data cleaning and preprocessing operations. Then, for semi-structured data, use appropriate parsers or conversion tools to convert it into a structured format that can be used by the system. This may involve operations such as XML parsing and JSON parsing to ensure that the data can be effectively processed and utilized by the system. Next, for unstructured data, use appropriate processing tools, such as natural language processing (NLP) technology, image processing tools, etc., to extract and convert the data into structured data that can be used by the system. This step usually includes operations such as data feature extraction, text segmentation, and image feature extraction to convert unstructured data into a form that can be analyzed and utilized by the system. Finally, perform data integration and consolidation to integrate data from different sources and formats into the system and ensure the consistency and integrity of the data. In this way, the system can obtain and process data from multiple data sources, providing strong support for subsequent analysis and application.

[0047] It should be noted that data cleaning is one of the key steps in data preprocessing, including processing missing values, outliers, and duplicate data. When processing missing values, you can choose to delete missing data, fill missing values, or use interpolation methods to fill in to ensure data integrity and accuracy. Outliers can be identified and processed using statistical methods or methods based on business logic. Duplicate data needs to be identified and deleted or merged to avoid introducing data redundancy and affecting the accuracy of analysis results.

[0048] When building a data storage system, you need to choose a suitable database or data storage system based on factors such as data volume, access frequency, and storage requirements. Common choices include relational databases (such as MySQL, PostgreSQL), NoSQL databases (such as MongoDB, Redis), and distributed storage systems (such as Hadoop, HBase). Choosing a suitable data storage system can ensure the accessibility and availability of data, while considering data security and backup mechanisms.

[0049] Establishing a data collection monitoring system helps to timely discover and resolve errors or abnormalities in the data collection process. The monitoring system can regularly check the execution of data collection tasks, monitor data collection speed, data integrity, error logs and other information. Once an abnormality is found, the monitoring system can promptly issue an alarm and take appropriate countermeasures to ensure the smooth progress of data collection.

[0050] Establishing data documentation and metadata helps record important information such as the source of data, collection time, field meaning, and data quality. Data documentation can include data dictionaries, data collection logs, data cleaning records, etc., which are used to record key information in the data collection and processing process. Metadata is data that describes data, including information such as the data set name, description, creator, field type and length, which is used to help team members understand and use data and ensure the comprehensibility and maintainability of data.

[0051] Perform data analysis on structured data, determine data relevance, and determine the data processing model based on data relevance and model performance.

[0052] In specific practice, by comprehensively considering factors such as tasks, data, model performance, resource consumption, and flexibly using deep learning models, we can select the model that best suits a specific scenario, thereby providing powerful learning capabilities for intelligent recommendation and scenario management systems. This can be summarized as model selection and model optimization.

[0053] In practice, model selection includes:

[0054] Data preprocessing: When determining the task type, you need to consider the characteristics of the data. For example, a recommendation system may involve processing user behavior data, while image classification requires considering the size and format of image data. Different preprocessing techniques may be required for different task types, such as word segmentation and standardization of text data.

[0055] Data analysis: Data exploratory analysis can include statistical description, visualization and other methods to better understand the distribution and structure of the data. By exploring the correlation of the data, it can be determined whether feature engineering is needed or a more complex model is selected to handle the complex relationships of the data.

[0056] Model complexity considerations: For simple tasks such as image classification, you can choose a lightweight model to reduce computational costs and resource consumption. For tasks involving multimodal feature information or complex relationships, you may need to use a deeper model, such as BERT.

[0057] Model performance and resource consumption evaluation: When evaluating model performance, in addition to considering accuracy, you should also pay attention to the model's inference speed and memory usage, especially in real-time recommendation systems. For scenarios with limited resources, you can use model pruning, quantization and other technologies to reduce the number of model parameters and computing costs.

[0058] Utilization of prior knowledge: By leveraging domain expertise, appropriate pre-trained models can be selected and fine-tuned to suit the needs of specific tasks. The selection of pre-trained models should consider the performance and applicability of the model in the relevant field, as well as whether it has interpretability and other characteristics.

[0059] Application of transfer learning: When the data set is small or the domain is similar, you can consider using transfer learning to accelerate the model training process and reduce the cost of data annotation and training time. Choose appropriate transfer learning strategies, such as feature extraction and fine-tuning, to maximize the use of the knowledge of the source task to improve the performance of the target task.

[0060] In practice, model optimization includes:

[0061] Model interpretability: For scenarios where model decisions need to be explained, you can choose an interpretable model structure, such as a decision tree, linear model, etc. Model interpretability techniques, such as local interpretability methods and global interpretability methods, can be used to explain the model's prediction results and decision-making process.

[0062] Hyperparameter tuning: Use cross-validation and other techniques to search for the best hyperparameter combination to achieve the best performance of the model. You can use automated parameter tuning tools or hyperparameter optimization algorithms such as grid search, random search, Bayesian optimization, etc.

[0063] Continuous monitoring and updating: Establish a monitoring system to monitor the performance and behavior of the model in the production environment, and promptly detect model performance degradation or abnormalities. Based on the monitoring results, regularly update and retrain the model to adapt to changes in data distribution and new business needs, and maintain the robustness and effectiveness of the model.

[0064] S130 processes the collected data to form a feature fusion layer, and introduces an attention mechanism and an information transfer mechanism between channels into the selected data processing model to optimize the selected data processing model.

[0065] In specific practice, the method for processing the collected data includes convolution, pooling, multi-scale feature extraction of the collected data, and splicing or superimposing feature maps from different convolutional layers.

[0066] In specific practice, the method for optimizing the selected data processing model also includes introducing regularization technology and activation function into the selected data processing model to enhance the performance of the data processing model.

[0067] It should be noted that in deep learning, feature fusion is a key step to integrate features from different sources or levels to improve model performance. Here we use the method of deep learning. Compared with traditional methods, the feature fusion method we proposed has significant advantages in processing multimodal data. By introducing convolution, pooling and multi-scale feature extraction, combined with attention mechanism, inter-channel information transfer, regularization technology and activation function, the effective fusion and extraction of multimodal features are achieved, so that the model can more comprehensively understand and utilize the diversity and complexity of data, thereby improving the performance and robustness of the model. In particular, through methods such as attention mechanism and further processing, the model can focus more on the key areas of the task, while enhancing the generalization ability and stability of the model, so it has better adaptability and performance when processing multimodal data.

[0068] The feature extraction of the output of the convolutional layer involves using a convolutional neural network (CNN) to perform a convolution operation on the input data to extract feature maps at different levels. Each feature map captures the characteristics of the input data at a different level of abstraction.

[0069] The use of pooling layers includes after convolutional layers, and pooling layers are usually used for downsampling. The maximum pooling or average pooling operation merges local areas in the feature map into one value, thereby reducing the spatial dimension of the feature map.

[0070] Multi-scale convolutional feature extraction involves introducing convolution kernels of different sizes to obtain multi-scale feature representations. This can be achieved by using convolution layers with different convolution kernel sizes in parallel.

[0071] The concatenation or stacking of feature maps involves concatenating or stacking feature maps from different convolutional layers. This can be done by concatenating along the channel dimension or using element-wise addition.

[0072] The introduction of attention mechanism involves using attention mechanism to enable the model to focus more on the areas that are more important for the task by giving greater weight to specific features. This can be achieved by introducing attention layer or attention mechanism module.

[0073] Information transfer between channels includes the introduction of information transfer mechanisms between channels, such as channel attention mechanisms, convolution operations between channels, etc. This helps to improve the feature integration effect between different channels.

[0074] Feature map re-pooling or global pooling includes re-pooling or global pooling operations on the integrated feature map. Global pooling combines the information of the entire feature map into one value, which helps to reduce the dimension of the feature map.

[0075] The design of the feature fusion layer includes designing a feature fusion layer to fuse the pooled feature maps. This can be a fully connected layer, element-by-element multiplication, element-by-element addition, and other operations.

[0076] The use of regularization and activation functions includes the introduction of regularization techniques such as Batch Normalization and activation functions such as ReLU after the feature fusion layer to enhance model performance.

[0077] Further processing after feature fusion includes introducing other levels of processing after feature fusion, such as Dropout layer, residual connection, etc., depending on the nature of the task.

[0078] The output of the final feature representation includes taking the fused feature representation as input, connecting it to the subsequent fully connected layer or output layer, and producing the final model output.

[0079] S140 trains the optimized data processing model to obtain the attention mechanism weight matrix, and adjusts the obtained attention weight matrix to achieve personalized recommendation.

[0080] In specific practice, the method for realizing personalized recommendation includes: analyzing the obtained attention weight matrix, and obtaining the decision basis of the data processing model based on the analysis results; adjusting the attention weight matrix according to user behavior feedback to obtain the attention weight priority table; determining the processing order of the optimized data according to the obtained attention weight matrix priority table to realize personalized recommendation.

[0081] In practice, the key steps of personalized recommendation systems and system recommendations include obtaining the attention weight matrix, conducting interpretable analysis, understanding user interests, implementing personalized recommendations, optimizing system recommendation strategies, user participation and feedback, and evaluation iterations. By analyzing the attention matrix, we can gain an in-depth understanding of the decision-making basis of the model, thereby achieving an understanding of user interests. In personalized recommendations, by adjusting the attention weights, we can better adapt to user needs and improve the personalization of recommendations. Optimizing system recommendation strategies and establishing user participation mechanisms can help improve the flexibility of the system and user experience. Through continuous evaluation and iteration, we can continuously optimize the recommendation effect and achieve a more intelligent, transparent, and user-satisfied recommendation system.

[0082] It can be understood that the method of obtaining the attention weight matrix is ​​that when training the model, the attention mechanism generates an attention matrix that records the degree of attention the model pays to the input data in a specific task. For each combination of user and item, the matrix reflects the degree of attention the model pays to each feature when predicting the user's interest in the item.

[0083] Interpretable analysis includes interpretable analysis of the attention weight matrix to understand the decision basis of the model. Visualization and other means can be used to show the degree of attention paid by the model to the input data and the importance of each feature.

[0084] Understanding user interests involves understanding how the model captures user interests based on the results of interpretable analysis. Analyzing the attention matrix to find out the specific items or features that the model focuses on in the user's historical behavior, thereby understanding the user's preferences and interests.

[0085] The method of personalized recommendation is to use attention weights to adjust the model's attention to different features based on the user's historical behavior and feedback. By increasing the weight of the features that the user is interested in, personalized recommendations can be achieved and the user's satisfaction with the recommendation results can be improved.

[0086] The method of optimizing the system recommendation strategy is to optimize the system recommendation strategy based on the results of the explanatory analysis. According to the information of the attention matrix, the system's integration method of different features is adjusted to make the system more flexible to adapt to different users and different scenarios.

[0087] The method of user participation and feedback is to establish an interactive mechanism between users and the system based on explainability, so that users can participate in the recommendation process. The attention weight can be dynamically adjusted according to user feedback to achieve personalized control of the recommendation results.

[0088] It should be noted that the evaluation and iteration method is to evaluate the recommendation effect based on user feedback and system performance. In actual applications, the model is continuously iterated based on feedback information to optimize the recommendation effect and user satisfaction.

[0089] This embodiment provides a computer-readable storage medium, in which computer-readable instructions are stored. When the computer-readable instructions are executed on a communication device, the above method is executed.

[0090] The present embodiment provides a scene management and intelligent recommendation system based on data space, including: a plan-making unit, which is used to select system functions in a preset database, match corresponding data types according to the selected system functions, and formulate corresponding data collection plans according to the system functions and data types; a data collection unit, which is used to collect data based on the formulated data collection plan and select a data processing model that matches the collected data according to the collected data; a data processing unit, which is used to process the collected data to form a feature fusion layer, and introduce an attention mechanism and an information transmission mechanism between channels into the selected data processing model to optimize the selected data processing model; a data output unit, which is used to train the optimized data processing model to obtain an attention mechanism weight matrix, and adjust the obtained attention weight matrix to achieve personalized recommendation.

[0091] Embodiment 2:

[0092] like Figure 2 A scene management and intelligent recommendation method based on data space shown includes:

[0093] S210 multi-modal data acquisition, cleaning and storage.

[0094] In practice, the first step in data collection in the data space is to clarify the collection goals. This includes clarifying business or research needs, defining the type of information to be collected, determining the spatial and temporal scope of the collection, and evaluating the feasibility of the data. Based on this information, a detailed data collection plan is developed, including the start time, end time, collection frequency, and the required collection tools. Key indicators and metrics should also be clearly defined to assess data quality. In addition, privacy and compliance need to be taken into account to ensure that the collection process complies with relevant regulations and ethical standards. Finally, in order to respond to business or technological changes, it is necessary to establish a change management plan to ensure the continued adaptability of the collection goals. This series of steps ensures that the direction and goals of the entire data collection process are clear at the beginning of data collection.

[0095] In practice, the sensor data collection process includes: First, the collection target needs to be clearly defined, such as monitoring ambient temperature and humidity. According to the collection target, select the appropriate sensor type, such as temperature sensor, humidity sensor, etc. Then, arrange the sensors reasonably in the collection area to ensure that their coverage and density meet the collection requirements. Once the sensors are arranged, start the data collection equipment and start collecting sensor data in real time or periodically. Finally, conduct real-time monitoring and quality control to ensure the accuracy and integrity of the data.

[0096] The process of collecting user behavior data includes: First, you need to clarify business needs, such as understanding user behavior when using the product. Design a data collection plan, including collection frequency, time range, etc. According to the plan, select the appropriate data source, which may involve applications, website analysis tools, etc. During the data collection process, implement real-time monitoring and quality control to ensure that the collected user behavior data meets expectations.

[0097] The image data acquisition process includes: Image data acquisition involves shooting, acquiring or collecting images. First, determine the target of image acquisition, such as the appearance of a building, natural scenery, etc. Select appropriate image acquisition equipment, such as cameras, satellite images, etc. Design an acquisition plan, including parameters such as shooting angle and resolution. Implement image acquisition and store the acquired image data. For large-scale image data, image processing technology may be required for quality control and feature extraction.

[0098] The text data collection process includes: Text data collection involves obtaining text information from sources such as documents, web pages, and social media. First, clarify the goal of text collection, such as obtaining news reports in a specific field. Select appropriate text collection tools, such as web crawlers, API calls, etc. Design a collection plan, including keyword screening, time range, etc. Perform text data collection and store the collected text information. For large-scale text data, text cleaning, standardization, and analysis may be required.

[0099] The audio data collection process includes: Audio data collection involves recording, acquiring or transmitting sound information. First, clarify the goal of audio collection, such as music creation, speech recognition, etc. Select appropriate audio collection equipment, such as microphones, recording equipment, etc. Design a collection plan, including parameters such as recording environment and sampling rate. Perform audio data collection and store the collected audio information. For large-scale audio data, audio processing, feature extraction and analysis may be required.

[0100] The data cleaning solution covers data quality assessment, missing value processing, outlier processing, data standardization and normalization, data deduplication, text data cleaning, and data validation. First, a data quality assessment is performed to check the completeness, accuracy, and consistency of the data, and to handle outliers, missing values, and duplicate values. Subsequently, missing values ​​are processed by interpolation, deletion, or filling, outliers are identified and processed, data is standardized and normalized, duplicate data is removed, text data is cleaned, and finally the logical consistency of the data is verified to ensure compliance with business rules and logic.

[0101] In the data preservation plan, select appropriate data storage media, such as relational databases, NoSQL databases, cloud storage, etc., according to the data scale and access requirements. Design the database structure, including table fields, indexes, etc., to improve data retrieval efficiency. Implement regular backup strategies to ensure data security and store backups in different locations. Take data security measures, including encryption, access control, and identity authentication, to protect sensitive data. Establish a metadata management mechanism to record key information such as the source of data, processing steps, and quality assessment to support subsequent data use and maintenance. Implement version control to track historical changes in data to support data backtracking and auditing. These plans can be adjusted according to specific scenarios and data characteristics to ensure that the data cleaning and preservation process meets business needs and data management standards.

[0102] Ultimately we divide data into two categories: structured data and unstructured data.

[0103] S220 feature extraction.

[0104] In practice, for structured data, it is first encoded into a feature matrix. Structured data usually exists in a table, where each row represents a sample and each column represents a feature. These features can be numerical, categorical, or other types. The goal of encoding is to convert these features into a matrix that can be processed by a deep learning model. Usually, numerical features can be directly used as part of the matrix, while categorical features may need to be converted to numerical types through operations such as one-hot encoding.

[0105] Next, we use a convolutional neural network (CNN) to process the feature matrix X of structured data. p . CNN is usually designed to process image data, but it can also be applied to one-dimensional structured data. In this context, the convolution kernel of CNN will slide on the feature matrix to extract local features. The convolution layer may be followed by a pooling layer to reduce the data dimension and extract more abstract features. Finally, through structures such as fully connected layers, the extracted features are mapped to the output layer for classification or regression. Each CNN layer encoder is described as follows:

[0106]

[0107] in and are the learnable weight filters (matrix) and bias vector in the CNN layer.

[0108] A common way to process unstructured data is to use a graph convolutional network (GCN). Unstructured data usually exists in the form of a graph, where nodes represent elements of the data and edges represent the relationship between elements. GCN can perform convolution operations on the graph and effectively capture the relationship between nodes and their neighbors.

[0109] First, a graph representation of the data is constructed, where nodes represent individual elements of the data (such as words in text, pixels in images), and edges represent the associations between elements. Then, the graph is convolved using GCN, and information about nodes and their neighbors is used to extract and learn features. GCN's convolution operation can be stacked in multiple layers to extract features at different levels. Finally, the extracted features are mapped to the output layer through operations such as pooling to achieve classification, clustering, or other tasks. The specific steps are as follows:

[0110] We can denote the ReLU activation function as σ and use and They represent the adjacency matrix and diagonal matrix with self-connection added respectively. represents the weight matrix. Therefore, if it is passed through the (l+1)th layer, it can be encoded as a feature matrix:

[0111]

[0112] In addition, the characteristics of the (l+1)th node in each layer can be expressed as:

[0113]

[0114] In this context, specifying a node and its entire set of neighboring nodes N i . C ij Serves as a normalization factor. W (l) refers to the weight associated with layer l, b (l) Corresponds to the intercept of the same layer.

[0115] Through the above steps, we can obtain rich feature information. These feature vectors are various types of information in the data space. We use the above methods to convert them into a form that can be recognized and learned by the deep learning model. For structured data, we encode it into a feature matrix and process it through CNN, so that the model can extract meaningful local and global features from it. For unstructured data, we construct a graph representation and use GCN for convolution operations, so that the model can effectively capture the relationship between elements.

[0116] In this process, we not only considered the original form of the data, but also performed data enhancement, feature engineering and other operations to improve the robustness and generalization ability of the model. The obtained feature vector reflects various types of information in the data space, including numerical and category information in structured data, and graph structure, text semantics and other information in unstructured data. Finally, the data in the two data forms are converted into feature matrices R through feature extraction. S and R K .

[0117] After being trained by a deep learning model, these feature vectors can be used for tasks such as recognition, classification, and regression. By learning the weights and associations of these features, the deep learning model can perform well on the training data and make predictions on new, unseen data. This process enables the flow of information from raw data to the deep learning model's understanding, providing powerful tools and methods for solving complex tasks. The above two feature matrices will be further processed and introduced into the next module to obtain the joint representation and attention matrix.

[0118] S230 feature fusion and scene management.

[0119] In practice, this module aims to study the interaction mechanism between each data and obtain their joint representation matrix and attention weight matrix. In our model, unstructured data features are extracted through graph structure, and structured features are obtained through sequence information. The interaction process involves multimodal feature fusion, so we adopted an unconventional approach. This approach is used at the intersection of computer vision and natural language processing, especially in tasks such as visual question answering (QVA), image captioning, and image retrieval. Considering similar challenges, we designed this module.

[0120] Since multiple linear transformations and transpositions are involved, various matrices will undergo multiple dimensional changes in the process. To enhance the clarity of our method, we can express it mathematically as follows.

[0121] We set the number of heads for parallel computing to h, where h must be divisible. i(1≤i≤h), we can define three weight matrices that will be used for linear transformation. The symbol · represents matrix multiplication. We can calculate the matrices Q, K, V: Q=W q ·R s , K=W k ·R K , V=W v ·R K , Here, R S and R K Represent the feature matrices we obtain through different data sources respectively. Different features are processed by a custom fully connected layer (FCNet) to map the original features to a new feature space respectively. The purpose of FCNet is to integrate and abstract the obtained features to achieve higher-level feature reconstruction. The interactive features are learned by the decoder of the transformer, which consists of a self-attention layer and a feedforward layer. In our work, the output of the decoder is an interaction sequence containing interactive features. The key technology is the multi-head self-attention layer. The multi-head self-attention layer consists of multiple scaled point attention layers to extract the interactive information between the encoder and the decoder. The self-attention layer accepts three inputs, key K, value V, and query Q, and calculates the attention as follows: , In the above formula, Indicates head i The dimension of the key vector in . It is used to scale the attention weights, which helps stabilize the training process. A softmax function is applied to normalize the scores to convert them into probability distributions. Next, we will concatenate the attention weight matrix H of each head: .

[0122] Q=W q ·R S ,

[0123] K=W k ·R K ,

[0124] V=W v ·R K .

[0125] Here, R S and R KRepresent the feature matrices we obtain through different data sources respectively. Different features are processed by a custom fully connected layer (FCNet) to map the original features to a new feature space respectively. The purpose of FCNet is to integrate and abstract the obtained features to achieve higher-level feature reconstruction. The interactive features are learned by the decoder of the transformer, which consists of a self-attention layer and a feedforward layer. In our work, the output of the decoder is an interaction sequence containing interactive features. The key technology is the multi-head self-attention layer. The multi-head self-attention layer consists of multiple scaled point attention layers to extract the interactive information between the encoder and the decoder. The self-attention layer accepts three inputs, key K, value V, and query Q, and calculates the attention as follows:

[0126]

[0127] In the above formula, Indicates head i The dimension of the key vector in . It is used to scale the attention weights, which helps stabilize the training process. A softmax function is applied to normalize the scores to convert them into probability distributions. Next, we will concatenate the attention weight matrix H of each head:

[0128] H=concat{attention(Q,K,V)}.

[0129] In general, the approach we take is committed to further strengthening the interactivity of the features we extract. And the unique mechanism of this module greatly facilitates the integration of various types of data we obtain in the data space. This involves embedding scene elements (such as positions, objects, states, etc.) as vector representations and inputting these representations into the model in a serialized manner. In this way, the complex relationships between elements in the scene can be learned and captured, thereby achieving management and understanding of the scene. This approach enables the adaptation of inputs containing rich scene information while data, providing powerful modeling capabilities for scene management tasks.

[0130] Finally, the joint representation is decoded through a multi-layer perceptron (MLP) to obtain the prediction result, that is, to determine whether there is a correlation between the two data sources. This step can be summarized as a binary classification problem, in which the goal of the model is to learn a discriminant function to predict the correlation between the two data sources.

[0131] Here are the general steps for this process:

[0132] Acquisition of joint representation: Through the aforementioned attention mechanism or other deep learning models, a joint representation of the two data sources is obtained. This joint representation captures the information and related features that the model considers important.

[0133] MLP decoding: The joint representation f is fed into a multi-layer perceptron (MLP). MLP is a deep learning model that consists of multiple fully connected layers and is capable of learning non-linear mappings. These layers gradually extract and combine features, ultimately outputting a prediction result of dimension 1.

[0134] Binary classification prediction: Through an appropriate activation function (such as the Sigmoid function), the output of the MLP is mapped to a range between 0 and 1, representing the probability of correlation between the two data sources. Usually, a threshold of 0.5 can be used to determine whether the two data sources are related. A probability exceeding 0.5 indicates that the two are related, and vice versa. This process can be expressed by the following formula:

[0135] Let the input layer be X = {x1, x2, ..., x n}, x i are the i-th elements of the input vector. Suppose the hidden layer l contains m neurons;

[0136] Assume that the output layer contains mL+1 neurons;

[0137] Suppose the weight matrix between the lth layer and the l+1th layer is W(l), and the bias of the lth layer is b(l); then the input of the l+1th layer is: z(l+1)=W(l)a(l)+b(l) where a(l) is the output of the lth layer, that is, the output of the activation function;

[0138] Activation functions usually use nonlinear functions, common ones include Sigmoid, ReLU, tanh, etc.

[0139] The output of the output layer can use different activation functions according to the specific task, such as:

[0140] For binary classification problems, the Sigmoid activation function can be used:

[0141]

[0142] For multi-classification problems, the Softmax activation function can be used:

[0143]

[0144] in, ej z(L+1) is the input of the jth output neuron. During the training process, the backpropagation algorithm is generally used to optimize the network parameters, such as weights and biases, to minimize the loss function.

[0145] Model training: Using labeled training data, the weights and parameters of the MLP are trained by minimizing a loss function such as binary cross entropy. This enables the model to learn the correlation patterns between data sources on the training data.

[0146] Prediction and Application: After training, the model can be used for new data sources to predict their correlation with other data sources. This prediction result can be used for decision-making, recommendation, or other intelligent applications, depending on the context of the problem. Through this MLP decoding process, the system can effectively learn and predict the relationship between two data sources, thereby providing more intelligent decision-making or recommendation services for your application.

[0147] S240 intelligent recommendation.

[0148] In practice, we extract the attention matrix obtained by the deep learning model through the above method and save it in the form of an array. Each value in this attention matrix represents the degree of attention paid by the model to different parts of the input data, or represents the correlation score between different parts.

[0149] First, for the attention matrix H, it can be regarded as a matrix in which each row corresponds to the model's attention weight for a position in the input sequence. The weight of each position indicates the degree of attention the model pays to the information at that position. In this way, we can flatten this matrix into an array to obtain a one-dimensional array of attention weights.

[0150] Next, through these attention weights, we can make personalized intelligent recommendations for the input data. Suppose we have multiple data sources, such as user historical behavior, preference information, real-time feedback, etc., each source corresponds to a part of the input data. Through attention weights, we can give different weights to data from different sources when integrating various data to highlight the information that the model considers important.

[0151] In the scenario of intelligent recommendation, if a certain weight in the attention matrix is ​​high, it means that the model believes that the user's historical behavior has a greater impact on the current recommendation. In this way, when making personalized recommendations, the weight of the historical behavior data will be increased, thereby emphasizing this part of information. If the weights of other parts are low, the model will reduce the influence of these sources. In this way, through attention weights, we can more flexibly consider and utilize data from different sources in the process of personalized recommendations, and improve the accuracy of recommendations and user satisfaction.

[0152] Suppose we are building a smart home system, where the user's health status and indoor environment are two important data sources. Through the attention matrix H obtained by the deep learning model, we can dynamically adjust the smart home system to better adapt to the user's health needs.

[0153] User health status: Our system can collect user health data, such as heart rate, sleep quality, exercise habits, etc. Through the deep learning model, we get the attention matrix H, in which each value represents the model's attention to different health indicators. If a certain indicator has a high weight in the attention matrix, it means that the model believes that this indicator has a greater impact on the user's health status.

[0154] Indoor environment: At the same time, we also collect data on the indoor environment, including temperature, humidity, air quality, etc. These data are also used in the deep learning model to obtain an attention matrix H about the indoor environment. The weights of different elements indicate the degree of attention the model pays to various factors in the indoor environment.

[0155] By combining these two attention matrices, we can achieve personalized smart home services:

[0156] If the weight of health indicators in the user's attention matrix is ​​high, the system may adjust the indoor environment based on these health indicators, such as adjusting the temperature and providing soft lighting, to promote the user's comfort and health.

[0157] On the contrary, if the weight of an item in the indoor environment's attention matrix is ​​high, the system may pay more attention to adjusting environmental factors, such as timely ventilation and adjusting the working status of the air purifier, to provide a more suitable indoor environment.

[0158] Through such personalized intelligent adjustments, the system can better meet the user's needs for health and comfort and improve the living experience. While paying attention to the user's health, this smart home system also fully considers the impact of the indoor environment on the user's quality of life.

[0159] In general, our model can achieve personalized intelligent recommendations. This technology uses users' historical data, preferences, and behavior patterns, and through intelligent algorithms and machine learning methods, provides users with customized suggestions and services to meet their personalized needs and improve user experience and satisfaction.

[0160] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0161] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0162] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0163] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0164] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A scene management and intelligent recommendation method based on data space, characterized in that: include: Select system functions in the preset database, match corresponding data types according to the selected system functions, and formulate corresponding data collection plans according to the system functions and data types; Collect data based on the formulated data collection plan and select a data processing model that matches the collected data; Process the collected data to form a feature fusion layer, and introduce an attention mechanism and an information transfer mechanism between channels into the selected data processing model to optimize the selected data processing model; The optimized data processing model is trained to obtain the attention mechanism weight matrix, and the obtained attention weight matrix is ​​adjusted to achieve personalized recommendation.

2. The scene management and intelligent recommendation method based on data space according to claim 1, characterized in that: The method of formulating a data collection plan includes: selecting a data source according to the data type, and determining the start time, frequency and duration of data collection according to system functions.

3. The scene management and intelligent recommendation method based on data space according to claim 1, characterized in that: Methods for selecting a data processing model that matches the collected data include: Perform data preprocessing on the collected data to convert the collected data into structured data; Clean and remove duplicate structured data; Establish a data repository to store the collected data metadata and the data documents of the cleaned and deduplicated structured data; Perform data analysis on structured data, determine data relevance, and determine the data processing model based on data relevance and model performance.

4. The scene management and intelligent recommendation method based on data space according to claim 3 is characterized in that: Methods for cleaning and deduplicating structured data include: processing missing values, outliers and duplicate data of structured data; among them, methods for processing missing values ​​include deleting missing data, filling missing values ​​and filling using interpolation methods; methods for processing outliers include identifying and processing using statistical methods or methods based on business logic; methods for processing duplicate data include identifying duplicate data and deleting or merging duplicate data.

5. The scene management and intelligent recommendation method based on data space according to claim 3 is characterized in that: The metadata includes a dataset name, description, creator, field type, and field length, and the metadata is used to describe the data; The data document includes a data dictionary, a data collection log, and a data cleaning record. The data document is used to record key information in the data collection and processing process.

6. The scene management and intelligent recommendation method based on data space according to claim 1 is characterized in that: The method for processing the collected data includes convolution, pooling, and multi-scale feature extraction on the collected data, and splicing or superimposing feature maps from different convolutional layers.

7. The scene management and intelligent recommendation method based on data space according to claim 1, characterized in that: The method for optimizing the selected data processing model also includes introducing regularization techniques and activation functions into the selected data processing model to enhance the performance of the data processing model.

8. The data processing method according to claim 1, characterized in that: Methods for implementing personalized recommendations include: Analyze the obtained attention weight matrix, and obtain the decision basis of the data processing model based on the analysis results; Adjust the attention weight matrix according to user behavior feedback to obtain an attention weight priority table; The processing order of the optimized data is determined according to the obtained attention weight matrix priority table to achieve personalized recommendation.

9. A computer-readable storage medium, characterized in that: The computer storage medium stores computer-readable instructions, and when the computer-readable instructions are executed on the communication device, the method according to any one of claims 1 to 8 is executed.

10. A scene management and intelligent recommendation system based on data space, characterized in that: include: A plan making unit, the plan making unit is used to select a system function in a preset database, match a corresponding data type according to the selected system function, and make a corresponding data collection plan according to the system function and the data type; A data collection unit, the data collection unit is used to collect data based on the formulated data collection plan and select a data processing model matching the collected data; A data processing unit, the data processing unit is used to process the collected data to form a feature fusion layer, and introduce an attention mechanism and an information transfer mechanism between channels into the selected data processing model to optimize the selected data processing model; A data output unit is used to train the optimized data processing model, obtain the attention mechanism weight matrix, and adjust the obtained attention weight matrix to achieve personalized recommendation.

Images