Smart home management method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a smart home management method and device, electronic equipment and storage medium, relating to the technical field of smart home, the method comprises: the edge device obtains the first device data of the smart home device, and analyzes and processes the first device data to obtain the first visualization data corresponding to the smart home device;The AR device receives the first visualization data sent by the edge device, displays the management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data, and acquires operation data corresponding to the control operation in response to the control operation for the smart home device;The edge device analyzes the operation data, generates the target control instruction for the smart home device, and sends the target control instruction to the smart home device;The smart home device executes the device operation corresponding to the target control instruction, thereby ensuring the control real-time and control efficiency of the smart home device, and reducing the control difficulty.

Description

Technical Field

[0001] This invention relates to the field of smart home technology, and in particular to a smart home management method, a smart home management device, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the gradual development of the Internet of Things (IoT), the existing simple smart operations can no longer meet people's needs. People are increasingly looking forward to and yearning for a better IoT-enabled life. Currently, the interaction methods in smart homes are relatively simple, usually operated through terminal applications, speakers, and other smart terminals. The management of smart home devices often involves cloud computing, which can cause latency, affecting the real-time performance of device control. Furthermore, when cloud computing malfunctions, it can easily lead to unresponsive device control. In addition, the complex user interfaces of smart home devices make them difficult for users to learn, impacting control efficiency. Summary of the Invention

[0003] The present invention provides a management method, device, electronic device, and computer-readable storage medium for smart homes, in order to solve or partially solve the problems of poor real-time performance, high operational difficulty, and low control efficiency in the management and control of smart home devices.

[0004] This invention discloses a smart home management method applied to a smart home management system. The smart home management system includes at least an AR device, an edge device, and smart home devices. The AR device provides a graphical user interface, and the content displayed by the graphical user interface includes at least the augmented reality scene corresponding to the smart home devices. The method includes:

[0005] The edge device acquires the first device data of the smart home device and analyzes and processes the first device data to obtain the first visual data corresponding to the smart home device.

[0006] The AR device receives first visualization data sent by the edge device, and displays the management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data;

[0007] The AR device responds to the user's input of control operations on the smart home device and obtains the operation data corresponding to the control operations;

[0008] The edge device analyzes the operation data, generates a target control command for the smart home device, and sends the target control command to the smart home device;

[0009] The smart home device performs the device operation corresponding to the target control command.

[0010] In some feasible implementations, the step of analyzing and processing the data of the first device to obtain the first visualized data corresponding to the smart home device includes:

[0011] The first device data is cleaned to remove invalid, redundant, and abnormal data, thereby obtaining cleaned first device data.

[0012] The status data of the smart home device is obtained by fusing the data from the first device after cleaning.

[0013] The status data corresponding to multiple smart home devices are aggregated to generate target data associated with specific management functions, and the target data is visualized to obtain the corresponding first visualized data.

[0014] In some feasible implementations, the aggregation of status data corresponding to multiple smart home devices to generate target data associated with a specific management function includes:

[0015] The first status data corresponding to the first smart home devices belonging to the same device type are aggregated to generate device overview data corresponding to the device type.

[0016] And / or, aggregate the second state data corresponding to the second smart home devices belonging to the same environmental space to generate device overview data corresponding to the environmental space.

[0017] In some feasible implementations, the visualization transformation of the target data to obtain the corresponding first visualization data includes:

[0018] The data type of the target data is obtained, and the data type includes at least one of the following: time series type, classification type, relation type, geographic location type, multidimensional type, and trend prediction type.

[0019] The target data is visualized according to one of the time series type, the classification type, the relationship type, the geographic location type, the multidimensional type, and the trend prediction type to obtain the corresponding first visualized data;

[0020] The first visualization data includes at least one of the following: bar chart, pie chart, bar graph, network diagram, tree diagram, matrix diagram, map, heat map, bubble chart, parallel coordinate diagram, radar chart, scatter matrix diagram, line chart, prediction interval diagram, and error bar chart.

[0021] In some feasible implementations, displaying the management interface corresponding to the smart home device in the augmented reality scene based on the first visualization data includes:

[0022] Obtain the device type corresponding to the smart home device;

[0023] Determine the management interface layout corresponding to the device type, and fill the management interface layout with the first visualization data to construct the management interface corresponding to the smart home device;

[0024] Obtain the user's target location and browsing perspective in the smart home scene;

[0025] The system locates the augmented reality scene corresponding to the target location within the smart home scene, displays the augmented reality scene according to the browsing perspective, and overlays the management interface onto the displayed augmented reality scene.

[0026] In some feasible implementations, the step of analyzing the operational data and generating target control commands for the smart home device includes:

[0027] The operation type, operation parameters, and device identifier extracted from the operation data;

[0028] Obtain the device target control command corresponding to the device identifier;

[0029] The operation type and the operation parameters are matched with preset control instructions to determine the target control instruction corresponding to the operation data.

[0030] Among some feasible implementation methods are:

[0031] The edge device collects second device data after the smart home device completes the device operation, analyzes and processes the second device data, and obtains second visual data corresponding to the smart home device.

[0032] The AR device receives second visualization data sent by the edge device and updates the management interface displayed in the augmented reality scene in real time based on the second visualization data.

[0033] This invention also discloses a smart home management device applied to a smart home management system. The smart home management system includes at least an AR device, an edge device, and smart home devices. The AR device provides a graphical user interface, and the content displayed by the graphical user interface includes at least the augmented reality scene corresponding to the smart home devices. The device includes:

[0034] The data acquisition module located at the edge device is used to acquire first device data of the smart home device, and analyze and process the first device data to obtain first visual data corresponding to the smart home device;

[0035] The interface display module located in the AR device is used to receive the first visualization data sent by the edge device, and display the management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data;

[0036] The operation response module located in the AR device is used to respond to the user's input of control operations on the smart home device and to obtain operation data corresponding to the control operations;

[0037] The instruction generation module located in the edge device is used to analyze the operation data, generate target control instructions for the smart home device, and send the target control instructions to the smart home device;

[0038] The instruction execution module located in the smart home device is used to execute device operations corresponding to the target control instruction.

[0039] In some feasible implementations, the data acquisition module is specifically used for:

[0040] The first device data is cleaned to remove invalid, redundant, and abnormal data, thereby obtaining cleaned first device data.

[0041] The status data of the smart home device is obtained by fusing the data from the first device after cleaning.

[0042] The status data corresponding to multiple smart home devices are aggregated to generate target data associated with specific management functions, and the target data is visualized to obtain the corresponding first visualized data.

[0043] In some feasible implementations, the data acquisition module is specifically used for:

[0044] The first status data corresponding to the first smart home devices belonging to the same device type are aggregated to generate device overview data corresponding to the device type.

[0045] And / or, aggregate the second state data corresponding to the second smart home devices belonging to the same environmental space to generate device overview data corresponding to the environmental space.

[0046] In some feasible implementations, the data acquisition module is specifically used for:

[0047] The data type of the target data is obtained, and the data type includes at least one of the following: time series type, classification type, relation type, geographic location type, multidimensional type, and trend prediction type.

[0048] The target data is visualized according to one of the time series type, the classification type, the relationship type, the geographic location type, the multidimensional type, and the trend prediction type to obtain the corresponding first visualized data;

[0049] The first visualization data includes at least one of the following: bar chart, pie chart, bar graph, network diagram, tree diagram, matrix diagram, map, heat map, bubble chart, parallel coordinate diagram, radar chart, scatter matrix diagram, line chart, prediction interval diagram, and error bar chart.

[0050] In some feasible implementations, the interface display module is specifically used for:

[0051] Obtain the device type corresponding to the smart home device;

[0052] Determine the management interface layout corresponding to the device type, and fill the management interface layout with the first visualization data to construct the management interface corresponding to the smart home device;

[0053] Obtain the user's target location and browsing perspective in the smart home scene;

[0054] The system locates the augmented reality scene corresponding to the target location within the smart home scene, displays the augmented reality scene according to the browsing perspective, and overlays the management interface onto the displayed augmented reality scene.

[0055] In some feasible implementations, the instruction generation module is specifically used for:

[0056] The operation type, operation parameters, and device identifier extracted from the operation data;

[0057] Obtain the device target control command corresponding to the device identifier;

[0058] The operation type and the operation parameters are matched with preset control instructions to determine the target control instruction corresponding to the operation data.

[0059] Among some feasible implementation methods are:

[0060] The data acquisition module located at the edge device is used to collect second device data after the smart home device completes the device operation, analyze and process the second device data, and obtain second visual data corresponding to the smart home device.

[0061] The interface update module located in the AR device is used to receive the second visualization data sent by the edge device and update the management interface displayed in the augmented reality scene in real time according to the second visualization data.

[0062] This invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0063] The memory is used to store computer programs;

[0064] When the processor executes a program stored in the memory, it implements the method described in the embodiments of the present invention.

[0065] This invention also discloses a computer-readable storage medium storing instructions that, when executed by one or more processors, cause the processors to perform the methods described in this invention.

[0066] The embodiments of the present invention have the following advantages:

[0067] In this embodiment of the invention, an application is made to a smart home management system. The smart home management system includes at least an AR device, an edge device, and smart home devices. The AR device provides a graphical user interface (GUI), and the content displayed in the GUI includes at least the augmented reality scene corresponding to the smart home device. During the management and control of the smart home devices, the edge device acquires first device data of the smart home devices and analyzes and processes the first device data to obtain first visual data corresponding to the smart home devices. The AR device receives the first visual data sent by the edge device and displays the management interface corresponding to the smart home devices in the augmented reality scene based on the first visual data. If the user inputs a corresponding control operation, the AR device... The device can respond to user input of control operations for smart home devices, acquire the corresponding operation data, then the edge device analyzes the operation data, generates target control commands for the smart home devices, and sends the target control commands to the smart home devices. Finally, the smart home devices execute the device operations corresponding to the target control commands. Thus, the device data of smart home devices is processed and analyzed based on the edge device. Compared with centralized computing, this improves the real-time performance of data processing. At the same time, the interface display and interaction based on AI devices effectively reduces the difficulty of controlling smart home devices, providing users with a more intuitive and convenient interaction method and improving the ease of user control over smart home devices. Attached Figure Description

[0068] Figure 1This is a flowchart illustrating the steps of a smart home management method provided in an embodiment of the present invention;

[0069] Figure 2 This is a structural block diagram of a smart home management device provided in an embodiment of the present invention. Detailed Implementation

[0070] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0071] As an example, with the gradual development of the Internet of Things (IoT), existing simple smart operations can no longer meet people's needs. People increasingly expect and yearn for a better IoT-enabled lifestyle. Currently, smart home interaction methods are relatively simple, usually operated through mobile applications, speakers, and other smart terminals. With the development of AR technology, AR technology can provide a more intuitive display interface and operation. Currently, smart homes use traditional centralized computing. Compared to traditional centralized computing, edge computing has advantages such as stronger real-time performance, reduced unnecessary data transmission, and the ability for edge nodes to respond urgently in case of central server failure.

[0072] In this invention, an application is made to a smart home management system. The smart home management system includes at least an AR device, an edge device, and smart home devices. The AR device provides a graphical user interface (GUI), and the GUI displays at least the augmented reality scene corresponding to the smart home device. During the management and control of the smart home devices, the edge device acquires first device data from the smart home devices and analyzes and processes this data to obtain first visual data corresponding to the smart home devices. The AR device receives the first visual data sent by the edge device and displays the management interface corresponding to the smart home devices in the augmented reality scene based on the first visual data. If the user inputs a corresponding control operation, the AR device... The device can respond to user input of control operations for smart home devices, acquire the corresponding operation data, then the edge device analyzes the operation data, generates target control commands for the smart home devices, and sends the target control commands to the smart home devices. Finally, the smart home devices execute the device operations corresponding to the target control commands. Thus, the device data of smart home devices is processed and analyzed based on the edge device. Compared with centralized computing, this improves the real-time performance of data processing. At the same time, the interface display and interaction based on AI devices effectively reduces the difficulty of controlling smart home devices, providing users with a more intuitive and convenient interaction method and improving the ease of user control over smart home devices.

[0073] Reference Figure 1This diagram illustrates a flowchart of a smart home management method provided in an embodiment of the present invention. The method is applied to a smart home management system, which includes at least an AR device, an edge device, and smart home devices. The AR device provides a graphical user interface (GUI), and the content displayed on the GUI includes at least the augmented reality scene corresponding to the smart home device. Specifically, the method may include the following steps:

[0074] Step 101: The edge device acquires the first device data of the smart home device and analyzes and processes the first device data to obtain the first visual data corresponding to the smart home device.

[0075] In this embodiment of the invention, the smart home management system may include AR devices, edge devices, and smart home devices.

[0076] Among them, AR devices can include AR glasses or smartphones for displaying virtual information and controlling smart home devices; edge devices can include edge gateways, for example, edge gateways can be used to handle data preprocessing, local decision-making and real-time response; smart home devices can include sensors and actuators, sensors can include temperature sensors, humidity sensors, human body sensors, etc., for collecting environmental data; actuators can include smart light bulbs, smart sockets, smart door locks, etc., for executing control commands.

[0077] In managing smart home devices, sensors collect environmental data and transmit it to edge devices via a local area network, while AR devices collect user input and control commands. Edge devices preprocess the data and make local decisions, such as automatically adjusting the air conditioner temperature based on temperature sensor data. For more complex processing, edge devices upload the data to the cloud. Edge devices control actuators based on local decisions or cloud commands, such as turning on smart light bulbs or unlocking smart door locks. AR devices and mobile devices display feedback information, allowing users to view and adjust settings. The cloud data center stores historical data for data analysis and machine learning model training. AR devices can provide corresponding user interfaces so users can view device status, historical records, and analysis reports.

[0078] In some feasible implementations, after the edge device obtains the first device data corresponding to the smart home device, it can perform data cleaning on the first device data to remove invalid, redundant, and abnormal data, thereby obtaining cleaned first device data. The cleaned first device data is then merged to obtain the status data of the smart home device. Then, the status data corresponding to multiple smart home devices are aggregated to generate target data associated with specific management functions, and the target data is visualized to obtain the corresponding first visualized data.

[0079] It should be noted that, for the preprocessing of the first device's data, the edge device can remove outliers, redundant data, and invalid data, and compress the data to reduce the amount of data and improve transmission efficiency. At the same time, the noise-reduced and compressed data is converted into the same format for subsequent processing.

[0080] For example, assuming the first device data acquired by the edge device includes the status of a smart bulb (on / off, brightness), the power consumption of a smart socket, and the temperature and humidity of a smart thermostat, the first device data can be filtered. For example, the temperature sensor may occasionally return abnormally high temperature values, and these abnormal data can be removed using threshold filtering. Then, data compression methods can be used to compress the data, such as merging consecutive data with the same status to reduce the amount of data. Finally, the collected data is converted into a unified JSON format for easier subsequent processing. The processed data can then be used for further data analysis.

[0081] During data aggregation, edge devices can aggregate the first state data corresponding to a first smart home device of the same device type to generate device overview data corresponding to the device type; and / or aggregate the second state data corresponding to a second smart home device of the same environmental space to generate device overview data corresponding to the environmental space. Device probability data can be obtained by aggregating the state data of multiple smart home devices, providing an overall status and performance overview of smart home devices. Device overview data can help users quickly understand and manage smart home devices of the same type or in the same environmental space, such as the on / off status, brightness, and color settings of light bulbs; statistical analysis of the overall energy consumption of smart home devices located in the living room; integration of temperature sensor data, humidity sensor data, and device operating status into a single device status record; and inference of whether the current environment is suitable for habitation based on temperature and humidity readings, adding this information to the device status, etc. This invention does not limit the scope of these limitations.

[0082] Optionally, the device overview data can be used for user interface display, intelligent control, system optimization, etc., such as:

[0083] 1. User Interface Display

[0084] Overview Interface: Displays device overview data in the user interface, helping users quickly understand and manage the status of multiple devices.

[0085] Visualization charts: Use visualization tools such as charts and dashboards to intuitively display equipment overview data, such as energy consumption curves and temperature trend charts.

[0086] 2. Intelligent control

[0087] Automation rules: Based on device overview data, set automation rules, such as automatically adjusting the air conditioner according to the average temperature in the living room.

[0088] Intelligent Recommendations: Based on trend analysis of equipment overview data, intelligent recommendations are provided to users, such as energy-saving suggestions and equipment maintenance reminders.

[0089] 3. System optimization

[0090] Performance monitoring: Monitor the overall performance of the smart home system through device overview data, and promptly identify and resolve system problems.

[0091] Data analytics: Utilize equipment overview data for in-depth analysis to optimize system configuration and equipment management strategies.

[0092] In one example, for smart home devices of the same type, the aggregated device overview data may include the following: Number of Devices: A total number of devices belonging to the same device type; Device Status Summary: A summary of device status information, such as the on / off status, average brightness, and color settings of all lighting devices; Statistical Indicators: Calculation of statistical indicators related to the device type, such as average energy consumption, usage frequency, and failure rate; Trend Analysis: Analyzing trends in device status data, such as daily energy consumption trends and seasonal temperature changes. For example, for the device overview data of lighting devices:

[0093] Number of devices: This counts the total number of all lighting devices in a household.

[0094] Equipment Status Summary: This summarizes the on / off status, average brightness value, color settings, etc. of all lighting equipment.

[0095] Statistical indicators: Calculate the average energy consumption and usage frequency of all lighting equipment.

[0096] Trend analysis: Analyze the daily variation trend of energy consumption of lighting equipment and the seasonal variation of usage frequency.

[0097] In another example, the aggregated device overview data for smart home devices in the same environment can include: Device Count: a total number of devices belonging to the same environment; Device Status Summary: a summary of the status information of all devices in the environment, such as the on / off status, temperature, and humidity of all devices in the living room; Statistical Indicators: calculation of statistical indicators related to the environment, such as average temperature, humidity, and energy consumption; Trend Analysis: analysis of trends in device status data within the environment, such as daily temperature trends and seasonal humidity changes. For example, for the device overview data of living room devices:

[0098] Quantity of equipment: Count the total number of all equipment in the living room.

[0099] Equipment Status Summary: This summarizes the on / off status, temperature, humidity, etc. of all equipment in the living room.

[0100] Statistical indicators: Calculate the average temperature, humidity, energy consumption, etc. of all equipment in the living room.

[0101] Trend analysis: Analyze the daily variation trends of temperature and humidity in the living room, as well as the seasonal changes in energy consumption.

[0102] After data aggregation is complete, the edge device can convert the aggregated data into visualized data. This allows AR devices to present the corresponding device status of smart home devices to users based on the visualized data. Specifically, the edge device can acquire the data type of the target data, which includes at least one of the following: time series, classification, relational, geographic location, multidimensional, and trend prediction. Then, it performs a visualization transformation on the target data according to one of these types to obtain the corresponding first visualized data. This first visualized data includes at least one of the following formats: bar chart, pie chart, column chart, network chart, tree diagram, matrix diagram, map, heatmap, bubble chart, parallel coordinate graph, radar chart, scatter matrix, line chart, prediction interval chart, and error bar chart.

[0103] Among them, time series data represents data that changes over time, such as temperature, humidity, and energy consumption; classification data represents data of different categories, such as equipment type and room type; relational data represents the relationships between different data, such as the dependencies between equipment and the relationship between users and equipment; geographic data represents data related to geographic location, such as the geographic distribution of equipment and regional energy consumption; multidimensional data represents data containing multiple dimensions, such as multiple indicators of equipment (temperature, humidity, and energy consumption); and trend prediction data represents trend data and prediction data that change over time, such as future energy consumption prediction.

[0104] In some examples, the visualization types used for time series data may include:

[0105] Line chart: Used to display the trend of data over time.

[0106] Area chart: Used to display the cumulative change of data over time.

[0107] Scatter plot: Used to display the distribution of data points on a time axis.

[0108] The corresponding implementation methods may include:

[0109] Line chart: Using time as the X-axis and data values ​​as the Y-axis, draw a line connecting the data points.

[0110] Area chart: Based on a line chart, fill the area below the line to show cumulative changes.

[0111] Scatter plot: Using time as the X-axis and data values ​​as the Y-axis, plot the scatter points of each data point.

[0112] For categorical data, the visualization types that can be used include:

[0113] Bar chart: Used to compare data of different categories.

[0114] Pie chart: Used to show the proportion of each category of data in the total.

[0115] Bar chart: Similar to column chart, but suitable for situations with many categories.

[0116] The corresponding implementation methods may include:

[0117] Bar chart: Use the category as the X-axis and the data value as the Y-axis to draw bars for different categories.

[0118] Pie chart: Convert the data values ​​of each category into angles, draw a circular chart, and display the proportion of each category.

[0119] Bar chart: Using categories as the Y-axis and data values ​​as the X-axis, bars are drawn for different categories.

[0120] For relational data, the visualization types that can be used include:

[0121] Network diagram: Used to show the relationships between nodes.

[0122] Tree diagram: Used to display hierarchical relationships.

[0123] Matrix diagram: Used to display two-dimensional relational data.

[0124] The corresponding implementation methods may include:

[0125] Network graph: Nodes and edges are drawn on a two-dimensional plane, where nodes represent entities and edges represent relationships.

[0126] Tree diagram: Expands a hierarchical structure from top to bottom or from left to right to show the hierarchical relationship.

[0127] Matrix diagram: Two-dimensional relational data is drawn in matrix form, with rows and columns representing the two dimensions respectively.

[0128] For geographic data, the visualization types that can be used include:

[0129] Map: Used to display the distribution of data in geographic space.

[0130] Heatmap: Used to display the density distribution of data in geographic space.

[0131] Bubble chart: Used to display the size distribution of data in geographic space.

[0132] The corresponding implementation methods may include:

[0133] Maps: Mapping geographic data onto a map, using different colors or symbols to represent data values.

[0134] Heatmap: Maps geographic data onto a map, using color gradients to represent data density.

[0135] Bubble chart: Maps geographic data onto a map, using bubbles of different sizes to represent data values.

[0136] For multidimensional data, the visualization types that can be used include:

[0137] Parallel coordinate graph: used to display the relationships between multidimensional data.

[0138] Radar chart: Used to display the distribution of multidimensional data.

[0139] Scatter matrix plot: used to display pairwise relationships between multidimensional data.

[0140] The corresponding implementation methods may include:

[0141] Parallel coordinate graph: Each dimension is treated as a parallel axis, and the projection of data points onto each axis represents the value of that dimension.

[0142] Radar chart: Each dimension is treated as an axis radiating outward from the center, and the projection of data points onto each axis represents the value of that dimension.

[0143] Scatter matrix plot: Combine each dimension in pairs to draw a scatter plot, showing the relationship between the dimensions.

[0144] For trend forecasting data, the visualization types that can be used include:

[0145] Line chart: Used to display trends and forecasts.

[0146] Prediction interval plot: Used to display the confidence interval of the predicted data.

[0147] Error bar chart: Used to display the error range of the predicted data.

[0148] The corresponding implementation methods may include:

[0149] Line chart: Plot a line with time as the X-axis and trend and forecast data as the Y-axis.

[0150] Prediction Interval Plot: Based on the line chart, plot the confidence interval range of the predicted data.

[0151] Error bar chart: Based on the line chart, error bars are drawn for the predicted data to show the error range.

[0152] Through the above process, information from smart home devices can be collected, preprocessed, analyzed, target data generated, and data visualized. This allows for the generation of target data corresponding to smart home devices, which is then presented to users in a visual manner for easy understanding and use.

[0153] Step 102: The AR device receives the first visualization data sent by the edge device, and displays the management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data;

[0154] Once the edge device generates the corresponding first visualization data, it can send the first visualization data to the AR device. The AR device can then display the management interface of the smart home device in the augmented reality scene based on the first visualization data. This allows users to intuitively understand the device status of the smart home device based on the management interface. Furthermore, based on the interface display and interaction of the AI ​​device, the difficulty of controlling smart home devices is effectively reduced, providing users with a more intuitive and convenient interaction method and improving the ease of control for users of smart home devices.

[0155] The management interface can display the status parameters of smart home devices. For example, for lighting devices, it can display the current brightness, color, and energy consumption; for air conditioning devices, it can display the current temperature, airflow, and airflow direction; and for humidity control devices, it can display the current humidity. This AI-based interface display and interaction effectively reduces the difficulty of controlling smart home devices, provides users with a more intuitive and convenient interaction method, and improves the ease of control for users of smart home devices.

[0156] It should be noted that, based on the description in the foregoing embodiments, the edge device can aggregate the status data of multiple smart home devices. In addition to displaying the device status parameters of a single smart home device, the management interface can also display the overall device parameters of smart home devices of the same type, or the overall device parameters of smart home devices in the same environment. At the same time, it supports users to input corresponding selection operations, switching operations, etc., so that users can control the content displayed in the management interface.

[0157] In some feasible implementations, the AR device can obtain the device type corresponding to the smart home device, then determine the management interface layout corresponding to the device type, fill the management interface layout with the first visualization data, construct the management interface corresponding to the smart home device, obtain the user's target position and browsing perspective in the smart home scene, then locate the augmented reality scene corresponding to the target position in the smart home scene, display the augmented reality scene according to the browsing perspective, and overlay the management interface on the displayed augmented reality scene.

[0158] In its implementation, the AR device communicates with an edge gateway to obtain the device type of the smart home devices and selects a management interface layout corresponding to that device type from a predefined interface layout library. Next, based on the target data received from the edge gateway, this data is populated into the management interface layout, and a complete management interface is constructed according to the populated layout. Simultaneously, the AR device uses cameras and sensors to capture the user's current location and viewing angle. Based on the user's target location, it locates the corresponding augmented reality scene from the smart home scene and displays the augmented reality scene in the real-world environment according to the user's viewing angle. Finally, the AR device overlays the constructed management interface onto the displayed augmented reality scene.

[0159] In one example, the AR device communicates with an edge gateway to obtain device type information for smart home devices (such as a "smart bulb"). Next, the AR device selects a management interface layout corresponding to the device type from a predefined library of interface layouts. This layout includes a power button and a brightness adjustment slider. The AR device receives target data from the edge gateway, containing the bulb's current state (on / off) and brightness value (50%), and populates this data into the management interface layout, making the power button display "on" and the brightness adjustment slider display 50%. Subsequently, the AR device constructs a complete management interface based on the populated layout, which will be displayed in the augmented reality scene. The AR device uses a camera and sensors to detect the user's current location in the living room and their viewing angle, thus obtaining the user's target location and viewing perspective. Based on the user's target location, the AR device locates the augmented reality scene of the living room within the smart home scene and displays this scene in the real-world environment according to the user's viewing perspective. Finally, the AR device overlays the constructed management interface (including power buttons and brightness adjustment sliders) onto the displayed augmented reality scene, allowing users to intuitively see and interact with the virtual management interface in the real environment. This interface display and interaction based on AI devices effectively reduces the difficulty of controlling smart home devices, provides users with a more intuitive and convenient interaction method, and improves the ease of control for users of smart home devices.

[0160] Step 103: The AR device responds to the user's input control operation for the smart home device and obtains the operation data corresponding to the control operation;

[0161] The management interface displayed on the AR device allows users to intuitively perceive the status of the corresponding smart home devices. When users want to control the corresponding smart home devices, they can input the corresponding control operations. The AR device will then respond to the user's input of the control operations, obtain the corresponding operation data, and send the operation data to the edge device. The edge device will then generate the corresponding control commands to control the smart home devices.

[0162] For control operations, these can include gesture operations, voice control operations, and touch operations. AR devices can detect user gesture operations in AR scenes, such as clicking, swiping, and zooming; they can also detect user voice commands, such as "turn on the lights" or "turn up the temperature"; and they can detect user clicks and swiping operations on the touchscreen. Based on user input control operations, AR devices can collect corresponding information according to the operation type. When a user's gesture operation is detected, information such as the type, location, and direction of the gesture is obtained; when a user's voice command is detected, information such as the content and semantics of the voice command is obtained; when a user's touchscreen operation is detected, information such as the location of the touch point and the operation type is obtained.

[0163] Furthermore, based on the acquired operational data, the AR device can encapsulate the acquired operational information into a unified data format, such as JSON or XML, for subsequent processing, and assign a unique identifier (such as an operation ID) to each operation for differentiation and association in subsequent processing.

[0164] Step 104: The edge device analyzes the operation data, generates a target control command for the smart home device, and sends the target control command to the smart home device;

[0165] After the edge device receives the operation data sent by the AR device, it can analyze the operation data to generate target control commands for the smart home devices, and then send the target control commands to the smart home devices to achieve control of the smart home devices.

[0166] In some feasible implementations, the edge device can extract the operation type, operation parameters, and device identifier from the operation data, then obtain the target control instruction corresponding to the device identifier, and then match the operation type and operation parameters with preset control instructions to determine the target control instruction corresponding to the operation data.

[0167] In practical implementation, a data receiving module can be configured in the edge device to receive operation information from the AR device and store it in the edge gateway's database. This ensures that operation information is received in real time, reflecting the user's latest actions. Next, the edge device can parse the operation information, extracting information such as operation type, device identifier, and operation parameters, and verify the legality of the operation to ensure it conforms to preset rules and permissions. For example, it checks whether the user has permission to control a certain device. Simultaneously, a corresponding command generation algorithm can be configured in the edge device or obtained from the cloud, and then control commands for smart home devices can be generated based on the operation information. Specifically, the device type (such as lights, temperature sensors, smart sockets, etc.) can be identified based on the device identifier, and then the operation information can be mapped to the corresponding device control commands. For example, the "turn on the lights" operation can be mapped to a light device on / off command, and then the specific parameters of the control command can be set according to the operation parameters. For example, the "adjust the temperature" operation can be mapped to a temperature adjustment command, and a specific temperature value can be set. After generating the corresponding control commands, the edge device can encapsulate the generated control commands into a unified data format, such as JSON, for subsequent transmission. At the same time, it assigns a unique identifier (such as command ID) to each control command for differentiation and association in subsequent processing. This allows the edge device to process and analyze the device data of smart home devices, improving the real-time performance of data processing compared to centralized computing.

[0168] Step 105: The smart home device executes the device operation corresponding to the target control command.

[0169] For smart home devices, after receiving the corresponding target control command, they can execute the device operation corresponding to the target control command. Thus, in the entire control process of smart home devices, the device data of smart home devices is processed and analyzed based on edge devices. Compared with centralized computing, the real-time performance of data processing is improved. At the same time, the interface display and interaction based on AI devices effectively reduces the control difficulty of smart home devices, provides users with a more intuitive and convenient interaction method, and improves the user's control convenience of smart home devices.

[0170] In addition, the edge device can also collect second device data after the smart home device completes the device operation, analyze and process the second device data to obtain second visualization data corresponding to the smart home device, and the AR device receives the second visualization data sent by the edge device and updates the management interface displayed in the augmented reality scene in real time according to the second visualization data. Thus, when the user controls the smart home device and the smart home device performs the corresponding device operation, the AR device can update the status parameters of the smart home device in the management interface in a timely manner to ensure the real-time nature of the data.

[0171] It should be noted that the embodiments of the present invention include, but are not limited to, the examples described above. It is understood that those skilled in the art can make further settings according to actual needs under the guidance of the ideas in the embodiments of the present invention, and the present invention does not limit such settings.

[0172] In this embodiment of the invention, an application is made to a smart home management system. The smart home management system includes at least an AR device, an edge device, and smart home devices. The AR device provides a graphical user interface (GUI), and the content displayed in the GUI includes at least the augmented reality scene corresponding to the smart home device. During the management and control of the smart home devices, the edge device acquires first device data of the smart home devices and analyzes and processes the first device data to obtain first visual data corresponding to the smart home devices. The AR device receives the first visual data sent by the edge device and displays the management interface corresponding to the smart home devices in the augmented reality scene based on the first visual data. If the user inputs a corresponding control operation, the AR device... The device can respond to user input of control operations for smart home devices, acquire the corresponding operation data, then the edge device analyzes the operation data, generates target control commands for the smart home devices, and sends the target control commands to the smart home devices. Finally, the smart home devices execute the device operations corresponding to the target control commands. Thus, the device data of smart home devices is processed and analyzed based on the edge device. Compared with centralized computing, this improves the real-time performance of data processing. At the same time, the interface display and interaction based on AI devices effectively reduces the difficulty of controlling smart home devices, providing users with a more intuitive and convenient interaction method and improving the ease of user control over smart home devices.

[0173] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the following examples are provided for illustrative purposes:

[0174] In one example, considers a scenario for visual control of smart home devices:

[0175] Background: Users use AR devices (such as AR glasses or smartphones) to control smart devices in their homes.

[0176] 1. Data Acquisition:

[0177] Sensors and smart devices report their current status.

[0178] AR devices obtain the user's location and perspective.

[0179] 2. Data preprocessing:

[0180] Edge gateways receive data from sensors and smart devices.

[0181] Perform data cleaning and formatting.

[0182] 3. Real-time analysis:

[0183] The edge gateway generates virtual information that is overlaid on the real environment based on the user's location and perspective.

[0184] For example, users can see the status icons of each smart device through AR devices.

[0185] 4. Decision-making:

[0186] Users select the device to control using gestures or voice commands.

[0187] The edge gateway parses user commands and generates control instructions.

[0188] 5. Control Execution:

[0189] The edge gateway sends control commands to the selected smart devices.

[0190] The smart device executes the instructions and sends the execution results back to the edge gateway.

[0191] AR devices display execution results, allowing users to see changes in device status.

[0192] 6. Data Upload:

[0193] The edge gateway backs up user operation records and device status to the cloud for subsequent analysis and optimization.

[0194] In another example, regarding the application scenario of smart home security monitoring:

[0195] Background: Users use AR devices to monitor home security in real time.

[0196] Data collection:

[0197] Smart cameras capture video streams.

[0198] Door and window sensors and human body sensors detect changes in the environment.

[0199] Data preprocessing:

[0200] The edge gateway receives data from smart cameras, door and window sensors, and human body sensors.

[0201] Perform data cleaning and formatting.

[0202] Real-time analysis:

[0203] Edge gateways use image recognition technology to analyze video streams captured by cameras and detect abnormal activity.

[0204] The home security status is determined based on data from door and window sensors and human body sensors.

[0205] Decision making:

[0206] If abnormal activity is detected, the edge gateway generates an alert.

[0207] Push alarm information and live video streams to the user's AR device.

[0208] 5. Control Execution:

[0209] Users can view live video streams and alarm information through AR devices.

[0210] Users can send commands via AR devices, such as activating the alarm system or contacting security services.

[0211] The edge gateway executes user commands and reports the execution results.

[0212] 6. Data Upload:

[0213] The edge gateway backs up video streams, sensor data, and user operation logs to the cloud for subsequent analysis and optimization.

[0214] In another example, regarding the application scenario of smart home energy management:

[0215] Background: Users are using AR devices to optimize home energy usage.

[0216] 1. Data Acquisition:

[0217] Smart meters detect household electricity consumption.

[0218] The smart socket detects the power consumption status of each device.

[0219] Solar panel power generation is measured.

[0220] 2. Data preprocessing:

[0221] The edge gateway receives data from smart meters, smart sockets, and solar panels.

[0222] Perform data cleaning and formatting.

[0223] 3. Real-time analysis:

[0224] The edge gateway analyzes household electricity consumption and power generation to generate energy usage reports.

[0225] Based on the user's location and perspective, virtual information is generated and overlaid on the real environment to display the power status of various devices.

[0226] 4. Decision-making:

[0227] Users can view energy usage reports and device status through AR devices.

[0228] Users can choose strategies to optimize energy use, such as turning off high-energy-consuming equipment or prioritizing the use of solar energy.

[0229] 5. Control Execution:

[0230] The edge gateway generates control commands based on the policy selected by the user.

[0231] The smart socket executes commands to adjust the power supply method.

[0232] Users can see the execution results through AR devices and confirm the changes in device status.

[0233] 6. Data Upload:

[0234] The edge gateway backs up electricity consumption data, power generation data, and user operation records to the cloud for subsequent analysis and optimization.

[0235] In another example, regarding the application scenario of smart home health monitoring:

[0236] Background: Users use AR devices to monitor the health status of family members.

[0237] 1. Data Acquisition:

[0238] The heart rate monitor detects the user's real-time heart rate.

[0239] A blood pressure monitor measures a user's blood pressure.

[0240] Smart mattresses detect the user's sleep patterns.

[0241] 2. Data preprocessing:

[0242] The edge gateway receives data from heart rate monitors, blood pressure monitors, and smart mattresses.

[0243] Perform data cleaning and formatting.

[0244] 3. Real-time analysis:

[0245] The edge gateway generates health reports based on the user's heart rate, blood pressure, and sleep status.

[0246] Based on the user's location and perspective, virtual information is generated and overlaid on the real environment to display health indicators.

[0247] 4. Decision-making:

[0248] Users can view health reports and health indicators through AR devices.

[0249] If an anomaly is detected, the edge gateway generates an alert and sends a notification to the user or medical professional.

[0250] 5. Control Execution:

[0251] Users can view alarm information and health advice through AR devices.

[0252] Users can take action based on the suggestions, such as adjusting their sleeping posture or increasing their exercise.

[0253] The edge gateway executes user commands and reports the execution results.

[0254] 6. Data Upload:

[0255] The edge gateway backs up heart rate data, blood pressure data, sleep data, and user operation records to the cloud for subsequent analysis and optimization.

[0256] In another example, for the application scenario of optimizing the smart home environment:

[0257] Background: Users use AR devices to optimize their home environment, such as air quality, humidity, and temperature.

[0258] Data collection:

[0259] Air quality sensors detect indoor air quality.

[0260] A temperature sensor detects the indoor temperature.

[0261] A humidity sensor detects indoor humidity.

[0262] Data preprocessing:

[0263] The edge gateway receives data from air quality sensors, temperature sensors, and humidity sensors.

[0264] Perform data cleaning and formatting.

[0265] Real-time analysis:

[0266] The edge gateway generates environmental reports based on indoor air quality, temperature, and humidity.

[0267] Based on the user's location and perspective, virtual information is generated and overlaid on the real environment to display environmental indicators.

[0268] Decision making:

[0269] Users can view environmental reports and environmental indicators through AR devices.

[0270] Users can choose strategies to optimize their environment, such as turning on an air purifier, adjusting the air conditioner temperature, or using a humidifier.

[0271] Control execution:

[0272] The edge gateway generates control commands based on the policy selected by the user.

[0273] The smart device executes instructions and adjusts environmental parameters.

[0274] Users can see the execution results through AR devices and confirm the changes in environmental parameters.

[0275] Data Upload:

[0276] The edge gateway backs up air quality data, temperature data, humidity data, and user operation records to the cloud for subsequent analysis and optimization.

[0277] In another example, for smart home entertainment and interaction applications:

[0278] Background: Users use AR devices to enhance their home entertainment experience, such as virtual games and interactive stories.

[0279] Data collection:

[0280] Sensors detect the user's location and movements.

[0281] The smart device reports its current status.

[0282] Data preprocessing:

[0283] Edge gateways receive data from sensors and smart devices.

[0284] Perform data cleaning and formatting.

[0285] Real-time analysis:

[0286] Edge gateways generate virtual game or interactive story scenarios based on the user's location and actions.

[0287] For example, users can interact with virtual characters in games using AR devices.

[0288] Decision making:

[0289] Users participate in virtual games or interactive stories through AR devices.

[0290] Users can control the development of the game or story through gestures or voice commands.

[0291] Control execution:

[0292] The edge gateway generates control commands based on user commands.

[0293] Smart devices execute commands to adjust the state of the game or story.

[0294] Users can see the results of the operation through AR devices and enjoy an enhanced entertainment experience.

[0295] Data Upload:

[0296] The edge gateway backs up user operation records and game data to the cloud for subsequent analysis and optimization.

[0297] In practical implementation, assuming a user's home has various smart home appliances such as air conditioners and refrigerators, and is also equipped with edge computing devices like a home gateway, when the user wears AR glasses, these appliances upload device information to the home gateway. The home gateway receives and analyzes this status information, then sends the processed information to the user's AR glasses. The AR glasses overlay virtual interfaces and information onto the real-world environment. The user can then see the status information of these appliances. When the user queries or operates these appliances, they can use gestures on the virtual interface. The AR glasses send the operation information to the home gateway, which receives, analyzes, and transmits it to the smart home devices, which then respond accordingly. Furthermore, there are many other application scenarios. When an air conditioner or refrigerator malfunctions, the user wearing AR glasses can view corresponding fault descriptions and search for solutions through the home gateway. They can also obtain suggestions and tips, allowing them to troubleshoot and resolve the problem step by step.

[0298] Through the above examples, processing and analyzing device data from smart home devices based on edge devices improves the real-time performance of data processing compared to centralized computing. At the same time, the interface display and interaction based on AI devices effectively reduces the difficulty of controlling smart home devices, providing users with a more intuitive and convenient interaction method and improving the ease of control for users of smart home devices.

[0299] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0300] Reference Figure 2 This diagram illustrates a structural block diagram of a smart home management device provided in an embodiment of the present invention. The device is applied to a smart home management system, which includes at least an AR device, an edge device, and smart home devices. The AR device provides a graphical user interface (GUI), and the content displayed by the GUI includes at least the augmented reality scene corresponding to the smart home devices. Specifically, it may include the following modules:

[0301] The data acquisition module 201 located at the edge device is used to acquire the first device data of the smart home device, and analyze and process the first device data to obtain the first visual data corresponding to the smart home device;

[0302] The interface display module 202 located in the AR device is used to receive the first visualization data sent by the edge device, and display the management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data;

[0303] The operation response module 203 located in the AR device is used to respond to the user's input of a control operation for the smart home device and obtain the operation data corresponding to the control operation.

[0304] The instruction generation module 204 located in the edge device is used to analyze the operation data, generate target control instructions for the smart home device, and send the target control instructions to the smart home device;

[0305] The instruction execution module 205 located in the smart home device is used to execute device operations corresponding to the target control instruction.

[0306] In some feasible implementations, the data acquisition module 201 is specifically used for:

[0307] The first device data is cleaned to remove invalid, redundant, and abnormal data, thereby obtaining cleaned first device data.

[0308] The status data of the smart home device is obtained by fusing the data from the first device after cleaning.

[0309] The status data corresponding to multiple smart home devices are aggregated to generate target data associated with specific management functions, and the target data is visualized to obtain the corresponding first visualized data.

[0310] In some feasible implementations, the data acquisition module 201 is specifically used for:

[0311] The first status data corresponding to the first smart home devices belonging to the same device type are aggregated to generate device overview data corresponding to the device type.

[0312] And / or, aggregate the second state data corresponding to the second smart home devices belonging to the same environmental space to generate device overview data corresponding to the environmental space.

[0313] In some feasible implementations, the data acquisition module 201 is specifically used for:

[0314] The data type of the target data is obtained, and the data type includes at least one of the following: time series type, classification type, relation type, geographic location type, multidimensional type, and trend prediction type.

[0315] The target data is visualized according to one of the time series type, the classification type, the relationship type, the geographic location type, the multidimensional type, and the trend prediction type to obtain the corresponding first visualized data;

[0316] The first visualization data includes at least one of the following: bar chart, pie chart, bar graph, network diagram, tree diagram, matrix diagram, map, heat map, bubble chart, parallel coordinate diagram, radar chart, scatter matrix diagram, line chart, prediction interval diagram, and error bar chart.

[0317] In some feasible implementations, the interface display module 202 is specifically used for:

[0318] Obtain the device type corresponding to the smart home device;

[0319] Determine the management interface layout corresponding to the device type, and fill the management interface layout with the first visualization data to construct the management interface corresponding to the smart home device;

[0320] Obtain the user's target location and browsing perspective in the smart home scene;

[0321] The system locates the augmented reality scene corresponding to the target location within the smart home scene, displays the augmented reality scene according to the browsing perspective, and overlays the management interface onto the displayed augmented reality scene.

[0322] In some feasible implementations, the instruction generation module 204 is specifically used for:

[0323] The operation type, operation parameters, and device identifier extracted from the operation data;

[0324] Obtain the device target control command corresponding to the device identifier;

[0325] The operation type and the operation parameters are matched with preset control instructions to determine the target control instruction corresponding to the operation data.

[0326] Among some feasible implementation methods are:

[0327] The data acquisition module located at the edge device is used to collect second device data after the smart home device completes the device operation, analyze and process the second device data, and obtain second visual data corresponding to the smart home device.

[0328] The interface update module located in the AR device is used to receive the second visualization data sent by the edge device and update the management interface displayed in the augmented reality scene in real time according to the second visualization data.

[0329] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0330] In addition, this invention also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described smart home management method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0331] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described smart home management method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0332] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0333] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, EEPROM, Flash, and eMMC, etc.) containing computer-usable program code.

[0334] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0335] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0336] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0337] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0338] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0339] The present invention has provided a detailed description of a smart home management method and a smart home management device. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A smart home management method, characterized in that, An application is made in a smart home management system, the smart home management system including at least an AR device, an edge device, and smart home devices, the AR device providing a graphical user interface, the content displayed by the graphical user interface including at least the augmented reality scene corresponding to the smart home device, the method including: The edge device acquires first device data of the smart home device, performs data cleaning on the first device data to remove invalid, redundant, and abnormal data, and obtains cleaned first device data; the cleaned first device data is then merged to obtain status data of the smart home device; the status data corresponding to multiple smart home devices are aggregated to generate target data associated with a specific management function, and the target data is then visualized to obtain first visualized data corresponding to the smart home device; The AR device receives first visualization data sent by the edge device, and displays the management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data; The AR device responds to the user's input of control operations on the smart home device and obtains the operation data corresponding to the control operations; The edge device analyzes the operation data, generates a target control command for the smart home device, and sends the target control command to the smart home device; The smart home device performs device operations corresponding to the target control command; The step of aggregating the status data corresponding to multiple smart home devices to generate target data associated with specific management functions includes: The first status data corresponding to the first smart home devices belonging to the same device type are aggregated to generate device overview data corresponding to the device type. And / or, aggregate the second status data corresponding to the second smart home devices belonging to the same environmental space to generate device overview data corresponding to the environmental space; the device overview data is used for the graphical user interface display, intelligent control and system optimization; the device overview data includes at least the number of devices, device status summary, statistical indicators and trend analysis.

2. The method according to claim 1, characterized in that, The step of performing visualization transformation on the target data to obtain corresponding first visualization data includes: The data type of the target data is obtained, and the data type includes at least one of the following: time series type, classification type, relation type, geographic location type, multidimensional type, and trend prediction type. The target data is visualized according to one of the time series type, the classification type, the relationship type, the geographic location type, the multidimensional type, and the trend prediction type to obtain the corresponding first visualized data; The first visualization data includes at least one of the following: bar chart, pie chart, bar graph, network diagram, tree diagram, matrix diagram, map, heat map, bubble chart, parallel coordinate diagram, radar chart, scatter matrix diagram, line chart, prediction interval diagram, and error bar chart.

3. The method according to claim 1, characterized in that, The step of displaying the management interface corresponding to the smart home device in the augmented reality scene based on the first visualization data includes: Obtain the device type corresponding to the smart home device; Determine the management interface layout corresponding to the device type, and fill the management interface layout with the first visualization data to construct the management interface corresponding to the smart home device; Obtain the user's target location and browsing perspective in the smart home scene; The system locates the augmented reality scene corresponding to the target location within the smart home scene, displays the augmented reality scene according to the browsing perspective, and overlays the management interface onto the displayed augmented reality scene.

4. The method according to claim 1, characterized in that, The step of analyzing the operational data to generate target control commands for the smart home device includes: The operation type, operation parameters, and device identifier extracted from the operation data; Obtain the device target control command corresponding to the device identifier; The operation type and the operation parameters are matched with preset control instructions to determine the target control instruction corresponding to the operation data.

5. The method according to claim 1, characterized in that, Also includes: The edge device collects second device data after the smart home device completes the device operation, analyzes and processes the second device data, and obtains second visual data corresponding to the smart home device. The AR device receives second visualization data sent by the edge device and updates the management interface displayed in the augmented reality scene in real time based on the second visualization data.

6. A smart home management device, characterized in that, An application is made in a smart home management system, the smart home management system including at least an AR device, an edge device, and smart home devices, the AR device providing a graphical user interface, the content displayed by the graphical user interface including at least the augmented reality scene corresponding to the smart home devices, the device comprising: The data acquisition module located at the edge device is used to acquire first device data of the smart home device, perform data cleaning on the first device data to remove invalid, redundant, and abnormal data, and obtain cleaned first device data; merge the cleaned first device data to obtain status data of the smart home device; aggregate the status data corresponding to multiple smart home devices to generate target data associated with a specific management function, and perform visualization transformation on the target data to obtain first visualized data corresponding to the smart home device; The interface display module located in the AR device is used to receive the first visualization data sent by the edge device, and display the management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data; The operation response module located in the AR device is used to respond to the user's input of control operations on the smart home device and to obtain operation data corresponding to the control operations; The instruction generation module located in the edge device is used to analyze the operation data, generate target control instructions for the smart home device, and send the target control instructions to the smart home device; The instruction execution module located in the smart home device is used to execute device operations corresponding to the target control instruction; The data acquisition module is further used for: The first status data corresponding to the first smart home devices belonging to the same device type are aggregated to generate device overview data corresponding to the device type. And / or, aggregate the second status data corresponding to the second smart home devices belonging to the same environmental space to generate device overview data corresponding to the environmental space; the device overview data is used for the graphical user interface display, intelligent control and system optimization; the device overview data includes at least the number of devices, device status summary, statistical indicators and trend analysis.

7. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method as described in any one of claims 1-5.

8. A computer-readable storage medium having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1-5.

Images