Smart home management method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a smart home management method and device, electronic equipment and a storage medium, and relates to the technical field of smart home, and the method comprises the steps that edge equipment obtains first equipment data of smart home equipment, carries out the analysis processing of the first equipment data, and obtains first visual data corresponding to the smart home equipment; the AR device receives first visual data sent by the edge device, displays a management interface corresponding to the smart home device in an augmented reality scene according to the first visual data, and obtains operation data corresponding to a control operation in response to the control operation for the smart home device; the edge device analyzes the operation data, generates a target control instruction for the smart home device, and sends the target control instruction to the smart home device; and the smart home equipment executes the equipment operation corresponding to the target control instruction, so that the control real-time performance and the control efficiency of the smart home equipment are ensured, and the control difficulty is reduced at the same time.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to a smart home management method, a smart home management device, an electronic device and a computer-readable storage medium. Background Art

[0002] With the gradual development of the Internet of Things, the existing simple intelligent operations can no longer meet people's needs. People are increasingly looking forward to and yearning for a better Internet of Things life. At present, the interaction method in smart homes is relatively simple, usually operated through terminal applications, speakers and other smart terminals. Among them, in the management process of smart home devices, cloud computing is often involved, and cloud computing may cause delays, affecting the real-time control of equipment, and when cloud computing is abnormal, it is easy to fail to respond to device control. In addition, the control process of smart home devices also has complex user interfaces, which makes it difficult for users to get started and affects control efficiency. Summary of the invention

[0003] The embodiments of the present invention provide a smart home management method, device, electronic device and computer-readable storage medium to solve or partially solve the problems of poor real-time performance, high operating difficulty and low control efficiency in the management and control process of smart home devices.

[0004] An embodiment of the present invention discloses a smart home management method, which is applied to a smart home management system, wherein the smart home management system includes at least an AR device, an edge device, and a smart home device, wherein the AR device provides a graphical user interface, and content displayed by the graphical user interface includes at least an augmented reality scene corresponding to the smart home device. The method includes:

[0005] The edge device acquires first device data of the smart home device, and analyzes and processes the first device data to obtain first visualization data corresponding to the smart home device;

[0006] The AR device receives the first visualization data sent by the edge device, and displays the management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data;

[0007] The AR device obtains operation data corresponding to the control operation in response to a control operation input by a user for the smart home device;

[0008] The edge device analyzes the operation data, generates a target control instruction for the smart home device, and sends the target control instruction to the smart home device;

[0009] The smart home device executes a device operation corresponding to the target control instruction.

[0010] In some feasible implementations, the analyzing and processing the first device data to obtain first visualization data corresponding to the smart home device includes:

[0011] Cleaning the first device data to remove invalid data, redundant data, and abnormal data in the first device data to obtain cleaned first device data;

[0012] The cleaned first device data is integrated to obtain the status data of the smart home device;

[0013] The status data corresponding to the plurality of smart home devices are aggregated to generate target data associated with a specific management function, and the target data is visualized and converted to obtain corresponding first visualization data.

[0014] In some feasible implementations, aggregating the status data corresponding to the plurality of smart home devices to generate target data associated with a specific management function includes:

[0015] aggregating first status data corresponding to first smart home devices belonging to the same device type to generate device overview data corresponding to the device type;

[0016] And / or, aggregating second status data corresponding to second smart home devices belonging to the same environment space to generate device overview data corresponding to the environment space.

[0017] In some feasible implementations, performing visualization conversion on the target data to obtain corresponding first visualization data includes:

[0018] Acquire a data type of the target data, the data type including at least one of a time series type, a classification type, a relationship type, a geographic location type, a multidimensional type, and a trend prediction type;

[0019] Performing visualization conversion on the target data according to one of the time series type, the classification type, the relationship type, the geographic location type, the multidimensional type, and the trend prediction type to obtain corresponding first visualization data;

[0020] Wherein, the first visualization data includes at least one of a bar chart, a pie chart, a bar chart, a web chart, a tree chart, a matrix chart, a map, a heat map, a bubble chart, a parallel coordinate chart, a radar chart, a scatter matrix chart, a line chart, a prediction interval chart and an error bar chart.

[0021] In some feasible implementations, displaying a management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data includes:

[0022] Obtain the device type corresponding to the smart home device;

[0023] Determine a management interface layout corresponding to the device type, and fill the first visualization data into the management interface layout to construct a management interface corresponding to the smart home device;

[0024] Obtaining the target location and browsing perspective of the user in the smart home scene;

[0025] The augmented reality scene corresponding to the target position is located in the smart home scene, and the augmented reality scene is displayed according to the browsing perspective, and the management interface is superimposed and displayed in the displayed augmented reality scene.

[0026] In some feasible implementations, analyzing the operation data to generate a target control instruction for the smart home device includes:

[0027] The operation type, operation parameters and device identification extracted from the operation data;

[0028] Obtaining a device target control instruction corresponding to the device identifier;

[0029] The operation type and the operation parameter are matched with preset control instructions respectively to determine a target control instruction corresponding to the operation data.

[0030] Some possible implementations also include:

[0031] The edge device collects second device data after the smart home device completes the device operation, analyzes and processes the second device data, and obtains second visualization data corresponding to the smart home device;

[0032] The AR device receives the second visualization data sent by the edge device, and updates the management interface displayed in the augmented reality scene in real time according to the second visualization data.

[0033] The embodiment of the present invention further discloses a smart home management device, which is applied to a smart home management system. The smart home management system includes at least an AR device, an edge device, and a smart home device. The AR device provides a graphical user interface. The content displayed by the graphical user interface includes at least an augmented reality scene corresponding to the smart home device. The device includes:

[0034] A data acquisition module located at the edge device, used to acquire first device data of the smart home device, and analyze and process the first device data to obtain first visualization data corresponding to the smart home device;

[0035] An interface display module located in the AR device, configured to receive first visualization data sent by the edge device, and display a management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data;

[0036] An operation response module located in the AR device, configured to respond to a control operation input by a user for the smart home device and obtain operation data corresponding to the control operation;

[0037] An instruction generation module located at the edge device, configured to analyze the operation data, generate a target control instruction for the smart home device, and send the target control instruction to the smart home device;

[0038] The instruction execution module located in the smart home device is used to execute the device operation corresponding to the target control instruction.

[0039] In some feasible implementations, the data acquisition module is specifically used to:

[0040] Cleaning the first device data to remove invalid data, redundant data, and abnormal data in the first device data to obtain cleaned first device data;

[0041] The cleaned first device data is integrated to obtain the status data of the smart home device;

[0042] The status data corresponding to the plurality of smart home devices are aggregated to generate target data associated with a specific management function, and the target data is visualized and converted to obtain corresponding first visualization data.

[0043] In some feasible implementations, the data acquisition module is specifically used to:

[0044] aggregating first status data corresponding to first smart home devices belonging to the same device type to generate device overview data corresponding to the device type;

[0045] And / or, aggregating second status data corresponding to second smart home devices belonging to the same environment space to generate device overview data corresponding to the environment space.

[0046] In some feasible implementations, the data acquisition module is specifically used to:

[0047] Acquire a data type of the target data, the data type including at least one of a time series type, a classification type, a relationship type, a geographic location type, a multidimensional type, and a trend prediction type;

[0048] Performing visualization conversion on the target data according to one of the time series type, the classification type, the relationship type, the geographic location type, the multidimensional type, and the trend prediction type to obtain corresponding first visualization data;

[0049] Wherein, the first visualization data includes at least one of a bar chart, a pie chart, a bar chart, a web chart, a tree chart, a matrix chart, a map, a heat map, a bubble chart, a parallel coordinate chart, a radar chart, a scatter matrix chart, a line chart, a prediction interval chart and an error bar chart.

[0050] In some feasible implementations, the interface display module is specifically used to:

[0051] Obtain the device type corresponding to the smart home device;

[0052] Determine a management interface layout corresponding to the device type, and fill the first visualization data into the management interface layout to construct a management interface corresponding to the smart home device;

[0053] Obtaining the target location and browsing perspective of the user in the smart home scene;

[0054] The augmented reality scene corresponding to the target position is located in the smart home scene, and the augmented reality scene is displayed according to the browsing perspective, and the management interface is superimposed and displayed in the displayed augmented reality scene.

[0055] In some feasible implementations, the instruction generation module is specifically used to:

[0056] The operation type, operation parameters and device identification extracted from the operation data;

[0057] Obtaining a device target control instruction corresponding to the device identifier;

[0058] The operation type and the operation parameter are matched with preset control instructions respectively to determine a target control instruction corresponding to the operation data.

[0059] Some possible implementations also include:

[0060] A data acquisition module located at the edge device, used to collect second device data after the smart home device completes the device operation, analyze and process the second device data, and obtain second visualization data corresponding to the smart home device;

[0061] The interface update module located in the AR device is used to receive the second visualization data sent by the edge device and update the management interface displayed in the augmented reality scene in real time according to the second visualization data.

[0062] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0063] The memory is used to store computer programs;

[0064] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0065] The embodiment of the present invention further discloses a computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, enables the processors to execute the method described in the embodiment of the present invention.

[0066] The embodiments of the present invention include the following advantages:

[0067] In an embodiment of the present invention, the smart home management system is applied, and the smart home management system includes at least an AR device, an edge device, and a smart home device. The AR device provides a graphical user interface, and the content displayed by the graphical user interface includes at least an augmented reality scene corresponding to the smart home device. In the process of managing and controlling the smart home device, the edge device obtains first device data of the smart home device, analyzes and processes the first device data, and obtains first visualization data corresponding to the smart home device. The AR device receives the first visualization data sent by the edge device, and displays the management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data. If the user enters the corresponding control operation, the AR The device can respond to the control operation of the smart home device input by the user, obtain the operation data corresponding to the control operation, and then the edge device analyzes the operation data, generates the target control instruction for the smart home device, and sends the target control instruction to the smart home device. Finally, the smart home device executes the device operation corresponding to the target control instruction, thereby processing and analyzing the device data of the smart home device based on the edge device. Compared with centralized computing, the real-time performance of data processing is improved. At the same time, the interface display and interaction based on AI devices effectively reduce the control difficulty of smart home devices, provide users with a more intuitive and convenient interaction method, and improve the convenience of user control of smart home devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1is a flowchart of a smart home management method provided in an embodiment of the present invention;

[0069] Figure 2 It is a structural block diagram of a smart home management device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0071] As an example, with the gradual development of the Internet of Things, the existing simple intelligent operations can no longer meet people's needs. People are increasingly looking forward to and yearning for a better Internet of Things life. At present, the interaction method in smart homes is relatively simple, usually operated through smart terminals such as mobile phone applications and speakers. With the development of AR technology, the use of AR technology can display interfaces and operations more intuitively. At present, traditional centralized computing is used in smart homes. Compared with traditional centralized computing, edge computing has the advantages of strong real-time performance, reduced unnecessary data transmission, central server failure, and emergency response of edge nodes.

[0072] In this regard, in the present invention, it is applied to a smart home management system, the smart home management system at least includes an AR device, an edge device and a smart home device, the AR device provides a graphical user interface, the content displayed by the graphical user interface at least includes an augmented reality scene corresponding to the smart home device, in the process of managing and controlling the smart home device, the edge device obtains first device data of the smart home device, and analyzes and processes the first device data to obtain first visualization data corresponding to the smart home device, the AR device receives the first visualization data sent by the edge device, and displays the management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data, if the user inputs a corresponding control operation, the AR The device can respond to the control operation of the smart home device input by the user, obtain the operation data corresponding to the control operation, and then the edge device analyzes the operation data, generates the target control instruction for the smart home device, and sends the target control instruction to the smart home device. Finally, the smart home device executes the device operation corresponding to the target control instruction, thereby processing and analyzing the device data of the smart home device based on the edge device. Compared with centralized computing, the real-time performance of data processing is improved. At the same time, the interface display and interaction based on AI devices effectively reduce the control difficulty of smart home devices, provide users with a more intuitive and convenient interaction method, and improve the convenience of user control of smart home devices.

[0073] Reference Figure 1, shows a flowchart of a smart home management method provided in an embodiment of the present invention, which is applied to a smart home management system, wherein the smart home management system at least includes an AR device, an edge device, and a smart home device, wherein the AR device provides a graphical user interface, and the content displayed by the graphical user interface at least includes an augmented reality scene corresponding to the smart home device, and specifically may include the following steps:

[0074] Step 101: the edge device acquires first device data of the smart home device, and analyzes and processes the first device data to obtain first visualization data corresponding to the smart home device;

[0075] In an embodiment of the present invention, a smart home management system may include AR devices, edge devices, smart home devices, and the like.

[0076] Among them, for AR devices, it may include AR glasses or smart phones for displaying virtual information and controlling smart home devices; for edge devices, it may be an edge gateway, etc. For example, the edge gateway can be responsible for data preprocessing, local decision-making and real-time response; for smart home devices, it may include sensors and actuators, etc. Sensors may include temperature sensors, humidity sensors, human body sensors, etc., for collecting environmental data; actuators may include smart light bulbs, smart sockets, smart door locks, etc., for executing control instructions.

[0077] In the process of managing smart home devices, sensors can collect environmental data and transmit it to edge devices through the local area network, while AR devices are responsible for collecting user input and control instructions. Edge devices pre-process the data and make local decisions, such as automatically adjusting the air conditioning temperature based on temperature sensor data. If more complex processing is required, the edge device uploads the data to the cloud. The edge device controls the actuator based on local decisions or cloud instructions, such as turning on a smart light bulb or unlocking a smart door lock. AR devices and mobile devices display feedback information, and users can view and adjust settings through these devices. The cloud data center stores historical data for data analysis and machine learning model training. AR devices can provide a corresponding user interface so that users can view device status, history, and analysis reports.

[0078] In some feasible implementations, after the edge device obtains the first device data corresponding to the smart home device, it can clean the first device data to remove invalid data, redundant data, and abnormal data in the first device data to obtain the cleaned first device data, and then fuse the cleaned first device data to obtain the status data of the smart home device, and then aggregate the status data corresponding to multiple smart home devices to generate target data associated with a specific management function, and then perform a visualization conversion on the target data to obtain the corresponding first visualization data.

[0079] It should be noted that for the preprocessing process of the first device data, the edge device can reduce the data volume to improve transmission efficiency by removing outliers, redundant data and invalid data, and compressing the data, and at the same time convert the noise reduced and compressed data into the same format for subsequent processing.

[0080] For example, assuming that the first device data obtained by the edge device includes the status of the smart bulb (on / off, brightness), the power consumption of the smart socket, and the temperature and humidity of the smart thermostat, the first device data can be filtered. For example, the temperature sensor may occasionally return abnormally high temperature values. These abnormal data can be removed by threshold filtering, and then data compression methods can be used to compress the data, such as merging continuous data of the same state to reduce the amount of data. The collected data is then converted into a unified JSON format for subsequent processing. The processed data can be further analyzed.

[0081] In the process of data aggregation, the edge device may aggregate the first status data corresponding to the first smart home device of the same device type to generate device overview data corresponding to the device type; and / or, aggregate the second status data corresponding to the second smart home device of the same environment space to generate device overview data corresponding to the environment space. Among them, for device probability data, it can be used to provide an overall status and performance overview of smart home devices by aggregating the status data of multiple smart home devices. The device overview data can help users quickly understand and manage smart home devices of the same type or smart home devices in the same environment space, such as the switch status, brightness and color settings of lamp devices; count the comprehensive energy consumption of smart home devices located in the living room; integrate temperature sensor data, humidity sensor data and device operating status into a device status record; infer whether the current environment is suitable for living based on the temperature and humidity readings, and add this information to the device status, etc. The present invention does not limit this.

[0082] Optionally, the device overview data can be used for user interface display, intelligent control, system optimization, etc., such as:

[0083] 1. User interface display

[0084] Overview interface: Displays device overview data in the user interface to help users quickly understand and manage the status of multiple devices.

[0085] Visualization charts: Use visualization tools such as charts and dashboards to intuitively display equipment overview data, such as energy consumption curves, temperature trend graphs, etc.

[0086] 2. Intelligent control

[0087] Automation rules: Based on device overview data, set automation rules, such as automatically adjusting the air conditioner based on the average temperature of the living room.

[0088] Smart recommendations: Based on trend analysis of device overview data, provide users with smart recommendations, such as energy-saving suggestions and device maintenance reminders.

[0089] 3. System optimization

[0090] Performance monitoring: Monitor the overall performance of the smart home system through device overview data, and promptly identify and resolve system problems.

[0091] Data analysis: Use device overview data to conduct in-depth analysis and optimize system configuration and device management strategies.

[0092] In one example, for the same type of smart home devices, the aggregated device overview data may include the following: Device quantity: count the total number of devices belonging to the same device type; Device status summary: summarize the device status information, such as the switch status, average brightness value, color setting, etc. of all lighting devices; Statistical indicators: calculate statistical indicators related to the device type, such as average energy consumption, usage frequency, failure rate, etc.; Trend analysis: analyze the trend of device status data, such as the daily change trend of energy consumption, seasonal change of temperature, etc. For example, for the device overview data of lighting devices:

[0093] Number of devices: Count the number of all lighting devices in the home.

[0094] Device Status Summary: Summarizes the switch status, average brightness value, color settings, etc. of all lighting devices.

[0095] Statistical indicators: calculate the average energy consumption, usage frequency, etc. of all lighting equipment.

[0096] Trend analysis: Analyze the daily trend of lighting equipment energy consumption, seasonal changes in frequency of use, etc.

[0097] In another example, for smart home devices in the same environment space, the aggregated device overview data may include: Device quantity: count the total number of devices belonging to the same environment space; Device status summary: summarize the status information of all devices in the environment space, such as the switch status, temperature, humidity, etc. of all devices in the living room; Statistical indicators: calculate statistical indicators related to the environment space, such as average temperature, humidity, energy consumption, etc.; Trend analysis: analyze the trend of device status data in the environment space, such as the daily change trend of temperature, seasonal change of humidity, etc. For example, for the device overview data of the living room device:

[0098] Number of devices: Count the number of all devices in the living room.

[0099] Device status summary: summarizes the switch status, temperature, humidity, etc. of all devices in the living room.

[0100] Statistical indicators: Calculate the average temperature, humidity, energy consumption, etc. of all equipment in the living room.

[0101] Trend analysis: Analyze the daily trends of temperature and humidity in the living room, seasonal changes in energy consumption, etc.

[0102] After the data aggregation is completed, the edge device can convert the aggregated data into visual data so that the AR device can present the corresponding device status of the smart home device to the user based on the visual data. Specifically, the edge device can obtain the data type of the target data, and the data type includes at least one of the time series type, classification type, relationship type, geographic location type, multidimensional type and trend prediction type; then the target data is visualized and converted according to one of the time series type, classification type, relationship type, geographic location type, multidimensional type and trend prediction type to obtain the corresponding first visual data. The first visual data includes at least one of a bar chart, a pie chart, a bar chart, a web chart, a tree chart, a matrix chart, a map, a heat map, a bubble chart, a parallel coordinate chart, a radar chart, a scatter matrix chart, a line chart, a prediction interval chart and an error bar chart.

[0103] Among them, the time series type represents data that changes over time, such as temperature, humidity, and energy consumption; the classification type represents data of different categories, such as device type, room type, etc.; the relationship type represents the relationship between different data, such as the dependency between devices, the relationship between users and devices, etc.; the geographic location type represents data related to the geographic location, such as the geographical distribution of devices, regional energy consumption, etc.; the multidimensional type represents data containing multiple dimensions, such as multiple indicators of equipment (temperature, humidity, energy consumption, etc.); the trend prediction type represents trend data and prediction data that change over time, such as future energy consumption prediction.

[0104] In some examples, for time series data, the visualization types used may include:

[0105] Line chart: used to show the trend of data changes over time.

[0106] Area chart: Used to show the cumulative change of data over time.

[0107] Scatter chart: used to show the distribution of data points on the time axis.

[0108] The corresponding implementation methods may include:

[0109] Line chart: Use time as the X-axis and data value as the Y-axis, and draw a line to connect each data point.

[0110] Area chart: Based on the line chart, fill the area under the line to show the cumulative change.

[0111] Scatter plot: Use time as the X-axis and data value as the Y-axis to plot the scatter of each data point.

[0112] For categorical data, the types of visualizations that can be used include:

[0113] Bar chart: used to compare different categories of data.

[0114] Pie chart: used to show the proportion of each category of data in the total.

[0115] Bar chart: Similar to a column chart, but suitable for situations with more categories.

[0116] The corresponding implementation methods may include:

[0117] Bar chart: Use the category as the X-axis and the data value as the Y-axis to draw columns of different categories.

[0118] Pie chart: Convert the data value of each category into an angle and draw a circular chart to show the proportion of each category.

[0119] Bar chart: Use the category as the Y-axis and the data value as the X-axis to draw bars of different categories.

[0120] For relational data, the visualization types that can be used include:

[0121] Network diagram: used to show the relationship between nodes.

[0122] Treemap: Used to display hierarchical relationships.

[0123] Matrix diagram: used to display two-dimensional relationship data.

[0124] The corresponding implementation methods may include:

[0125] Network diagram: Nodes and edges are drawn on a two-dimensional plane, where nodes represent entities and edges represent relationships.

[0126] Treemap: Expand the hierarchy from top to bottom or from left to right to display the hierarchical relationship.

[0127] Matrix diagram: Plot two-dimensional relational data in matrix form, with rows and columns representing two dimensions respectively.

[0128] For geographic data, the types of visualizations that can be used include:

[0129] Map: Used to display the distribution of data in geographic space.

[0130] Heat map: used to show the density distribution of data in geographic space.

[0131] Bubble chart: used to show the size distribution of data in geographic space.

[0132] The corresponding implementation methods may include:

[0133] Maps: Map geographic data onto a map, using different colors or symbols to represent data values.

[0134] Heatmap: Map geographic data onto a map, using color gradients to indicate data density.

[0135] Bubble chart: Map geographic data onto a map, using bubbles of varying sizes to represent data values.

[0136] For multidimensional data, the visualization types that can be used include:

[0137] Parallel coordinates plot: used to show the relationship between multidimensional data.

[0138] Radar chart: used to display the distribution of multidimensional data.

[0139] Scatter matrix plot: used to display pairwise relationships between multidimensional data.

[0140] The corresponding implementation methods may include:

[0141] Parallel coordinates plot: Each dimension is treated as a parallel axis, and the projection of the data points on each axis represents the value of that dimension.

[0142] Radar chart: Each dimension is taken as an axis radiating outward from the center, and the projection of the data points on each axis represents the value of the dimension.

[0143] Scatter matrix plot: Combine each dimension in pairs and draw a scatter plot to show the relationship between the dimensions.

[0144] For trend forecast data, the visualization types used may include:

[0145] Line chart: Used to display trend and forecast data.

[0146] Prediction Interval Plot: Used to display confidence intervals for predicted data.

[0147] Error bar chart: used to display the error range of predicted data.

[0148] The corresponding implementation methods may include:

[0149] Line chart: Use time as the X-axis and trend and forecast data as the Y-axis to draw a line.

[0150] Prediction interval chart: Based on the line chart, draw the confidence interval range of the predicted data.

[0151] Error bar chart: Based on the line chart, draw error bars of the predicted data to show the error range.

[0152] Through the above process, the collection, preprocessing, analysis, target data generation and data visualization of smart home device information can be realized, so as to generate target data corresponding to the smart home devices and display them to users in a visual way to facilitate user understanding and use.

[0153] Step 102, the AR device receives the first visualization data sent by the edge device, and displays the management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data;

[0154] After the edge device generates the corresponding first visualization data, the first visualization data can be sent to the AR device, so that the AR device can display the management interface corresponding to the smart home device in the augmented reality scene based on the first visualization data, so that the user can intuitively understand the device status corresponding to the smart home device based on the management interface, and then based on the interface display and interaction of the AI ​​device, the control difficulty of the smart home device is effectively reduced, providing users with a more intuitive and convenient interaction method, and improving the convenience of users' control over smart home devices.

[0155] Among them, the management interface can display the device status parameters corresponding to the smart home devices. For example, for lighting equipment, the current brightness, color and energy consumption of the lighting equipment can be displayed; for air-conditioning equipment, the current temperature, air volume and air outlet direction of the air-conditioning equipment can be displayed; for humidity equipment, the current humidity of the humidity equipment can be displayed, etc., thereby effectively reducing the control difficulty of smart home equipment based on the interface display and interaction of AI devices, providing users with a more intuitive and convenient interaction method, and improving the convenience of user control of smart home devices.

[0156] It should be noted that, based on the description in the aforementioned embodiments, the edge device can aggregate the status data corresponding to multiple smart home devices. In addition to displaying the device status parameters corresponding to a single smart home device in the management interface, the overall device parameters corresponding to the same type of smart home devices, or the overall device parameters corresponding to the smart home devices in the same environment space can also be displayed. At the same time, it supports users to input corresponding selection operations, switching operations, etc., so that users can control the content displayed in the management interface.

[0157] In some feasible implementations, the AR device can obtain the device type corresponding to the smart home device, then determine the management interface layout corresponding to the device type, and fill the first visualization data into the management interface layout, construct the management interface corresponding to the smart home device, and obtain the user's target position and browsing perspective in the smart home scene, and then locate the augmented reality scene corresponding to the target position from the smart home scene, and display the augmented reality scene according to the browsing perspective, and overlay the management interface on the displayed augmented reality scene.

[0158] In the specific implementation, the AR device obtains the device type of the smart home device by communicating with the edge gateway, and selects the management interface layout corresponding to the device type from the predefined interface layout library. Then, based on the target data received from the edge gateway, these data are filled into the management interface layout, and then a complete management interface is constructed based on the interface layout filled with data. At the same time, the AR device can capture the user's current position and browsing perspective through the camera and sensor, and then locate the corresponding augmented reality scene from the smart home scene according to the user's target position, and display the augmented reality scene in the real environment according to the user's browsing perspective. Finally, the AR device can overlay the constructed management interface on the displayed augmented reality scene.

[0159] In one example, the AR device obtains the device type information (such as "smart light bulb") of the smart home device by communicating with the edge gateway. Then, the AR device selects the management interface layout corresponding to the device type from the predefined interface layout library, which includes a switch button and a brightness adjustment slider. The AR device receives the target data sent by the edge gateway, which contains the current state (on / off) and brightness value (50%) of the light bulb, and fills these data into the management interface layout, so that the switch button is displayed as "on" and the brightness adjustment slider is displayed as 50%. Subsequently, the AR device builds a complete management interface based on the interface layout filled with data, which will be used to display in the augmented reality scene. The AR device captures the user's current location in the living room and is viewing the lights in the living room through the camera and sensor, thereby obtaining the user's target location and browsing perspective. According to the user's target location, the AR device locates the augmented reality scene of the living room from the smart home scene, and displays the scene in the real environment according to the user's browsing perspective. Finally, the AR device will superimpose the constructed management interface (including the switch button and the brightness adjustment slider) into the displayed augmented reality scene, allowing users to intuitively see the virtual management interface in the real environment and interact with it. The interface display and interaction based on the AI ​​device effectively reduces the difficulty of controlling smart home devices, provides users with a more intuitive and convenient interaction method, and improves the convenience of users' control over smart home devices.

[0160] Step 103: the AR device obtains operation data corresponding to the control operation input by the user for the smart home device;

[0161] For the management interface displayed on the AR device, the user can intuitively perceive the device status of the smart home device. When the user wants to control the corresponding smart home device, he can input the corresponding control operation so that the AR device responds to the control operation input by the user for the smart home device, obtains the operation data corresponding to the control operation, and then sends the operation data to the edge device, which generates the corresponding control instructions to realize the control of the smart home device.

[0162] Among them, for control operations, it can include gesture operations, voice control operations, and touch operations. The AR device can detect the user's gesture operations in the AR scene, such as clicking, sliding, zooming, etc.; it can also detect the user's voice commands, such as "turn on the lights", "raise the temperature", etc.; it can also detect the user's clicks, sliding, and other operations on the touch screen. Based on the control operations input by the user, the AR device can collect corresponding information according to the operation type. When the user's gesture operation is detected, the type, position, direction, and other information of the gesture are obtained; when the user's voice command is detected, the content, semantics, and other information of the voice command are obtained; when the user's touch screen operation is detected, the location of the touch point, the operation type, and other information are obtained.

[0163] Furthermore, based on the acquired operation data, the AR device can encapsulate the acquired operation information into a unified data format, such as JSON or XML, for subsequent processing, and assign a unique identifier (such as an operation ID) to each operation for distinction and association in subsequent processing.

[0164] Step 104: the edge device analyzes the operation data, generates a target control instruction for the smart home device, and sends the target control instruction to the smart home device;

[0165] After the edge device receives the operation data sent by the AR device, it can analyze the operation data to generate target control instructions for the smart home device, and send the target control instructions to the smart home device, thereby realizing the control of the smart home device.

[0166] In some feasible implementations, the edge device can extract the operation type, operation parameters and device identification from the operation data, then obtain the device target control instruction corresponding to the device identification, and then match the operation type and operation parameters with the preset control instructions respectively to determine the target control instruction corresponding to the operation data.

[0167] In a specific implementation, a data receiving module can be configured in the edge device to receive operation information from the AR device and store it in the database of the edge gateway, thereby ensuring that the operation information can be received in real time to reflect the user's latest operation. Then, the edge device can parse the operation information, extract information such as the operation type, device identification, and operation parameters, and verify the legitimacy of the operation to ensure that the operation complies with the preset rules and permissions. For example, check whether the user has the authority to control a certain device. At the same time, the corresponding instruction generation algorithm can be configured in the edge device or the corresponding instruction generation algorithm can be obtained from the cloud, and then the control instructions for the smart home device can be generated according to the operation information. Specifically, the device type (such as light, temperature sensor, smart socket, etc.) can be identified according to the device identification, and then the operation information can be mapped to the corresponding device control instruction. For example, the "turn on the light" operation is mapped to the switch instruction of the lighting device, and then the specific parameters of the control instruction are set according to the operation parameters. For example, the "increase the temperature" operation is mapped to the temperature adjustment instruction, and the specific temperature value is set. After generating the corresponding control instructions, the edge device can encapsulate the generated control instructions into a unified data format, such as Json, for subsequent transmission, and assign a unique identifier (such as instruction ID) to each control instruction for distinction and association in subsequent processing, thereby processing and analyzing the device data of smart home devices based on the edge device, which improves the real-time performance of data processing compared to centralized computing.

[0168] Step 105: The smart home device executes a device operation corresponding to the target control instruction.

[0169] For smart home devices, after receiving the corresponding target control instructions, they can execute the device operations corresponding to the target control instructions, so that in the entire control process of the smart home devices, the device data of the smart home devices can be processed and analyzed based on the edge devices. Compared with centralized computing, the real-time performance of data processing is improved. At the same time, the interface display and interaction based on AI devices effectively reduce the control difficulty of smart home devices, provide users with a more intuitive and convenient interaction method, and improve the convenience of user control of smart home devices.

[0170] In addition, the edge device can also collect the second device data of the smart home device after completing the device operation, analyze and process the second device data, and obtain the second visualization data corresponding to the smart home device. The AR device receives the second visualization data sent by the edge device and updates the management interface displayed in the augmented reality scene in real time according to the second visualization data. Therefore, when the user controls the smart home device and the smart home device performs the corresponding device operation, the AR device can promptly update the status parameters of the smart home device in the management interface to ensure the real-time nature of the data.

[0171] It should be noted that the embodiments of the present invention include but are not limited to the above examples. It is understandable that those skilled in the art can also make settings according to actual needs under the guidance of the ideas of the embodiments of the present invention, and the present invention is not limited to this.

[0172] In an embodiment of the present invention, the smart home management system is applied, and the smart home management system includes at least an AR device, an edge device, and a smart home device. The AR device provides a graphical user interface, and the content displayed by the graphical user interface includes at least an augmented reality scene corresponding to the smart home device. In the process of managing and controlling the smart home device, the edge device obtains first device data of the smart home device, analyzes and processes the first device data, and obtains first visualization data corresponding to the smart home device. The AR device receives the first visualization data sent by the edge device, and displays the management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data. If the user enters the corresponding control operation, the AR The device can respond to the control operation of the smart home device input by the user, obtain the operation data corresponding to the control operation, and then the edge device analyzes the operation data, generates the target control instruction for the smart home device, and sends the target control instruction to the smart home device. Finally, the smart home device executes the device operation corresponding to the target control instruction, thereby processing and analyzing the device data of the smart home device based on the edge device. Compared with centralized computing, the real-time performance of data processing is improved. At the same time, the interface display and interaction based on AI devices effectively reduce the control difficulty of smart home devices, provide users with a more intuitive and convenient interaction method, and improve the convenience of user control of smart home devices.

[0173] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the following examples are used for exemplary description:

[0174] In one example, for the application scenario of visual control of smart home devices:

[0175] Background: Users use AR devices (such as AR glasses or smartphones) to control smart devices in their homes.

[0176] 1. Data collection:

[0177] Sensors and smart devices report their current status.

[0178] The AR device obtains the user's position and perspective.

[0179] 2. Data preprocessing:

[0180] Edge gateways receive data from sensors and smart devices.

[0181] Perform data cleaning and formatting.

[0182] 3. Real-time analysis:

[0183] The edge gateway generates virtual information and overlays it on the real environment based on the user's location and perspective.

[0184] For example, users can see the status icon of each smart device through the AR device.

[0185] 4. Decision Making:

[0186] The user selects the device to control through gestures or voice commands.

[0187] The edge gateway parses user commands and generates control instructions.

[0188] 5. Control execution:

[0189] The edge gateway sends control instructions to the selected smart devices.

[0190] The smart device executes the instructions and feeds back the execution results to the edge gateway.

[0191] The AR device displays the execution results, and users can see changes in device status.

[0192] 6. Data upload:

[0193] The edge gateway backs up user operation records and device status to the cloud for subsequent analysis and optimization.

[0194] In another example, for the application scenario of smart home security monitoring:

[0195] Background: Users use AR devices to monitor home security status in real time.

[0196] Data collection:

[0197] Smart cameras capture video streams.

[0198] Door and window sensors and human motion sensors detect environmental changes.

[0199] Data preprocessing:

[0200] The edge gateway receives data from smart cameras, door and window sensors, and human presence sensors.

[0201] Perform data cleaning and formatting.

[0202] Real-time analytics:

[0203] The edge gateway uses image recognition technology to analyze the video stream captured by the camera and detect abnormal activities.

[0204] Determine the home security status based on the data from door and window sensors and human body sensors.

[0205] Decision Making:

[0206] If unusual activity is detected, the edge gateway generates an alert.

[0207] Push alert information and real-time video stream to the user's AR device.

[0208] 5. Control execution:

[0209] Users view real-time video streams and alarm information through AR devices.

[0210] Users can send instructions through AR devices, such as setting off an alarm system or contacting security services.

[0211] The edge gateway executes user instructions and feeds back the execution results.

[0212] 6. Data upload:

[0213] The edge gateway backs up video streams, sensor data, and user operation records to the cloud for subsequent analysis and optimization.

[0214] In another example, for the application scenario of smart home energy management:

[0215] Background: Users use AR devices to optimize home energy usage.

[0216] 1. Data collection:

[0217] Smart meters monitor household electricity usage.

[0218] Smart sockets detect the power usage status of each device.

[0219] Solar panels detect power generation.

[0220] 2. Data preprocessing:

[0221] The edge gateway receives data from smart meters, smart plugs, and solar panels.

[0222] Perform data cleaning and formatting.

[0223] 3. Real-time analysis:

[0224] The edge gateway analyzes household electricity consumption and generation and generates energy usage reports.

[0225] Based on the user's position and perspective, virtual information is generated and superimposed on the real environment to display the power usage status of each device.

[0226] 4. Decision Making:

[0227] Users view energy usage reports and equipment status through AR devices.

[0228] Users can choose strategies to optimize energy use, such as turning off high-energy-consuming devices or prioritizing solar power use.

[0229] 5. Control execution:

[0230] The edge gateway generates control instructions based on the strategy selected by the user.

[0231] The smart socket executes instructions and adjusts the power supply mode.

[0232] Users see the execution results through AR devices and confirm changes in device status.

[0233] 6. Data upload:

[0234] The edge gateway backs up electricity consumption data, power generation data and user operation records to the cloud for subsequent analysis and optimization.

[0235] In another example, for the application scenario of smart home health monitoring:

[0236] Background: Users use AR devices to monitor the health status of family members.

[0237] 1. Data collection:

[0238] The heart rate monitor detects the user's real-time heart rate.

[0239] The sphygmomanometer detects the user's blood pressure.

[0240] Smart mattresses detect the user's sleeping status.

[0241] 2. Data preprocessing:

[0242] The edge gateway receives data from heart rate monitors, blood pressure monitors, and smart mattresses.

[0243] Perform data cleaning and formatting.

[0244] 3. Real-time analysis:

[0245] The edge gateway generates a health report based on the user's heart rate, blood pressure, and sleep status.

[0246] Based on the user's position and perspective, virtual information is generated and superimposed on the real environment to display health indicators.

[0247] 4. Decision Making:

[0248] Users view health reports and health indicators through AR devices.

[0249] If an abnormal situation is detected, the edge gateway generates an alert and sends a notification to the user or medical professional.

[0250] 5. Control execution:

[0251] Users view alert information and health advice through AR devices.

[0252] Users can then take action based on the recommendations, such as adjusting their sleeping position or increasing their physical activity.

[0253] The edge gateway executes user instructions and feeds back the execution results.

[0254] 6. Data upload:

[0255] The edge gateway backs up heart rate data, blood pressure data, sleep data, and user operation records to the cloud for subsequent analysis and optimization.

[0256] In another example, for the application scenario of smart home environment optimization:

[0257] Background: Users use AR devices to optimize home environments such as air quality, humidity, and temperature.

[0258] Data collection:

[0259] Air quality sensors detect indoor air quality.

[0260] The temperature sensor detects the indoor temperature.

[0261] The humidity sensor detects the indoor humidity.

[0262] Data preprocessing:

[0263] The edge gateway receives data from air quality sensors, temperature sensors, and humidity sensors.

[0264] Perform data cleaning and formatting.

[0265] Real-time analytics:

[0266] The edge gateway generates an environmental report based on indoor air quality, temperature, and humidity.

[0267] Based on the user's position and perspective, virtual information is generated and superimposed on the real environment to display environmental indicators.

[0268] Decision Making:

[0269] Users view environmental reports and environmental indicators through AR devices.

[0270] Users can choose strategies to optimize their environment, such as turning on an air purifier, adjusting the temperature of an air conditioner, or the humidity of a humidifier.

[0271] Control execution:

[0272] The edge gateway generates control instructions based on the strategy selected by the user.

[0273] Smart devices execute instructions and adjust environmental parameters.

[0274] Users can see the execution results through AR devices and confirm the changes in environmental parameters.

[0275] Data upload:

[0276] The edge gateway backs up air quality data, temperature data, humidity data and user operation records to the cloud for subsequent analysis and optimization.

[0277] In another example, for the application scenario of smart home entertainment interaction:

[0278] Background: Users use AR devices to enhance their home entertainment experience, such as virtual games, interactive stories, etc.

[0279] Data collection:

[0280] Sensors detect the user's position and movements.

[0281] The smart device reports the current status.

[0282] Data preprocessing:

[0283] Edge gateways receive data from sensors and smart devices.

[0284] Perform data cleaning and formatting.

[0285] Real-time analytics:

[0286] The edge gateway generates scenes for virtual games or interactive stories based on the user's location and movements.

[0287] For example, users can interact with virtual characters in games through AR devices.

[0288] Decision Making:

[0289] Users participate in virtual games or interactive stories through AR devices.

[0290] Users can control the development of the game or story through gestures or voice commands.

[0291] Control execution:

[0292] The edge gateway generates control instructions based on user commands.

[0293] Smart devices execute instructions and adjust the state of the game or story.

[0294] Users see the execution results through AR devices and enjoy an enhanced entertainment experience.

[0295] Data upload:

[0296] The edge gateway backs up user operation records and game data to the cloud for subsequent analysis and optimization.

[0297] In the specific implementation, it is assumed that the user's home has multiple smart home appliances such as air conditioners and refrigerators, and is also equipped with edge computers such as home gateways. When the user wears AR glasses, multiple home appliances such as air conditioners and refrigerators upload device information to the home gateway. When the home gateway receives the status information of home appliances such as air conditioners and refrigerators, it analyzes and processes it, and sends the analyzed and processed information to the user's AR glasses. AR glasses superimpose virtual interfaces and information in the real environment. After wearing AR glasses, users can see the status information of home appliances such as air conditioners and refrigerators. When users query and operate home appliances such as air conditioners and refrigerators, they directly use gestures to operate on the virtual interface. AR glasses send the operation information to the home gateway. After receiving the operation information sent by AR glasses, the home gateway analyzes and processes it and transmits it to smart home devices, and the smart home devices respond to the corresponding operations. In addition, there are many actual application scenarios. When the air conditioner or refrigerator fails, the user can view the corresponding fault description by wearing AR glasses and can also query the corresponding solution through the home gateway. You can also get many suggestions and tips, and users can gradually troubleshoot and solve the problem according to the suggestions and tips.

[0298] Through the above examples, the device data of smart home devices is processed and analyzed based on edge devices, which improves the real-time performance of data processing compared to centralized computing. At the same time, the interface display and interaction based on AI devices effectively reduce the control difficulty of smart home devices, provide users with a more intuitive and convenient interaction method, and improve the convenience of user control of smart home devices.

[0299] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0300] Reference Figure 2 , shows a structural block diagram of a smart home management device provided in an embodiment of the present invention, which is applied to a smart home management system. The smart home management system includes at least an AR device, an edge device, and a smart home device. The AR device provides a graphical user interface. The content displayed by the graphical user interface includes at least an augmented reality scene corresponding to the smart home device. Specifically, the following modules may be included:

[0301] The data acquisition module 201 located at the edge device is used to acquire first device data of the smart home device, and analyze and process the first device data to obtain first visualization data corresponding to the smart home device;

[0302] An interface display module 202 located in the AR device is used to receive the first visualization data sent by the edge device, and display the management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data;

[0303] An operation response module 203 located in the AR device is used to respond to a control operation input by a user for the smart home device and obtain operation data corresponding to the control operation;

[0304] The instruction generation module 204 located at the edge device is used to analyze the operation data, generate a target control instruction for the smart home device, and send the target control instruction to the smart home device;

[0305] The instruction execution module 205 located in the smart home device is used to execute the device operation corresponding to the target control instruction.

[0306] In some feasible implementations, the data acquisition module 201 is specifically used for:

[0307] Cleaning the first device data to remove invalid data, redundant data, and abnormal data in the first device data to obtain cleaned first device data;

[0308] The cleaned first device data is integrated to obtain the status data of the smart home device;

[0309] The status data corresponding to the plurality of smart home devices are aggregated to generate target data associated with a specific management function, and the target data is visualized and converted to obtain corresponding first visualization data.

[0310] In some feasible implementations, the data acquisition module 201 is specifically used for:

[0311] aggregating first status data corresponding to first smart home devices belonging to the same device type to generate device overview data corresponding to the device type;

[0312] And / or, aggregating second status data corresponding to second smart home devices belonging to the same environment space to generate device overview data corresponding to the environment space.

[0313] In some feasible implementations, the data acquisition module 201 is specifically used for:

[0314] Acquire a data type of the target data, the data type including at least one of a time series type, a classification type, a relationship type, a geographic location type, a multidimensional type, and a trend prediction type;

[0315] Performing visualization conversion on the target data according to one of the time series type, the classification type, the relationship type, the geographic location type, the multidimensional type, and the trend prediction type to obtain corresponding first visualization data;

[0316] Wherein, the first visualization data includes at least one of a bar chart, a pie chart, a bar chart, a web chart, a tree chart, a matrix chart, a map, a heat map, a bubble chart, a parallel coordinate chart, a radar chart, a scatter matrix chart, a line chart, a prediction interval chart and an error bar chart.

[0317] In some feasible implementations, the interface display module 202 is specifically used for:

[0318] Obtain the device type corresponding to the smart home device;

[0319] Determine a management interface layout corresponding to the device type, and fill the first visualization data into the management interface layout to construct a management interface corresponding to the smart home device;

[0320] Obtaining the target location and browsing perspective of the user in the smart home scene;

[0321] The augmented reality scene corresponding to the target position is located in the smart home scene, and the augmented reality scene is displayed according to the browsing perspective, and the management interface is superimposed and displayed in the displayed augmented reality scene.

[0322] In some feasible implementations, the instruction generation module 204 is specifically used to:

[0323] The operation type, operation parameters and device identification extracted from the operation data;

[0324] Obtaining a device target control instruction corresponding to the device identifier;

[0325] The operation type and the operation parameter are matched with preset control instructions respectively to determine a target control instruction corresponding to the operation data.

[0326] Some possible implementations also include:

[0327] A data acquisition module located at the edge device, used to collect second device data after the smart home device completes the device operation, analyze and process the second device data, and obtain second visualization data corresponding to the smart home device;

[0328] The interface update module located in the AR device is used to receive the second visualization data sent by the edge device and update the management interface displayed in the augmented reality scene in real time according to the second visualization data.

[0329] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0330] In addition, an embodiment of the present invention further provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned smart home management method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0331] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the above-mentioned smart home management method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0332] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0333] It should be understood by those skilled in the art that the embodiments of the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, EEPROM, Flash, and eMMC, etc.) containing computer-usable program codes.

[0334] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0335] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0336] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0337] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0338] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0339] The above is a detailed introduction to a smart home management method and a smart home management device provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A smart home management method, characterized in that: Applied to a smart home management system, the smart home management system includes at least an AR device, an edge device, and a smart home device, the AR device provides a graphical user interface, and the content displayed by the graphical user interface includes at least an augmented reality scene corresponding to the smart home device, and the method includes: The edge device acquires first device data of the smart home device, and analyzes and processes the first device data to obtain first visualization data corresponding to the smart home device; The AR device receives the first visualization data sent by the edge device, and displays the management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data; The AR device obtains operation data corresponding to the control operation in response to a control operation input by a user for the smart home device; The edge device analyzes the operation data, generates a target control instruction for the smart home device, and sends the target control instruction to the smart home device; The smart home device executes a device operation corresponding to the target control instruction.

2. The method according to claim 1, characterized in that The analyzing and processing the first device data to obtain first visualization data corresponding to the smart home device includes: Cleaning the first device data to remove invalid data, redundant data, and abnormal data in the first device data to obtain cleaned first device data; The cleaned first device data is integrated to obtain the status data of the smart home device; The status data corresponding to the plurality of smart home devices are aggregated to generate target data associated with a specific management function, and the target data is visualized and converted to obtain corresponding first visualization data.

3. The method according to claim 2, characterized in that The aggregating the status data corresponding to the plurality of smart home devices to generate target data associated with a specific management function includes: aggregating first status data corresponding to first smart home devices belonging to the same device type to generate device overview data corresponding to the device type; And / or, aggregating second status data corresponding to second smart home devices belonging to the same environment space to generate device overview data corresponding to the environment space.

4. The method according to claim 2 or 3, characterized in that: The performing visual conversion on the target data to obtain corresponding first visual data includes: Acquire a data type of the target data, the data type including at least one of a time series type, a classification type, a relationship type, a geographic location type, a multidimensional type, and a trend prediction type; Performing visualization conversion on the target data according to one of the time series type, the classification type, the relationship type, the geographic location type, the multidimensional type, and the trend prediction type to obtain corresponding first visualization data; Wherein, the first visualization data includes at least one of a bar chart, a pie chart, a bar chart, a web chart, a tree chart, a matrix chart, a map, a heat map, a bubble chart, a parallel coordinate chart, a radar chart, a scatter matrix chart, a line chart, a prediction interval chart and an error bar chart.

5. The method according to claim 1, characterized in that The displaying the management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data includes: Obtain the device type corresponding to the smart home device; Determine a management interface layout corresponding to the device type, and fill the first visualization data into the management interface layout to construct a management interface corresponding to the smart home device; Obtaining the target location and browsing perspective of the user in the smart home scene; The augmented reality scene corresponding to the target position is located in the smart home scene, and the augmented reality scene is displayed according to the browsing perspective, and the management interface is superimposed and displayed in the displayed augmented reality scene.

6. The method according to claim 1, characterized in that The step of analyzing the operation data to generate a target control instruction for the smart home device includes: The operation type, operation parameters and device identification extracted from the operation data; Obtaining a device target control instruction corresponding to the device identifier; The operation type and the operation parameter are matched with preset control instructions respectively to determine a target control instruction corresponding to the operation data.

7. The method according to claim 1, characterized in that Also includes: The edge device collects second device data after the smart home device completes the device operation, analyzes and processes the second device data, and obtains second visualization data corresponding to the smart home device; The AR device receives the second visualization data sent by the edge device, and updates the management interface displayed in the augmented reality scene in real time according to the second visualization data.

8. A smart home management device, characterized in that: Applied to a smart home management system, the smart home management system includes at least an AR device, an edge device, and a smart home device, the AR device provides a graphical user interface, and the content displayed by the graphical user interface includes at least an augmented reality scene corresponding to the smart home device, and the device includes: A data acquisition module located at the edge device, used to acquire first device data of the smart home device, and analyze and process the first device data to obtain first visualization data corresponding to the smart home device; An interface display module located in the AR device, configured to receive first visualization data sent by the edge device, and display a management interface corresponding to the smart home device in the augmented reality scene according to the first visualization data; An operation response module located in the AR device, configured to respond to a control operation input by a user for the smart home device and obtain operation data corresponding to the control operation; An instruction generation module located at the edge device, configured to analyze the operation data, generate a target control instruction for the smart home device, and send the target control instruction to the smart home device; The instruction execution module located in the smart home device is used to execute the device operation corresponding to the target control instruction.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 7 when executing the program stored in the memory.

10. A computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 7.

Images