Testing system and method for smart home equipment

Abstract

The invention relates to a test system and method for smart home equipment, and belongs to the technical field of equipment testing, and the method comprises an offline server which is used for collecting path data used by a multi-dimensional end side, carrying out the preprocessing and feature extraction of the path data, obtaining a model feature vector, inputting the model feature vector into an offline model, and obtaining a test result; obtaining an offline test path library, obtaining data of an uncovered target path synchronized by an online server, and performing incremental training on the data of the uncovered target path based on an offline model; and the online server is used for collecting user dynamic paths in real time, identifying uncovered target paths from the user dynamic paths based on a two-dimensional comparison method, determining a test strategy based on the types of the uncovered target paths, executing the test strategy, and synchronizing data of the uncovered target paths to the offline server. According to the system, high-coverage, high-efficiency and low-cost testing of a user use path is realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of device testing, and particularly relates to a testing system and method for smart home devices. BACKGROUND

[0002] With the acceleration of technology iteration, intelligent terminal devices (such as smart phones, smart home devices, etc.) are continuously integrating complex functions such as Internet of Things interaction, AI multi-modal response, and cross-scene collaboration, which directly promotes the explosive growth of the demand for functional testing, compatibility testing, and stability testing. Not only does it need to cover compatibility verification of different hardware configurations, operating system versions, and network environments, but it also needs to cope with boundary scenarios and extreme working condition testing under high-frequency functional updates. The traditional manual testing mode has highlighted the shortcomings of low efficiency, incomplete coverage, and high error rate. Therefore, how to better implement device testing has become a problem to be solved. SUMMARY

[0003] In view of the deficiencies of the prior art, the purpose of the present application is to provide a testing system and method for smart home devices.

[0004] In a first aspect of the present application, a testing system for smart home devices is provided, comprising: an offline server configured to collect path data used by multi-dimensional terminal sides, pre-process and extract features from the path data to obtain model feature vectors, input the model feature vectors into an offline model to obtain an offline test path library, and deploy the offline test path library to a test terminal, and further configured to obtain data of an uncovered target path synchronized by an online server, perform incremental training on the data of the uncovered target path based on the offline model, and update the offline test path library; and the online server is configured to collect user dynamic paths in real time, identify the uncovered target path from the user dynamic paths based on a double-dimensional comparison method, determine a test strategy based on the type of the uncovered target path, execute the test strategy, record the node state of the uncovered target path during execution of the test strategy, and further configured to synchronize the data of the uncovered target path to the offline server.

[0005] Further, the collection sources of the path data include local logs on the terminal side, App operation caches on the terminal side, and communication data packets between terminal sides in the local area network. The types of the path data include path data of different device types, path data of different user scenarios, and abnormal scenario path data. The path data includes path nodes, node dependency relationships, path execution time, and abnormal feedback information.

[0006] Further, the path data is preprocessed and feature extracted to obtain a model feature vector, including: removing the same path data in the path data to obtain removed path data; determining a path with a missing node from the removed path data based on a template matching rule, and completing the missing node to obtain path data after the missing node is completed, and labeling the path data after the missing node based on multiple dimensions to obtain an offline training data set, wherein the multiple dimensions include path type, user usage frequency and fault impact range; extracting a target feature from the offline training data set, and converting the target feature into the model feature vector based on a matrix operation tool, wherein the target feature includes behavior structure feature, behavior attribute feature, device collaboration feature and abnormal behavior feature.

[0007] Further, the offline model is a two-stage model, and the two-stage model includes a first stage model and a second stage model, wherein the first stage model is trained by: obtaining a labeled data set, and dividing the labeled data set into a training set and a validation set; inputting the training set into the first stage model for training, the first stage model calculating a predicted value through forward propagation, calculating an error between the predicted value and a true label through a loss function, dynamically adjusting parameters of the offline model using a back propagation algorithm and an optimizer, and minimizing the loss function; inputting the validation set into the first stage model for verification to obtain a business indicator on the validation set, and adjusting parameters of the first stage model based on the business indicator; wherein the model feature vector is input into the offline model to obtain an offline test path library, including: inputting the model feature vector into the first stage model to obtain a path integrity indicator and a path user association indicator; inputting the path integrity indicator and the path user association indicator into the second stage model, the second stage model obtaining a first test path library based on a path reward score = a*path coverage reward + b*test efficiency reward, wherein a represents a weight of path coverage, and b represents a weight of test efficiency; obtaining a path priority score of each path in the first test path library based on a path priority score = a*user dependence of the path * b*weight of the fault impact range of the path; sorting the path priority score of each path, and taking the sorted first test path library as the offline test path library.

[0008] Further, the uncovered target path is identified from the user dynamic path based on a two-dimensional comparison method, including: structurally comparing the user dynamic path with a test path in the offline test path library to obtain an uncovered new path; extracting features of the uncovered new path, and evaluating the extracted features to obtain the uncovered target path based on an evaluation result.

[0009] Further, the test strategy is determined based on the type of the uncovered target path, including: obtaining a rule base of path types and test strategies, the rule base including the test strategy corresponding to the path type; and matching the type of the uncovered target path with the rule base to obtain the test strategy corresponding to the type of the uncovered target path.

[0010] Further, in the case where the uncovered target path is obtained, a task value score of the uncovered target path is obtained based on task value score = weight of urgency of the uncovered target path * user demand frequency of the uncovered target path * weight of fault influence range of the uncovered target path, and the uncovered target path is sorted based on the task value score of the uncovered target path.

[0011] In a second aspect, the present application provides a test method of an intelligent home device, including: collecting path data of multi-dimensional end-side use, pre-processing and feature extraction are performed on the path data to obtain a model feature vector, the model feature vector is input into an offline model to obtain an offline test path library; real-time collection of user dynamic paths, identification of an uncovered target path from the user dynamic paths based on a double-dimensional comparison method, determination of a test strategy based on the type of the uncovered target path, and execution of the test strategy, while synchronizing data of the uncovered target path to the offline model to enable the offline model to perform incremental training on the data of the uncovered target path, and updating the offline test path library.

[0012] In a third aspect, the present application provides an electronic device, including: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute the method of the second aspect.

[0013] In a fourth aspect, the present application provides a non-transitory computer readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method of the second aspect.

[0014] The present application has the following advantages:

[0015] The test system and method of the smart home device, the offline server, are used for collecting path data used by a multi-dimensional terminal side, pre-processing and feature extraction of the path data, obtaining a model feature vector, inputting the model feature vector into an offline model, obtaining an offline test path library, and deploying the offline test path library to a test terminal, and are also used for obtaining data of an uncovered target path synchronized by an online server, performing incremental training on the data of the uncovered target path based on the offline model, and updating the offline test path library; the online server is used for collecting a user dynamic path in real time, identifying an uncovered target path from the user dynamic path based on a double-dimension comparison method, determining a test strategy based on a type of the uncovered target path, and executing the test strategy, while recording a node state of the uncovered target path in the process of executing the test strategy, and is also used for synchronizing the data of the uncovered target path to the offline server. The system combines static learning of the offline server and dynamic exploration of the online server, and realizes high-coverage, high-efficiency, and low-cost testing of a user use path. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the application and, together with the description, serve to explain the principles of the application. In the drawings:

[0017] Figure 1 FIG. 1 is a schematic diagram of a test system of a smart home device according to an embodiment of the application;

[0018] Figure 2 FIG. 2 is a flowchart of a test method of a smart home device according to an embodiment of the application;

[0019] Figure 3 FIG. 3 is a structural block diagram of an electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0020] In order to make the personnel in the art better understand the technical solutions in the embodiments of the application, the technical solutions of the application will be clearly and completely described below in combination with the drawings. Obviously, the described embodiments are some of the embodiments of the application, rather than all the embodiments of the application. It should be understood that these descriptions are exemplary, and are not used to limit the scope of the application. Based on the embodiments of the application, all other embodiments obtained by those of ordinary skill in the art without creative labor should belong to the scope of protection of the application.

[0021] In addition, in the following description, the description of well-known structures and techniques is omitted to avoid unnecessary confusion of the concepts disclosed in the application.

[0022] In the description of the present application, it should be noted that, unless otherwise explicitly specified and limited, the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance. The terms "mounting", "connection", "connection" should be broadly understood, for example, it can be fixed connection, or detachable connection, or integral connection; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or the communication between two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0023] The exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Rather, they are merely examples of methods and systems consistent with some aspects of the present application as detailed in the appended claims.

[0024] With the acceleration of technology iteration, intelligent terminal devices (such as smart phones, smart home devices, etc.) are constantly integrating complex functions such as Internet of Things interaction, AI multi-modal response, cross-scene collaboration, etc., directly driving the demand for functional testing, compatibility testing and stability testing to grow explosively. Not only does it need to cover compatibility verification of different hardware configurations, operating system versions and network environments, but it also needs to cope with boundary scenarios and extreme working condition testing under high-frequency functional updates. The traditional manual testing mode has highlighted the shortcomings of low efficiency, incomplete coverage and high error rate.

[0025] Currently, the related testing of such smart home devices mainly adopts the general scheme of "typical path sampling + pathed automatic script + manual supplementary testing": during testing, user high-frequency end-side paths and some key low-frequency paths are selected, the complete path steps are simulated through automatic tools, the device status of each node in the path is captured, and whether there is lag, link interruption or state disorder in the path is recorded. Finally, manual supplementary testing is required for small-path coverage and recovery logic after path abnormalities.

[0026] The existing scheme has three core technical problems:

[0027] 1. Incomplete coverage of end-side usage paths, multi-focus high-frequency paths, missing niche but critical paths, leading to untested failures in actual user usage;

[0028] 2. Low path adaptation efficiency, differences in end-side usage paths of new devices, requiring re-design of test scripts and path nodes, long adaptation period, difficult to match the rapid iteration pace of devices;

[0029] 3. Difficulty in locating path failures, multiple links in end-side paths, when a link is abnormal, it is difficult to distinguish whether it is a device itself problem, a path connection problem or a network fluctuation;

[0030] Incomplete coverage of end-side usage paths and low adaptation efficiency, unable to match the needs of user diversification and device rapid iteration, current most test solutions do not cover complete end-side path verification according to the standard, only focus on a single link in the path, still rely on manual supplement to check the full link of the path.

[0031] Therefore, the present application provides a smart home device testing system, method and related equipment, specifically, the smart home device testing system, method and related equipment of the embodiments of the present application are described below with reference to the accompanying drawings.

[0032] Figure 1 is a schematic diagram of a smart home device testing system according to an embodiment of the present application. It should be noted that the smart home device testing system of the embodiments of the present application can be applied to the smart home device testing method of the embodiments of the present application. The smart home device testing method can be configured on an electronic device or in a server. The embodiments of the present application do not limit this.

[0033] As shown in Figure 1 , the smart home device testing system comprises:

[0034] The offline server 110 is configured to collect path data of multi-dimensional end-side usage, pre-process and extract features of the path data, obtain a model feature vector, input the model feature vector to an offline model, obtain an offline test path library, deploy the offline test path library to a test terminal, and further configured to obtain data of un-covered target paths synchronized by the online server 120, perform incremental training on the data of un-covered target paths based on the offline model, and update the offline test path library.

[0035] The online server 120 is configured to collect user dynamic paths in real time, identify un-covered target paths from the user dynamic paths based on a double-dimensional comparison method, determine a test strategy based on the type of the un-covered target paths, and execute the test strategy, while recording the node state of the un-covered target paths in the process of executing the test strategy, and further configured to synchronize the data of the un-covered target paths to the offline server 110.

[0036] That is, the offline server 110 trains the offline model based on historical path data to generate an offline test path library. The online server 120 monitors new paths and intelligently explores in a real-time environment. The online server 120 identifies un-covered target paths and feeds back to the offline server 110 to drive incremental updates of the offline model, forming a self-evolving test system. The system combines static learning of the offline server and dynamic exploration of the online server to achieve high-coverage, high-efficiency, and low-cost testing of user usage paths.

[0037] In an embodiment of the present application, the sources of path data collection include end-side local logs, end-side App operation caches, and end-side intercommunication data packets within a local area network. The types of path data include path data of different device types, path data of different user scenarios, and abnormal scenario path data. The path data includes path nodes, node dependency relationships, path execution durations, and abnormal feedback information.

[0038] The path data of different device types includes, but is not limited to, path data of environmental sensors, execution devices, control terminals, and the like. For example, a linkage path in which an execution device is started and stopped when a sensor threshold is triggered.

[0039] The path data of different user scenarios includes, but is not limited to, simplified operation paths for the elderly and custom scenario paths for young people.

[0040] The abnormal scenario path data includes, but is not limited to, path data such as network disconnection and reconnection and device offline recovery.

[0041] The path nodes include, but are not limited to, code scanning and network configuration, instruction input, and linkage condition judgment.

[0042] The node dependency relationships include, but are not limited to, binding a device after successful network configuration and logging in after inputting a password.

[0043] The abnormal feedback information includes, but is not limited to, network configuration failure prompt pop-up windows and verification code input error prompt pop-up windows.

[0044] In an embodiment of the present invention, when the offline server 110 collects path data, it can remove identical path data from the path data to obtain path data after removal; based on template matching rules, it can determine the paths with missing nodes from the path data after removal, and fill in the missing nodes to obtain path data after filling in the missing nodes; and it can annotate the path data after filling in the missing nodes based on multiple dimensions to obtain an offline training dataset, wherein the multiple dimensions include path type, user usage frequency and fault impact range; it can extract target features from the offline training dataset and convert the target features into model feature vectors based on matrix operation tools.

[0045] This involves removing identical path data (e.g., the same user executing the command to turn on the living room light multiple times on the same day). The aim is to eliminate data redundancy and prevent the offline model from becoming overly biased towards high-frequency, simple paths.

[0046] Based on template matching rules, paths with missing nodes are identified from the removed path data, and these missing nodes are then filled in to obtain the path data with the missing nodes. In other words, different paths have corresponding nodes in the template. By matching the removed path data with the template, it can be determined whether there are missing nodes in the path, and then the missing nodes are filled in based on the template. For example, in a "device sharing" path, only the "initiate sharing" node and the "sharing successful" node are recorded, but the intermediate "select permissions (read-only / controllable)" node is missing. According to the path logic rule base (i.e., the template), this missing key node is identified and inserted. This ensures the logical integrity of the path.

[0047] In this process, after obtaining the path data with missing nodes filled in, the path with missing nodes filled in is manually annotated based on path type, user frequency, and fault impact range, thereby obtaining the offline training dataset.

[0048] Among them, the path type is normal or abnormal, the user usage frequency is high, medium or low, and the scope of the fault impact is core or secondary.

[0049] The user usage frequency is either high-frequency, medium-frequency, or low-frequency. Different types of user usage frequencies have different weights. For example, a high-frequency user has a weight of 0.5, a low-frequency user has a weight of 0.35, and a low-frequency user has a weight of 0.15.

[0050] The impact range of a fault is either core or secondary, meaning that different types of fault impact ranges have different weights. For example, if the impact range of a fault is core, the corresponding weight of the impact range is 0.8, and if the impact range of a fault is secondary, the corresponding weight of the impact range is 0.2.

[0051] In this process, given an offline training dataset, the target features for each path are extracted. This involves transforming the target features of each path into mathematical feature vectors that the machine can understand and compute—the model feature vectors. These target features include behavioral structure features, behavioral attribute features, device collaboration features, and abnormal behavior features.

[0052] Among them, behavioral structural features include, but are not limited to, path node sequences, key node locations, etc. (e.g., in the distribution network → binding → control, "binding" is a key node);

[0053] Among them, behavioral attribute features include, but are not limited to, path execution frequency and user dependence, such as "paths in the scenario of leaving home" having high frequency and high dependence;

[0054] Among them, device collaboration features include, but are not limited to, the number of interactions and command delays in multi-device paths, such as the number of interactions in "door lock → camera → light";

[0055] Among them, abnormal behavior characteristics include, but are not limited to, abnormal node types, recovery path length, etc., such as the number of recovery nodes after "reconnecting after network outage".

[0056] In an embodiment of the present invention, when the model feature vector is obtained, the model feature vector can be input into an offline model to obtain an offline test path library.

[0057] The offline model is a two-stage model, consisting of a first-stage model and a second-stage model. The first-stage model is trained as follows: a labeled dataset is obtained and divided into a training set and a validation set; the training set is input into the first-stage model for training, where the first-stage model calculates the predicted values ​​through forward propagation, calculates the error between the predicted values ​​and the true labels using a loss function, and dynamically adjusts the parameters of the offline model using a backpropagation algorithm and an optimizer to minimize the loss function; the validation set is input into the first-stage model for validation, and business metrics on the validation set are obtained, and the parameters of the first-stage model are adjusted based on the business metrics.

[0058] The first-stage model can be an improved LSTM model. That is, the traditional LSTM model is good at processing sequential data (such as path node sequences), but it has a large number of parameters. This application improves upon this model by adapting to the limited computing power of offline servers, including but not limited to reducing the number of hidden layer units and using techniques such as grouped convolution.

[0059] In other words, an improved LSTM model is used as the first-stage model. LSTM excels at processing sequential data (such as path node sequences), and "lightweight improvement" refers to reducing network parameters to adapt to the limited computing power of the local server. For example, during training: over 50,000 labeled local historical path data points are divided into training and validation sets. The model parameters are adjusted inversely using a loss function (calculating the error between the model's predicted "completeness / relevance" and the manually labeled true values). Minimizing the loss improves the model's accuracy in identifying path features. After training, the model becomes an expert system capable of "scoring" any input path.

[0060] The process involves inputting the model feature vectors into the first-stage model to obtain path integrity and path user relevance metrics. These metrics are then input into the second-stage model, which uses a path reward score (α * path coverage reward + β * test efficiency reward) to generate a first test path library, where α represents the weight of path coverage and β represents the weight of test efficiency. A path priority score is then generated for each path in the first test path library based on a path priority score (path user dependence weight × path failure impact range weight). Finally, the path priority scores of each path are sorted, and the sorted first test path library is used as the offline test path library.

[0061] The path priority score of each path can be sorted from highest to lowest.

[0062] The path coverage reward is calculated as Σ(path value * path integrity score). This means that for each new path i added to the library, the "novelty value" it brings is diminished by its integrity score I_i. If a path is incomplete (low I_i), even if it appears novel, its contribution to coverage will be reduced or even zero (if an integrity threshold is set). This ensures that all generated paths are executable and logically complete.

[0063] The test efficiency reward is calculated as follows: Σ(Weight of path's user dependency * Weight of path's failure impact range * Priority factor). In other words, the test efficiency reward is calculated as Σ(Path priority score * Priority factor).

[0064] The user dependence of the path is high, medium or low, and the weights corresponding to different levels of user dependence are not the same. For example, the weight of user dependence is 0.5 for high user dependence, 0.3 for medium user dependence, and 0.2 for low user dependence.

[0065] For example, taking the "smart door lock temporary password authorization path" (niche critical path) of smart home devices as an example, the offline model training process is as follows: ① Extract 1000+ path data of this type from the local database, preprocess and extract features from the path data to obtain the model feature vector. Among them, after preprocessing, key nodes ("enter password settings → select temporary password → set validity period → generate password → share password") and abnormal features ("validity period setting too long prompt") can be labeled; the extracted feature vector includes: the behavioral structure feature is "5 nodes, the 3rd node (setting validity period) is the critical node", and the behavioral attribute feature is "low execution frequency, user dependence". The device collaboration feature is "only the door lock interacts with the APP, without multi-device linkage"; ② The model feature vector is input into the improved LSTM model for training, and multiple iterations are used to improve the model's accuracy in recognizing path features; ③ A test path is generated based on the reinforcement learning algorithm (second-stage model): simulating the entire process of "entering password settings → setting a 1-hour validity period → generating a password → sharing with visitors → visitors entering the password to unlock → verifying the unlocking record and synchronizing it with the APP". At the same time, the priority of this path is set to "medium", which takes precedence over the "door lock bell adjustment path" (low priority) for testing. The entire training process is completed on the offline server 110 without the need to connect to the network to call cloud resources.

[0066] Existing offline testing models suffer from two major shortcomings: first, feature extraction is limited (often focusing only on node sequences and ignoring device collaboration and abnormal features), resulting in low path recognition accuracy (typically below 80%); second, the network structure is complex (numerous parameters and high computational requirements), requiring cloud-based training and making them susceptible to network fluctuations. This application optimizes the offline model through targeted design: ① Multi-feature fusion improves path recognition accuracy to over 90%, accurately distinguishing between "high-urgency fault paths" and "low-value custom paths," avoiding misjudgments; ② Lightweight structure + local training reduces cloud resource consumption by 80%, and the consistency fluctuation range of test results is controlled within ±3% (far superior to the ±10% fluctuation of cloud models), adapting to diverse scenarios such as weak network labs and offline testing.

[0067] In an embodiment of the present invention, the trained offline model and offline test path library are deployed to the test terminal, and the offline automation tool is invoked to automatically execute tests according to path priority: simulating user operation to complete the entire path on the terminal side (such as the path in the scenario of leaving home: turn off the air conditioner → turn on the camera → lock the door), capturing the status of each node device in real time (command response time, linkage result, abnormal prompts), and storing it in the local test result library; for failed paths (such as "locking the door timed out"), the failed nodes and associated devices are automatically marked, and an offline test report (such as path execution success rate, abnormal node distribution, etc.) is generated.

[0068] In embodiments of the present invention, test results (e.g., new paths not covered, failed path nodes) are periodically added to the offline test path library, the offline model training process is restarted (no need to add manual annotations, the model automatically reuses historical feature logic), the path generation accuracy is optimized, and a local closed loop of "data collection-offline training-test execution-result feedback" is achieved.

[0069] In an embodiment of the present invention, an online server 120 is used to collect user dynamic paths in real time and identify uncovered target paths from user dynamic paths based on a two-dimensional comparison method.

[0070] Among them, real-time collection of user dynamic paths includes dynamic path data in actual user use (such as new scene paths, new operation paths after device firmware updates), and the dimensions of dynamic path data include path node sequence, execution frequency, and device collaboration relationship.

[0071] Specifically, the user's dynamic path is compared in a structured manner with the test paths in the offline test path library to obtain the uncovered new paths; features are extracted from the uncovered new paths, and the extracted features are evaluated to obtain the uncovered target paths based on the evaluation results.

[0072] Feature extraction for uncovered new paths includes identifying their urgency, complexity, and potential user needs. For example, paths handling device fault alarms are considered high-urgency; paths with ≥5 nodes are considered high-complexity; and recently frequently appearing "simplified paths specifically for the elderly" are considered high-complexity.

[0073] In other words, the online server 120 collects two types of data in real time: the offline test path library and the real-time user dynamic path. By comparing the real-time user dynamic path with the path in the offline test path library in a structured manner, new uncovered paths are obtained. Feature extraction is performed on the new uncovered paths, and the extracted features are evaluated. Based on the evaluation results, the uncovered target path is obtained.

[0074] Structured comparison includes, but is not limited to, comparing node sequences, key node locations, and device collaboration relationships.

[0075] In an embodiment of the present invention, when an uncovered target path is obtained, a testing strategy can be determined based on the type of the uncovered target path.

[0076] This involves obtaining a rule base for path types and test strategies, including test strategies corresponding to path types; and matching the types of target paths not covered with the rule base to obtain the test strategies corresponding to the types of target paths not covered.

[0077] For example, for paths of type "high urgency + high user demand" (such as "device offline alarm → automatic reconnection → user notification"), a "full node deep exploration" strategy is adopted: the operation is simulated sequentially according to the node dependency relationship (such as "alarm → reconnection → notification") to ensure that the interaction logic of each link is covered.

[0078] For paths of type "medium complexity + low frequency" (such as "device historical data export → cloud backup → local download"), the "key node priority exploration" strategy is adopted: focus on core nodes (such as "cloud backup") and simplify the testing steps of non-key nodes (such as "export format selection").

[0079] For "new morphology variation paths" (such as adding nodes to traditional paths), an "incremental exploration" strategy is adopted: only test the connection logic between the new node (such as "Bluetooth assisted network configuration") and the original node (such as whether "Bluetooth network configuration → device binding" is compatible), to avoid repeatedly testing existing nodes;

[0080] In an embodiment of the present invention, when an uncovered target path is obtained, a task value score for the uncovered target path is obtained based on the following formula: Task Value Score = Weight of the urgency of the uncovered target path × Frequency of user demand for the uncovered target path × Weight of the fault impact range of the uncovered target path. The uncovered target paths are then ranked based on their task value scores. This ensures that high-value uncovered paths are explored first.

[0081] The urgency level is categorized as high, medium, or low, with different weights corresponding to different urgency levels. For example, high urgency corresponds to a user dependency weight of 0.6, medium urgency to 0.3, and low urgency to 0.1.

[0082] For example, regarding the newly emerging "smart curtain and wind and rain sensor linkage path" (not covered by the offline model), the online server 120 analysis process is as follows: ① Real-time monitoring reveals a new path in the user operation log: "Wind and rain sensor detects rainfall → curtains automatically close → APP push notification"; ② Comparison with the offline test path library confirms that this path is completely uncovered, and the characteristics of the new path are extracted: high urgency (real-time response required in rainfall scenarios), medium complexity (3 nodes), and a recent 30% increase in user demand frequency, thus identifying this new path as an uncovered target path; ③ Based on this uncovered target path, the corresponding testing strategy can be determined as the "full-node deep exploration" strategy, which simulates the entire process of "wind and rain sensor triggers rainfall signal → curtains receive command and close → APP receives status synchronization and pushes notification", testing the response latency of each node (e.g., whether the command transmission from the sensor to the curtain is ≤1 second) and anomaly handling (e.g., whether a fault is reported when the curtain freezes); ④ The priority of this uncovered target path is set to "highest", and it is inserted at the front of the current test queue for priority execution.

[0083] In embodiments of the present invention, by calling the device control interface to send instructions (such as triggering the wind and rain sensor or controlling the curtain action), the device status (response results, log feedback) and abnormal information (such as lost instructions or asynchronous status) of each node are captured and synchronously recorded in the online result database; new paths discovered during the exploration (such as "curtain closing failure → automatic retry → notify user") are automatically included in the uncovered path pool and a second exploration is initiated.

[0084] In embodiments of the present invention, uncovered path data (including features and exploration results) can be synchronized to the offline test path library daily or at preset time intervals, triggering incremental training of the offline model (only updating new path features, without full retraining), updating the offline test path library, and realizing a closed loop of "online exploration completion → offline model iteration → online strategy optimization". Continuous optimization is achieved through an iterative mechanism: data interoperability avoids the problems of "outdated offline test path library and lack of basis for online exploration", improving the synchronization between test paths and real user scenarios; ② incremental training shortens the offline model update cycle from "7 days of full retraining" to "1 day of incremental update", adapting to the "weekly iteration" testing rhythm of smart home devices, and the model accuracy can be steadily improved with long-term use.

[0085] In embodiments of the present invention, the offline model covers most static existing paths, while the online module shortens the time for dynamically added paths. The collaboration between the two improves overall path coverage and reduces the incidence of "untested faults" in actual user use. The offline model is trained locally to avoid the impact of cloud fluctuations, and the consistency of test results is within a controllable range. The online-offline collaboration reduces the need for cloud data transmission and computation, while also reducing the workload of manual testing and lowering the overall testing cost. It can stably adapt to diverse testing environments such as weak network labs and offline scenarios. The offline model ensures accurate coverage of users' long-term habitual paths, while the online module tracks users' new scenario needs in real time, improving the synchronization between test paths and users' actual usage paths, and providing more realistic feedback data for device function optimization.

[0086] In an embodiment of the present invention, the online server 120 constructs a precise and efficient mechanism for exploring uncovered paths: ① At the identification level, through a two-dimensional approach of "structured comparison (comparing path node sequences / device collaboration relationships) + feature matching (associating with an offline test path library)," "completely new paths" and "mutated paths" are accurately marked; ② At the exploration level, a hierarchical strategy is generated based on "urgency + complexity + user needs": high-urgency paths employ "full-node deep exploration," medium-to-low complexity paths employ "key node priority exploration," and new mutated paths employ "incremental exploration." The two-dimensional identification improves the discovery rate of uncovered paths and can capture subtle mutated paths resulting from "node additions / removals"; the hierarchical exploration strategy improves the efficiency of exploring high-value paths and reduces resource consumption for low-value paths, avoiding the drawbacks of traditional solutions that prioritize quantity over value.

[0087] According to an embodiment of the present invention, the testing system for smart home devices constructs a dual-module collaborative architecture of "offline foundation-online completion": the offline server learns all static edge path features (including high-frequency paths and low-frequency critical paths) and generates a standardized test path library through a lightweight LSTM + reinforcement learning model; the online server collects dynamic path data in real time, identifies uncovered paths and actively explores them through a "dual-dimensional comparison + hierarchical exploration strategy"; the two achieve a closed loop through "online data feedback - offline incremental training", forming a testing system of "full coverage of static existing paths + real-time completion of dynamic incremental paths". The system achieves a dual breakthrough through the "collaborative architecture": ① It solves the dual blind spots of "static missed tests and dynamic lag", with the offline server ensuring more than 90% static path coverage and the online server reducing the dynamic path identification delay from 24 hours to within 1 hour, achieving an overall path coverage of 98%; ② It avoids the waste of resources in a single module, with the offline server undertaking core feature learning (reducing online computing power consumption) and the online server focusing on high-value uncovered paths (reducing ineffective exploration), resulting in a 60% improvement in overall testing efficiency compared to purely offline / purely online solutions.

[0088] In an embodiment of the present invention, a testing method for smart home devices is also proposed, such as... Figure 2 As shown, the testing method for this smart home device includes:

[0089] S210 collects multi-dimensional path data used on the edge, preprocesses and extracts features from the path data to obtain model feature vectors, and inputs the model feature vectors into the offline model to obtain the offline test path library.

[0090] S220 collects user dynamic paths in real time, identifies uncovered target paths from user dynamic paths based on a two-dimensional comparison method, determines a test strategy based on the type of uncovered target paths, executes the test strategy, and simultaneously synchronizes the data of uncovered target paths to the offline model so that the offline model can perform incremental training on the data of uncovered target paths, while updating the offline test path library.

[0091] According to the testing method for smart home devices of the present invention, the method combines static learning on an offline server with dynamic exploration on an online server to achieve high coverage, high efficiency, and low cost testing of user usage paths.

[0092] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0093] According to one aspect of the present invention, an electronic device is provided.

[0094] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Figure 3 As shown, an electronic device may include one or more ( Figure 3 Only one is shown in the image. A processor 102 (which may include, but is not limited to, a microprocessor unit (MPU) or a programmable logic device (PLD)) and a memory 104 for storing data are also shown. In one exemplary embodiment, the electronic device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that...Figure 3 The structure shown is for illustrative purposes only and does not limit the structure of the terminal device described above. For example, the terminal device may also include components that are more... Figure 3 The more or fewer components shown, or having the same Figure 3 Equivalent functions or ratios shown Figure 3 The functions shown have more different configurations.

[0095] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the testing method for smart home devices in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the aforementioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to terminal devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0096] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the switching device. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0097] This invention proposes a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute a test method for smart home devices.

[0098] The applicant of this invention has provided a detailed description of the embodiments of the invention in conjunction with the accompanying drawings. However, those skilled in the art should understand that the above embodiments are merely preferred embodiments of the invention. The detailed description is only intended to help readers better understand the spirit of the invention and is not intended to limit the scope of protection of the invention. On the contrary, any improvements or modifications made based on the inventive spirit of the invention should fall within the scope of protection of the invention.

[0099] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0100] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A testing system for smart home devices, characterized in that, include: An offline server is used to collect multi-dimensional path data used on the client-side, preprocess and extract features from the path data to obtain model feature vectors, input the model feature vectors into an offline model to obtain an offline test path library, and deploy the offline test path library to the test terminal. It is also used to obtain data of uncovered target paths synchronized by the online server, perform incremental training on the data of uncovered target paths based on the offline model, and update the offline test path library at the same time; The online server is used to collect user dynamic paths in real time, identify uncovered target paths from the user dynamic paths based on a two-dimensional comparison method, determine a test strategy based on the type of the uncovered target paths, execute the test strategy, and simultaneously record the node status of the uncovered target paths during the execution of the test strategy. It is also used to synchronize the data of the uncovered target path to the offline server.

2. The testing system for smart home devices according to claim 1, characterized in that, The sources of the path data collection include local logs on the device side, operation cache of the device side app, and communication data packets between devices within the local area network. The types of path data include path data of different device types, path data of different user scenarios, and path data of abnormal scenarios. The path data includes path nodes, node dependencies, path execution time, and abnormal feedback information.

3. The testing system for smart home devices according to claim 1, characterized in that, The path data is preprocessed and feature extracted to obtain a model feature vector, including: Remove duplicate path data from the path data to obtain the removed path data; Based on template matching rules, paths with missing nodes are identified from the removed path data, and the missing nodes are filled in to obtain path data with the missing nodes filled in. The path data with the missing nodes filled in is then labeled based on multiple dimensions to obtain an offline training dataset. The multiple dimensions include path type, user frequency of use, and scope of fault impact. Target features are extracted from the offline training dataset and transformed into the model feature vector using a matrix operation tool. The target features include behavioral structure features, behavioral attribute features, device collaboration features, and abnormal behavior features.

4. The testing system for smart home devices according to claim 1, characterized in that, The offline model is a two-stage model, which includes a first-stage model and a second-stage model. The first-stage model is trained in the following way: Obtain the labeled dataset and divide it into a training set and a validation set; The training set is input into the first-stage model for training. The first-stage model calculates the predicted value through forward propagation, calculates the error between the predicted value and the true label through a loss function, and dynamically adjusts the parameters of the offline model using a backpropagation algorithm and an optimizer to minimize the loss function. The validation set is input into the first-stage model for validation to obtain business metrics on the validation set, and the parameters of the first-stage model are adjusted based on the business metrics. The offline test path library is obtained by inputting the model feature vector into the offline model, including: The model feature vector is input into the first stage model to obtain the path integrity index and the path user relevance index; The completeness index and user relevance index of the path are input into the second stage model. The second stage model obtains the first test path library based on the path reward score = α * path coverage reward + β * test efficiency reward, where α represents the weight of path coverage and β represents the weight of test efficiency. The path priority score is obtained for each path in the first test path library based on the formula: path priority score = weight of user dependence on path × weight of path failure impact range. The path priority scores of each path are sorted, and the sorted first test path library is used as the offline test path library.

5. The testing system for smart home devices according to claim 1, characterized in that, Identifying the uncovered target path from the user's dynamic path based on a two-dimensional comparison method includes: The user's dynamic path is compared in a structured manner with the test paths in the offline test path library to obtain the new paths that are not covered. Feature extraction is performed on the uncovered new path, and the extracted features are evaluated. The uncovered target path is obtained based on the evaluation results.

6. The testing system for smart home devices according to claim 1, characterized in that, The testing strategy is determined based on the type of the uncovered target path, including: A rule base for obtaining path types and test strategies is provided, wherein the rule base includes the test strategies corresponding to the path types. The type of the uncovered target path is matched with the rule base to obtain the test strategy corresponding to the type of the uncovered target path.

7. The testing system for smart home devices according to claim 1, characterized in that, Given the uncovered target path, the task value score of the uncovered target path is obtained based on the following formula: Task value score = weight of urgency of the uncovered target path × frequency of user demand for the uncovered target path × weight of fault impact range of the uncovered target path. The uncovered target paths are then sorted based on their task value scores.

8. A testing method for smart home devices, characterized in that, include: Collect multi-dimensional path data used on the edge, preprocess and extract features from the path data to obtain model feature vectors, and input the model feature vectors into the offline model to obtain an offline test path library; The system collects user dynamic paths in real time, identifies uncovered target paths from these paths using a two-dimensional comparison method, determines a testing strategy based on the type of the uncovered target paths, executes the testing strategy, and simultaneously synchronizes the data of the uncovered target paths to the offline model so that the offline model can perform incremental training on the data of the uncovered target paths, while also updating the offline test path library.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of claim 8.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to claim 8.

Images