A device testing method, apparatus, electronic device, and storage medium

Abstract

This invention discloses a device testing method, apparatus, electronic device, and storage medium. The invention can acquire device status information; retrieve puncture simulation scenarios from a puncture library based on the device status information; simulate punctures according to the puncture simulation scenarios to perform puncture scenario testing on the device; and inspect the device that has completed the puncture scenario testing to obtain the test results of the device's online capability. This invention automates device testing, accurately completing various puncture scenario simulation tests. While ensuring sufficiently large and refined test coverage, it improves testing efficiency. Furthermore, when device failure occurs later, the device testing method of this invention can locate the corresponding fault point based on the corresponding puncture simulation scenario. This invention not only achieves automated testing, reducing labor costs, but also improves testing efficiency and reduces repair costs after device failure.

Description

Technical Field

[0001] This application relates to the field of computers, specifically to a device testing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Staying online is a crucial indicator of a smart device's quality and its ability to maintain a stable communication link with the cloud. This ensures the device can promptly receive user control commands and report device status data. Currently, testers typically analyze the percentage of time a device is online within a given period to assess its online staying power.

[0003] Because there are many test scenarios for the ability of devices to stay online, each test scenario requires manual monitoring for a long time, and each test scenario also requires simulating the occurrence of corresponding glitches, the testing efficiency is low. Summary of the Invention

[0004] This application provides a device testing method, apparatus, electronic device, and storage medium to improve device testing efficiency.

[0005] This application provides a device testing method, the method comprising:

[0006] Obtain device status information;

[0007] Based on the device's status information, a glitch simulation scenario is obtained from the glitch library. The glitch library stores glitch simulation scenarios, and the glitch in the glitch simulation scenario is an unexpected signal received by the device during signal processing.

[0008] Based on the burr simulation scenario, burrs are simulated, and burr scenario tests are performed on the equipment;

[0009] The equipment that has completed the glitch scenario test is inspected to obtain the test results of the equipment's online capability. The online capability is used to characterize the equipment's ability to normally handle unexpected signals.

[0010] This application embodiment also provides a device testing apparatus, which includes an information acquisition module, a burr scene acquisition module, a burr scene testing module, and a device detection module;

[0011] The information acquisition module is used to acquire the device's status information;

[0012] The glitch scene acquisition module is used to obtain glitch simulation scenes from the glitch library based on the device's status information. The glitch library stores glitch simulation scenes, and the glitch in the glitch simulation scene is an unexpected signal received by the device during signal processing.

[0013] The burr scenario testing module is used to simulate burrs based on burr simulation scenarios and perform burr scenario testing on the device.

[0014] The equipment testing module is used to test the equipment that has completed the puncture scenario test and obtain the test results of the equipment's online capability. The online capability is used to characterize the equipment's ability to normally handle unexpected signals.

[0015] In some embodiments, the apparatus further includes a device network configuration module, which is used to send a trigger command to the device so as to trigger the device to enter the network configuration mode and connect to the network.

[0016] In some embodiments, the device distribution module includes a distribution submodule, which is used for:

[0017] When the device is a remote control device, a remote control signal is sent to the device through a simulated remote control so that the device can enter the network distribution mode;

[0018] When the device is a panel-controlled device, the buttons on the device panel are operated by an automatic robot so that the device can enter the network distribution mode;

[0019] When the device is difficult to control via a panel, use a serial port tool to connect to the device's WiFi module and send a network configuration command to the device so that the device can execute the network configuration command through the debugging bridge service and thus enter the network configuration mode.

[0020] In some embodiments, the information acquisition module includes an information acquisition first module, which is used for:

[0021] The app associated with the device is opened using an automated testing framework;

[0022] The device interface of the APP is used to perform character recognition by character recognition methods to obtain offline character recognition information;

[0023] Read device status values ​​via XPath;

[0024] The offline character recognition information and the device status value are used as the status information of the device.

[0025] In some embodiments, the information acquisition module includes a second information acquisition module, which is used for:

[0026] Call the cloud interface to read the status information reported by the device to the cloud.

[0027] In some embodiments, the information acquisition module includes a third information acquisition module, which is used for:

[0028] Based on the log interface, and using serial port tools to capture device logs, the device status information is obtained from the device logs.

[0029] In some embodiments, the puncture scene acquisition module includes a judgment submodule, which is used to acquire a puncture simulation scene from the puncture library when the state information meets a preset information condition and the strategy timer meets a preset duration.

[0030] In some embodiments, the glitch simulation scenarios include network glitch simulation scenarios, power glitch simulation scenarios, and / or system glitch simulation scenarios; the network glitch simulation scenario simulates glitch scenarios under network failure conditions; the power glitch simulation scenario simulates glitch scenarios under power supply failure conditions; and the system glitch simulation scenario simulates glitch scenarios under system resource failure conditions.

[0031] In some embodiments, the glitch scenario testing module includes a network glitch simulation module, which is used for:

[0032] When the glitch simulation scenario is a glitch scenario simulating a network outage, an automated script connects to the router's backend and cancels the WAN port connection on the router's backend.

[0033] When the glitch simulation scenario is a glitch scenario where the simulated signal disappears, an automated script connects to the router's backend and turns off the WiFi hotspot signal on the router's backend.

[0034] When the glitch simulation scenario is a glitch scenario simulating a weak network condition, an automated script connects to the router backend and limits the network speed or sets a packet loss rate higher than the packet loss rate threshold on the router backend.

[0035] In some embodiments, the glitch scenario testing module includes a power glitch simulation module, which is used for:

[0036] When the glitch simulation scenario is a glitch scenario simulating a power failure, the controllable instrument simulates the equipment being in different voltage change states or a power failure state.

[0037] In some embodiments, the burr scenario testing module includes a system burr simulation module, which is used for:

[0038] When the glitch simulation scenario is a glitch scenario simulating system resource failure, the cloud requests the device to report status information under high-frequency request conditions and / or sends control commands to the device under high-frequency delivery conditions, so that the device is in the business logic of high-frequency operation, thereby simulating glitch under system resource failure conditions; the high-frequency request condition is that the number of requests sent by the cloud to the device within a preset request duration is greater than the request count threshold; the high-frequency delivery condition is that the number of commands sent by the cloud to the device within a preset delivery duration is greater than the delivery count threshold.

[0039] In some embodiments, the device detection module includes a stage identifier detection module, a status information detection module, and an instruction verification detection module;

[0040] The stage identifier detection module is used to acquire the stage identifier generated by the device during burr scenario testing, and to detect the stage identifier. When a stage identifier is detected to be missing, the device is determined to be in an abnormal state when processing burrs. The stage identifier is used to characterize the various stages of the device during operation and the processing results of different stages.

[0041] The status information detection module is used to detect the status information of the device. When it detects that the status information reported by the device to the cloud is missing, it determines that the status of the device in processing glitch is abnormal.

[0042] The instruction verification and detection module is used to verify the function instructions of the device. When it is detected that the device does not respond according to the function instructions, the device is determined to be in an abnormal state of glitch processing.

[0043] This application also provides an electronic device, including a processor and a memory. The memory stores multiple instructions, and the processor loads the instructions to execute the steps in any of the device testing methods provided in this invention.

[0044] This application also provides a storage medium storing multiple instructions adapted for loading by a processor to execute steps in any of the device testing methods provided in this invention.

[0045] This invention can acquire device status information; retrieve glitch simulation scenarios from a glitch library based on the device status information (the glitch library stores glitch simulation scenarios, where glitch represents unexpected signals received by the device during signal processing); simulate glitch based on the glitch simulation scenarios to perform glitch scenario testing on the device; and then inspect the device that has completed the glitch scenario testing to obtain the test result of the device's online capability, which characterizes the device's ability to properly handle unexpected signals. Thus, this invention establishes a glitch library storing various glitch simulation scenarios. During device testing, it can automatically retrieve the corresponding glitch simulation scenario from the glitch library, perform glitch scenario testing on the device based on the glitch simulation scenario, and then determine whether the device can correctly handle glitch by inspecting the device that has completed the glitch scenario testing.

[0046] This invention automates equipment testing, enabling precise simulation of various burr scenarios and ensuring comprehensive coverage of online testing scenarios. While maintaining sufficient and detailed testing coverage, it saves labor costs and further improves testing efficiency. Furthermore, when equipment malfunctions later, the invention's testing method can pinpoint the corresponding fault location based on the simulated burr scenarios. This invention not only automates testing and reduces labor costs but also improves testing efficiency and lowers repair costs after equipment failure. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic diagram of a scenario for the equipment testing system provided in an embodiment of this application;

[0049] Figure 2 This is a schematic diagram of the first type of device testing method provided in the embodiments of this application;

[0050] Figure 3 This is a schematic diagram of the second process of the device testing method provided in the embodiments of this application;

[0051] Figure 4 This is a schematic diagram of the structure of the equipment testing device provided in the embodiments of this application.

[0052] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] This application provides a device testing method, apparatus, electronic device, and storage medium.

[0055] Specifically, the testing device can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet, smart Bluetooth device, laptop, or personal computer (PC); the server can be a single server or a server cluster consisting of multiple servers.

[0056] In some embodiments, the device testing apparatus may also be integrated into multiple electronic devices, such as multiple servers, with multiple servers implementing the device testing method of this application.

[0057] In some embodiments, the server may also be implemented as a terminal.

[0058] For example, taking the integration of a device testing apparatus into an electronic device as an example, the electronic device can acquire the device's status information; retrieve glitch simulation scenarios from a glitch library based on the device's status information. The glitch library stores glitch simulation scenarios, and the glitch in the glitch simulation scenarios represents unexpected signals received by the device during signal processing; simulate glitch based on the glitch simulation scenarios to perform glitch scenario testing on the device; and inspect the device that has completed the glitch scenario testing to obtain the test results of the device's online capability. The online capability is used to characterize the device's ability to normally process unexpected signals, etc.

[0059] Please see Figure 1 , Figure 1 This is a schematic diagram of a device testing system provided in an embodiment of this application. The system may include a test server 10, a user terminal 11, a cloud 12, and a smart device 13. The test server 10, user terminal 11, cloud 12, and smart device 13 communicate with each other, which will not be described in detail here.

[0060] The user terminal 11 includes, but is not limited to, tablet computers, laptops, personal computers (PCs), microprocessor boxes, or other devices. The user terminal 11 can run a smart device application, which provides a graphical user interface (GUI) to display device status information. This GUI can also provide different behavioral controls, including click controls and click-to-view controls. The smart device 13 can respond to the triggering of these behavioral controls. The cloud 12 includes cloud servers. The smart devices 13 include smart TVs in the living room, smart refrigerators in the kitchen, smart door locks in the entryway, water immersion sensors, smoke sensors, or cameras. The test server 10 includes processors and memory.

[0061] It should be noted that, Figure 1 The system scenario diagram shown is merely an example. The test server and scenario described in this application embodiment are for the purpose of more clearly illustrating the technical solutions of this application embodiment and do not constitute a limitation on the technical solutions provided by this application embodiment. As those skilled in the art will know, with the evolution of systems and the emergence of new business scenarios, the technical solutions provided by this application embodiment are also applicable to similar technical problems. Detailed descriptions are provided below. It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of embodiments.

[0062] This disclosure primarily aims to provide a device testing method. The method involves acquiring device status information; retrieving glitch simulation scenarios from a glitch library based on the device status information (the glitch library stores glitch simulation scenarios where glitch signals are unexpected signals received by the device during signal processing); simulating glitch signals based on the glitch simulation scenarios to perform glitch scenario testing on the device; and then inspecting the device after completing the glitch scenario test to obtain the test result of the device's online capability. Therefore, this solution achieves automated testing and improves device testing efficiency.

[0063] like Figure 2 As shown, the specific process of this equipment testing method can be as follows:

[0064] 110. Obtain device status information.

[0065] The device status information includes whether the device has been successfully configured to the network, the duration of online activity, whether it is currently offline, whether the device has been restarted, whether the device configuration has been modified, whether the network egress status is stable, and whether the device port status is stable.

[0066] For example, by combining the device's product logs, we can obtain the device's status information, which includes information such as successful network configuration, online for 3 hours, and currently online.

[0067] In some embodiments, before obtaining the device status information, the following steps are included:

[0068] Send a trigger command to the device so that it can enter the network configuration mode and connect to the network.

[0069] Before testing the equipment, it is necessary to put the equipment into network configuration mode so that the equipment can be connected to the network and maintain a normal communication link between the equipment and the cloud. This allows the equipment to receive control commands or report equipment status data in a timely manner, so that subsequent glitch scenario testing can be carried out.

[0070] Network configuration modes include direct configuration and intelligent configuration. Direct configuration involves transmitting the SSID (Service Set Identifier) ​​and password directly to the device's WiFi module via a host interface (including UART serial port, SPI port, or SDIO port) according to a communication protocol. Upon receiving the SSID and password, the WiFi module connects to the WiFi hotspot signal or router and returns the connection result from the host interface. Intelligent configuration uses the user terminal's WiFi hotspot signal. At the MAC layer, the SSID and password are filled into the unencrypted header of the MAC packet according to a specific protocol format. The SSID and password are then transmitted in segments from the user terminal to the device's WiFi module using broadcast and packet capture methods.

[0071] In some embodiments, sending a trigger command to the device includes:

[0072] When the device is a remote control device, a remote control signal is sent to the device through a simulated remote control so that the device can enter the network distribution mode;

[0073] When the device is a panel-controlled device, the buttons on the device panel are operated by an automatic robot to enable the device to enter the network distribution mode;

[0074] When the device is difficult to control via a panel, use a serial port tool to connect to the device's WiFi module and send a network configuration command to the device so that the device can execute the network configuration command through the debugging bridge service and thus enter the network configuration mode.

[0075] Remote control devices include infrared, Bluetooth, and wireless remote control devices; easy-to-control devices are those with easily operable panel buttons; difficult-to-control devices are those with relatively difficult-to-operate panel buttons. The serial port tool is used to receive connection requests from the server, send connection commands to the device's communication module via the serial port, receive connection data returned by the communication module via the serial port, and output the connection data to the server. The debug bridge is a window for inputting command lines; the debug bridge service is used for information exchange between the device and the server.

[0076] This invention employs different network configuration methods based on the characteristics of different devices. For example, devices operated via simulated remote controls, such as air conditioners, televisions, and clothes dryers, can be configured by sending remote control signals to the device through the simulated remote control. Devices operated via touch buttons, such as refrigerators, washing machines, and door locks, can be configured by having an automated robot operate the buttons on the device panel. Devices with difficult button operation, such as water immersion sensors, smoke sensors, and cameras, can be configured by connecting to the device's WiFi module using a serial port tool and sending a network configuration command to the device. The device can then execute the network configuration command through the debugging bridge service.

[0077] In some embodiments, obtaining device status information includes:

[0078] Open the app associated with the device through an automated testing framework;

[0079] The device interface of the APP is used to perform character recognition by character recognition methods to obtain offline character recognition information;

[0080] Read device status values ​​via XPath;

[0081] Use offline character recognition information and device status values ​​as the device's status information.

[0082] In this embodiment of the application, the automated testing framework can be the Airtest automated testing framework, the Appium automated testing framework, or other types of automated testing frameworks.

[0083] Character recognition methods can be optical character recognition (OCR), component tree character recognition, or other methods. When using component tree character recognition, the device interface image of the app is first acquired. Connected components in the device interface image are extracted using a component tree. Simultaneously, a classifier is built based on the glyph features of the characters. The positive samples output by the classifier are scaled uniformly and then input into a Hough forest. Each character is treated as a separate category, and character recognition is completed during the detection process. In this embodiment, connected components are obtained by building a component tree; a set of features is calculated for each connected component and input into the classifier for recognition; the positive samples output by the classifier are used as candidate character regions and scaled to the same size; each candidate character region is input into the Hough forest, which outputs the probability distribution of the character; based on the posterior probability output by the Hough forest, the position of the character is estimated, and the text result is output.

[0084] XPath, short for XML (a subset of Standard Generalized Markup Language), is a language used to locate specific parts of an XML document. Based on the tree structure of XML, XPath has different types of nodes. It uses path expressions to select nodes or sets of nodes within an XML document; an XPath path expression is a written sequence of steps from one XML node (the current context node) to another node or a group of nodes.

[0085] This application embodiment automatically opens the APP associated with the device through an automated testing framework; at the same time, it performs character recognition on the device interface of the APP using a character recognition method to analyze whether the device interface has characters such as "offline", thereby obtaining offline character recognition information; then it reads the device status value through the XPath path, and then uses the offline character recognition information and the device status value as the device status information.

[0086] In some embodiments, obtaining device status information includes:

[0087] Call the cloud interface to read the status information reported by the device to the cloud.

[0088] In this embodiment of the application, the cloud interface is used to communicate with the device. The test server can call the cloud interface to read the status information reported by the device to the cloud.

[0089] In some embodiments, obtaining device status information includes:

[0090] Based on the log interface, and using serial port tools to capture device logs, the device status information is obtained from the device logs.

[0091] The log interface is used to record the device's execution operation information into a log file.

[0092] In routine network maintenance, device logs are frequently used to troubleshoot and locate device faults. For example, if a network is interrupted for a period of time and then automatically recovers, troubleshooting can be done by checking whether the device has been restarted, whether its configuration has been modified, whether the network egress status is stable, and whether the device port status is stable. Additionally, the real-time status of the device can be viewed using status query commands. If the real-time status is normal, the device's historical records can be viewed through the device logs.

[0093] In this embodiment of the application, the test server can use a log interface and a serial port tool to capture relevant device logs in the log file, and obtain the device status information based on the device logs.

[0094] 120. Obtain burr simulation scenarios from the burr library based on the device status information. The burr library stores burr simulation scenarios.

[0095] In the glitch simulation scenario, glitches are unexpected signals received by the device during signal processing. These glitches can affect the performance of the device and affect its normal operation.

[0096] For example, the equipment may be affected by glitches such as network outages or weak networks during normal operation; or the equipment may be affected by glitches such as sudden voltage changes or power outages during normal operation.

[0097] In some embodiments, obtaining a glitch simulation scenario from a glitch library based on device status information includes:

[0098] When the status information meets the preset information conditions and the strategy timer meets the preset duration, the glitch simulation scenario is obtained from the glitch library.

[0099] In this embodiment of the invention, several preset information conditions can be set according to the device's status information. These preset information conditions can be set to things like successful network configuration, the device being online for xx hours, or currently being online. The preset duration can be set to simulate a spike every x hours during the device's online period, with each spike simulation lasting y seconds, etc.

[0100] For example, the server receives status information from the device such as successful network configuration, the device has been online for 3 hours, and is currently online. If the device's online time (i.e., 3 hours) does not meet the preset online time (xx hours), it cannot proceed to subsequent steps; that is, it cannot retrieve a glitch simulation scenario from the glitch database. The server is equipped with a policy timer that tracks the device's normal operating status. When the timer reaches x hours of online time, a glitch simulation begins, lasting y seconds, before starting the next round of timing.

[0101] In some embodiments, the glitch simulation scenario includes a network glitch simulation scenario, a power glitch simulation scenario, and / or a system glitch simulation scenario.

[0102] The network spike simulation scenario simulates spikes under network failure conditions; the power spike simulation scenario simulates spikes under power supply failure conditions; and the system spike simulation scenario simulates spikes under system resource failure conditions.

[0103] This invention establishes a spike library, which stores different spike simulation scenarios. Different spike simulation scenarios generate different spikes. The server can continuously update the spike library according to test requirements, thereby adding new spike simulation scenarios. In addition to network spike simulation scenarios, power spike simulation scenarios, and system spike simulation scenarios, the spike library can also store other spike simulation scenarios.

[0104] 130. Simulate burrs based on the burr simulation scenario and conduct burr scenario testing on the equipment.

[0105] The test server can extract one type of puncture simulation scenario for puncture simulation each time, or it can extract multiple puncture simulation scenarios for puncture simulation each time.

[0106] In some embodiments, simulating burrs according to a burr simulation scenario includes:

[0107] When the glitch simulation scenario is a glitch scenario simulating a network outage, an automated script connects to the router's backend and cancels the WAN port connection on the router's backend.

[0108] When the glitch simulation scenario is a glitch scenario where the simulated signal disappears, an automated script connects to the router's backend and turns off the WiFi hotspot signal on the router's backend.

[0109] When the glitch simulation scenario is a glitch scenario simulating a weak network condition, an automated script connects to the router backend and limits the network speed or sets a packet loss rate higher than the packet loss rate threshold on the router backend.

[0110] An automation script is a script written in an interpreted programming language within an automated testing framework, which can perform some automated operations.

[0111] In this embodiment, the network spike simulation scenario can include spike scenarios under network outage, signal loss, or weak network conditions, as well as spike scenarios under other network faults. This embodiment uses an automated script to connect to the router's backend, and then utilizes the router's backend to automatically cancel the WAN port connection, automatically turn off the WiFi hotspot signal, limit network speed, or set a packet loss rate higher than the packet loss rate threshold, thereby simulating the network spike scenario.

[0112] In some embodiments, simulating burrs according to a burr simulation scenario includes:

[0113] When the glitch simulation scenario is a glitch scenario simulating a power failure, the controllable instrument simulates the equipment being in different voltage change states or a power failure state.

[0114] In this embodiment, the glitch simulation scenario can be a glitch scenario simulating a power supply failure. Controllable instruments include programmable voltage sources or disconnectable sockets, etc.

[0115] For example, different voltage levels can be achieved using a programmable voltage source, and then the device can be controlled to work at different voltage levels for a period of time; another example is to simulate power outage glitches using a power-off socket, and then observe whether the device can continue to stay online under these power glitches.

[0116] In some embodiments, simulating burrs according to a burr simulation scenario includes:

[0117] When the glitch simulation scenario is a glitch scenario simulating system resource failure, the cloud requests the device to report status information under high-frequency request conditions and / or issues control commands to the device under high-frequency sending conditions, so that the device is in the business logic of high-frequency operation, thereby simulating glitch under system resource failure conditions.

[0118] The high-frequency request condition is that the number of requests sent from the cloud to the device within a preset request duration is greater than the request count threshold; the high-frequency delivery condition is that the number of instructions sent from the cloud to the device within a preset delivery duration is greater than the delivery count threshold.

[0119] In this embodiment of the application, the test scenario for constructing system glitch can be achieved by frequently requesting the device to report its status or frequently issuing control commands through the cloud, so that the device is in a high-frequency operation business logic, which in turn leads to high-frequency use of CPU, memory or disk space resources, thereby simulating the occurrence of system glitch.

[0120] 140. The device that has completed the glitch scenario test is inspected to obtain the test results of the device's online capability. The online capability is used to characterize the device's ability to normally process the unexpected signal.

[0121] In some embodiments, the device that has completed the burr scene test is inspected, including:

[0122] The stage identifier generated by the device during the burr scene test is obtained, and the stage identifier is detected. When the stage identifier is missing, the device is determined to be in an abnormal state when processing burrs.

[0123] The system detects the status information of the device. When it detects that the status information reported by the device to the cloud is missing, it determines that the device's status in processing glitch is abnormal.

[0124] The device is subjected to functional command verification. When the device fails to respond according to the functional command, the device is deemed to be in an abnormal state in handling glitch.

[0125] In this embodiment of the application, the stage identifier is used to characterize the various stages of the device's operation and the processing results of different stages.

[0126] For example, a stage identifier of "dns-query:begin, dns-query:success" indicates that the DNS (Domain Name Server) query was successful, and the device can obtain the IP address corresponding to the domain name in the cloud. When a simulated glitch scenario occurs, the stage identifier can be used to determine whether the device has started the DNS query stage (i.e., whether the stage identifier contains "dns-query"), and the results of these stages (i.e., whether the stage identifier contains "success").

[0127] The most basic requirement for judging a device's ability to stay online is that it communicates normally with the cloud and can report device status information. Therefore, this embodiment of the application can also determine whether the device has correctly handled glitch by detecting the status information reported by the device to the cloud.

[0128] For example, in a scenario without glitches, the device reported a total of 10 logs within half an hour, each with a timestamp. However, after a glitch occurred, the device only reported 9 logs, with some gaps in the connection. Therefore, the device's glitch handling status can be determined to be abnormal. Further calculation of the corresponding online rate metric reveals that the device's ability to remain online while handling this glitch has decreased.

[0129] Verifying the device's functional commands is the core of testing its online capability. For example, if the device can be successfully controlled after a control command is issued, or if the device can be successfully upgraded after an OTA (Over-the-Air Technology) command is issued, the effectiveness of the device's online capability can be determined.

[0130] In this embodiment, after testing the device that has completed the burr scenario test, the method further includes calculating the device online rate index, which is the proportion of the device's online time to the total test time. After obtaining the device online rate index, a test result of the device's ability to remain online is generated. The test result includes the device's online rate index in the corresponding burr simulation scenario and whether the device correctly processed the burr detection information.

[0131] This application provides a device testing method. The method first obtains the device's status information; then, based on the device's status information, it retrieves a glitch simulation scenario from a glitch library. The glitch library stores glitch simulation scenarios, where the glitch represents an unexpected signal received by the device during signal processing. Next, it simulates glitch based on the glitch simulation scenario to perform a glitch scenario test on the device. Finally, it tests the device that has completed the glitch scenario test to obtain the test result of the device's online capability. The online capability characterizes the device's ability to normally process unexpected signals.

[0132] In this way, the present invention can automatically retrieve the corresponding burr simulation scenario from the burr simulation scenario when testing the equipment by setting up a burr library that stores a variety of burr simulation scenarios, and perform burr scenario testing on the equipment according to the burr simulation scenario. Then, by inspecting the equipment that has completed the burr scenario test, it can be determined whether the equipment can correctly handle burrs.

[0133] This invention automates equipment testing, enabling precise simulation of various burr scenarios and ensuring comprehensive coverage of online testing scenarios. While maintaining sufficient and detailed testing coverage, it also saves labor costs and further improves testing efficiency. Furthermore, when equipment malfunctions later, the invention's testing method can pinpoint the corresponding fault location based on the simulated burr scenarios. This invention not only automates testing and reduces labor costs but also improves testing efficiency and lowers repair costs after equipment failure.

[0134] The device testing scheme provided in this application can be applied to various device testing scenarios. For example, taking smart home products as an example, currently, different users have multiple different device scenarios in their homes. Some homes have only one smart device, some have one of the same model in different rooms, some have one of each type of device (smart TV in the living room, smart refrigerator in the kitchen, smart door lock in the entryway, etc.), and some homes not only have multiple types of devices but also multiple units of each type (same type of door lock installed in the entryway / bedroom, same type of TV placed in the living room and bedroom, etc.). The online performance of the devices is inconsistent in these scenarios. Moreover, for each scenario, corresponding glitches need to be simulated to conduct online rate tests (insufficient device memory, network jitter, etc.). These test scenarios and test points often have a permutation and combination logic, that is, it is necessary to test devices with different protocols and different numbers of scenarios. For each scenario, various glitches need to be simulated for each device. After each simulated glitch, it is necessary to observe for a period of time before obtaining the online rate index. Therefore, it takes a considerable amount of time to complete all these permutations and combinations of test content.

[0135] However, the solution provided in this application can automate the testing of the device, accurately complete various burr scenario simulation tests, ensure the coverage of online rate scenario tests, and save labor costs while ensuring that the test coverage is large and detailed enough, and further improve test efficiency and user experience.

[0136] The method described in the above embodiments will be further described in detail below.

[0137] In this embodiment, the method of this application embodiment will be described in detail using the example of how to conduct device testing based on smart home products.

[0138] like Figure 3 As shown, the specific process of a device testing method is as follows:

[0139] 209. The test server sends a trigger command to the smart product so that the smart product can enter the network configuration mode and connect to the network.

[0140] Before testing smart products, they need to be put into network configuration mode so that they can be connected to the network. This ensures a normal communication link between the smart product and the cloud, allowing the smart product to receive control commands or report device status data in a timely manner, which will enable further testing of glitch scenarios.

[0141] Network configuration modes include direct configuration and intelligent configuration. Direct configuration involves transmitting the SSID (Service Set Identifier) ​​and password directly to the device's WiFi module via a host interface (including UART serial port, SPI port, or SDIO port) according to a communication protocol. Upon receiving the SSID and password, the WiFi module connects to the WiFi hotspot signal or router and returns the connection result from the host interface. Intelligent configuration uses the user terminal's WiFi hotspot signal. At the MAC layer, the SSID and password are filled into the unencrypted header of the MAC packet according to a specific protocol format. The SSID and password are then transmitted in segments from the user terminal to the device's WiFi module using broadcast and packet capture methods.

[0142] This application embodiment includes different types of smart products, each with different network configuration methods based on its characteristics. These include: (a) using a simulated remote control (like a red mouse or test wizard) to send infrared signals to operate the device into network configuration mode; (b) using a robotic arm to operate the relevant panel on the device to enter network configuration mode; and (c) using a serial port tool to connect to the device's WiFi module, a data cable to connect to the device's ADB service, and sending specific commands to enable the smart device to enter network configuration mode. These network configuration methods allow the device to enter network configuration mode. Finally, by combining this with automated testing frameworks such as AirTest / Appium, the smart home app can be used to click on the relevant device to enter the network configuration process, thus achieving the online configuration of the smart product.

[0143] In some embodiments, sending a trigger command to the device includes:

[0144] When the device is a remote control device, a remote control signal is sent to the device through a simulated remote control so that the device can enter the network distribution mode;

[0145] When the device is a panel-controlled device, the buttons on the device panel are operated by an automatic robot to enable the device to enter the network distribution mode;

[0146] When the device is difficult to control via a panel, use a serial port tool to connect to the device's WiFi module and send a network configuration command to the device so that the device can execute the network configuration command through the debugging bridge service and thus enter the network configuration mode.

[0147] Remote control devices include infrared, Bluetooth, and wireless remote control devices; easy-to-control devices are those with easily operable panel buttons; difficult-to-control devices are those with relatively difficult-to-operate panel buttons. The serial port tool is used to receive connection requests from the server, send connection commands to the device's communication module via the serial port, receive connection data returned by the communication module via the serial port, and output the connection data to the server. The debug bridge is a window for inputting command lines; the debug bridge service is used for information exchange between the device and the server.

[0148] The configuration method depends on the characteristics of the smart product itself. If the smart product, such as an air conditioner, television, or clothes dryer, is operated by an infrared remote control, it can be achieved through (a). If the smart product, such as a refrigerator, washing machine, or door lock, is operated by a human touch button, it can be achieved through (b). If the smart product, such as a water immersion sensor, smoke sensor, or camera, is difficult to operate by button, it can be achieved through (c).

[0149] 210. Test the server to obtain the status information of the smart product.

[0150] The status information of smart products includes whether the smart product has been successfully configured to the network, the duration of online operation, whether it is currently offline, whether the smart product has been restarted, whether the smart product has been modified in its configuration, whether the network egress status is stable, and whether the device port status is stable.

[0151] For example, by combining the product logs of smart products, we can obtain the status information of smart products, including information such as successful network configuration, online for 3 hours, and currently online.

[0152] In some embodiments, obtaining the status information of a smart product includes:

[0153] Open the app associated with the smart product through an automated testing framework;

[0154] The device interface of the APP is used to perform character recognition by character recognition methods to obtain offline character recognition information;

[0155] Read device status values ​​via XPath;

[0156] Offline character recognition information and device status values ​​are used as status information for smart products.

[0157] In this embodiment of the application, the automated testing framework can be the Airtest automated testing framework, the Appium automated testing framework, or other types of automated testing frameworks.

[0158] Character recognition methods can be optical character recognition (OCR), component tree character recognition, or other methods. When using component tree character recognition, the device interface image of the app is first acquired. Connected components in the device interface image are extracted using a component tree. Simultaneously, a classifier is built based on the glyph features of the characters. The positive samples output by the classifier are scaled uniformly and then input into a Hough forest. Each character is treated as a separate category, and character recognition is completed during the detection process. In this embodiment, connected components are obtained by building a component tree; a set of features is calculated for each connected component and input into the classifier for recognition; the positive samples output by the classifier are used as candidate character regions and scaled to the same size; each candidate character region is input into the Hough forest, which outputs the probability distribution of the character; based on the posterior probability output by the Hough forest, the position of the character is estimated, and the text result is output.

[0159] XPath, short for XML (a subset of Standard Generalized Markup Language), is a language used to locate specific parts of an XML document. Based on the tree structure of XML, XPath has different types of nodes. It uses path expressions to select nodes or sets of nodes within an XML document; an XPath path expression is a written sequence of steps from one XML node (the current context node) to another node or a group of nodes.

[0160] This application embodiment automatically opens the APP associated with the device through an automated testing framework; at the same time, it performs character recognition on the device interface of the APP using a character recognition method to analyze whether the device interface has characters such as "offline", thereby obtaining offline character recognition information; then it reads the device status value through the XPath path, and then uses the offline character recognition information and the device status value as the device status information.

[0161] In some embodiments, obtaining the status information of a smart product includes:

[0162] Call the cloud interface to read the status information reported by the device to the cloud.

[0163] In this embodiment of the application, the cloud interface is used to communicate with the device. The server can call the cloud interface to read the status information reported by the device to the cloud.

[0164] In some embodiments, obtaining the status information of a smart product includes:

[0165] Based on the log interface, and using serial port tools to capture device logs, the device status information is obtained from the device logs.

[0166] The log interface is used to record the device's execution operation information into a log file.

[0167] In routine network maintenance, device logs are frequently used to troubleshoot and locate device faults. For example, if a network is interrupted for a period of time and then automatically recovers, troubleshooting can be done by checking whether the device has been restarted, whether its configuration has been modified, whether the network egress status is stable, and whether the device port status is stable. Additionally, the real-time status of the device can be viewed using status query commands. If the real-time status is normal, the device's historical records can be viewed through the device logs.

[0168] In this embodiment of the application, the server can capture relevant device logs in the log file based on the log interface and simultaneously use a serial port tool to obtain the device status information based on the device logs.

[0169] Users can view the online / offline status and some status attributes of smart products through a smart app (for example, if the smart product is an air conditioner, it will have status attributes such as temperature and mode). For the test server, this device information and status are captured in several ways: using automated testing frameworks such as airtest / appium to simulate a human opening the app, analyzing the device interface for characters like "offline" using OCR / component trees, and reading corresponding values ​​using XPath paths to obtain the device status; calling cloud interfaces to directly read the device status and offline information reported by the device to the cloud; and using log interfaces and serial port tools to capture device logs and check the corresponding string output to determine the device's information and status.

[0170] After obtaining the relevant information and status of the device through the above methods, further analysis will be conducted. For example, if several similar devices are found to be connected to the same WiFi name and belong to the same user account, it can be considered that this is simulating a user scenario: there are multiple similar devices in the user's home.

[0171] 220. The test server retrieves puncture simulation scenarios from the puncture library based on the status information of the smart product. The puncture library stores puncture simulation scenarios.

[0172] In the glitch simulation scenario, glitches are unexpected signals received by the device during signal processing. These glitches can affect the performance of the device and affect its normal operation.

[0173] For example, the equipment may be affected by glitches such as network outages or weak networks during normal operation; or the equipment may be affected by glitches such as sudden voltage changes or power outages during normal operation.

[0174] In some embodiments, obtaining a glitch simulation scenario from a glitch library based on device status information includes:

[0175] When the status information meets the preset information conditions and the strategy timer meets the preset duration, the glitch simulation scenario is obtained from the glitch library.

[0176] In this embodiment of the invention, several preset information conditions can be set based on the device's status information. These preset information conditions can be set to things like successful network configuration, the device being online for xx hours, or currently being online. The preset duration can be set to simulate a spike every x hours during the device's online period, with each spike simulation lasting y seconds, etc.

[0177] For example, the test server receives status information from the device such as successful network configuration, the device has been online for 3 hours, and is currently online. If the device's online time (i.e., 3 hours) does not meet the preset online time (xx hours), it cannot proceed to subsequent steps; that is, it cannot retrieve a glitch simulation scenario from the glitch database. The server is equipped with a policy timer that tracks the device's normal operating status. When the timer reaches x hours of online time, a glitch simulation begins, lasting y seconds, before starting the next round of timing.

[0178] When conducting online capability tests, testers often need to observe the device for a considerable period to calculate the online rate metric. For example, they might calculate the online rate every 72 hours. During these 72 hours, the device status is checked every 0.5 seconds to see if the device is online, thus calculating the percentage of time the device is online within those 72 hours. For instance, if the device is online for 71 hours under normal circumstances, the online rate metric would be 0.986. Since malfunctions (glitch-prone scenarios) also need to be simulated, testers need to conduct an additional 72 hours of testing. During these 72 hours, a glitch is simulated every 8 hours, with each glitch lasting 30 seconds. The online rate metric is then calculated again. If the online rate metric drops significantly during this additional 72 hours, it indicates that the product's ability to handle glitch issues is poor.

[0179] This application uses a certain testing strategy to simulate the occurrence of spikes in order to calculate the online rate index. For example, as mentioned above, every 72 hours is used as a test round, and a spike is simulated every 8 hours in each test round. The spike lasts for 30 seconds. These are written into the test server in advance as configuration files. The test server will time the time according to the configuration files and through the strategy timer. After the time is up, it will perform some corresponding actions.

[0180] In some embodiments, the glitch simulation scenario includes a network glitch simulation scenario, a power glitch simulation scenario, and / or a system glitch simulation scenario.

[0181] The network spike simulation scenario simulates spikes under network failure conditions; the power spike simulation scenario simulates spikes under power supply failure conditions; and the system spike simulation scenario simulates spikes under system resource failure conditions.

[0182] This invention establishes a spike library, which stores different spike simulation scenarios. Different spike simulation scenarios generate different spikes. The server can continuously update the spike library according to test requirements, thereby adding new spike simulation scenarios. In addition to network spike simulation scenarios, power spike simulation scenarios, and system spike simulation scenarios, the spike library can also store other spike simulation scenarios.

[0183] When it is necessary to simulate glitch scenarios, glitch simulation scenarios are extracted sequentially from the glitch library, and then the occurrence of glitch is simulated at a preset time, so that the intelligent product can perform relevant business logic processing, thereby determining the quality performance of the intelligent product's ability to stay online.

[0184] 230. The test server simulates burrs based on the burr simulation scenario to perform burr scenario testing on smart products.

[0185] The test server can extract one type of puncture simulation scenario for puncture simulation each time, or it can extract multiple puncture simulation scenarios for puncture simulation each time.

[0186] When the test server needs to simulate glitches at a preset time according to the above test strategy, it can extract glitch simulation scenarios from the glitch library and then simulate the relevant test scenarios. For example, when it is necessary to simulate glitches for each device (refrigerator, air conditioner, washing machine, etc.) in each user scenario (single device, multiple devices of the same type, one device of each type, multiple devices of multiple types, etc.), and it is also necessary to simulate each type of glitch (insufficient CPU, network speed limit, router power failure, etc.) for a sufficient duration (e.g., 30 seconds), the relevant glitch simulation scenarios can be extracted from the glitch library, thereby enabling the testing of the online capability module of smart products. The test server combines product logs (indicating successful network configuration, online status for xx hours, etc.) and policy timers (such as performing a glitch simulation every 8 hours for 30 seconds as mentioned above) to determine whether a glitch simulation is needed. If so, it extracts the corresponding simulation method from the glitch library. For example, the disk space shortage method can generate a large file to occupy all the storage space of the smart product. Due to insufficient disk space, the smart product may be unable to store relevant device logs and instruction data transmitted from the cloud while maintaining online status, which may lead to module logic errors and prevent the device from staying online, resulting in a decrease in online rate and other indicators. At this time, the test can be conducted to see whether the smart product can correctly handle insufficient disk space, that is, the product quality performance under this glitch scenario.

[0187] In some embodiments, simulating burrs according to a burr simulation scenario includes:

[0188] When the glitch simulation scenario is a glitch scenario simulating a network outage, an automated script connects to the router's backend and cancels the WAN port connection on the router's backend.

[0189] When the glitch simulation scenario is a glitch scenario where the simulated signal disappears, an automated script connects to the router's backend and turns off the WiFi hotspot signal on the router's backend.

[0190] When the glitch simulation scenario is a glitch scenario simulating a weak network condition, an automated script connects to the router backend and limits the network speed or sets a packet loss rate higher than the packet loss rate threshold on the router backend.

[0191] An automation script is a script written in an interpreted programming language within an automated testing framework, which can perform some automated operations.

[0192] In this embodiment, the network spike simulation scenario can include spike scenarios under network outage, signal loss, or weak network conditions, as well as spike scenarios under other network faults. This embodiment uses an automated script to connect to the router's backend, and then utilizes the router's backend to automatically cancel the WAN port connection, automatically turn off the WiFi hotspot signal, limit network speed, or set a packet loss rate higher than the packet loss rate threshold, thereby simulating the network spike scenario.

[0193] When smart products are working normally, many network operations need to be performed, such as reporting device status information to the cloud and listening for control commands issued by the cloud. If network problems occur while the smart product is online, it may cause certain business logic of the module to get stuck or other anomalies to occur, continuously attempting to connect to the network until the operation is completed. Therefore, the embodiments of this application can simulate various network fluctuations to perform glitches, such as network outages, signal loss, and weak networks. For network outages, an automated script can connect to the router's backend and cancel the WAN port connection, causing the router to lose network access, and the devices connected to it will also lose network access. For signal loss, an automated script can also connect to the router's backend and turn off the WiFi hotspot signal, so that the devices cannot find the hotspot. The same applies to weak networks, where network speed limits and packet loss rate settings can be configured in the router's backend. Many network fluctuation glitches can be implemented by automatically connecting to the router's backend and making relevant settings, thereby enabling the testing of indicators such as the online rate of smart products under such scenarios.

[0194] In some embodiments, simulating burrs according to a burr simulation scenario includes:

[0195] When the glitch simulation scenario is a glitch scenario simulating a power failure, the controllable instrument simulates the equipment being in different voltage change states or a power failure state.

[0196] In this embodiment, the glitch simulation scenario can be a glitch scenario simulating a power supply failure. Controllable instruments include programmable voltage sources or disconnectable sockets, etc.

[0197] For example, different voltage levels can be achieved using a programmable voltage source, and then the device can be controlled to work at different voltage levels for a period of time; another example is to simulate power outage glitches using a power-off socket, and then observe whether the device can continue to stay online under these power glitches.

[0198] Whether it's a smart product, a network module, or a router, all require a power source to function. Therefore, by using instruments such as programmable voltage regulators and power-off sockets, it's possible to simulate hardware experiencing various voltage fluctuations, and even glitches during power outages. For example, a programmable power supply can be used to set different voltage levels, allowing a smart TV to operate at each level for a period of time. This allows observation of whether the smart product can remain online under these power glitches, ensuring that the user doesn't perceive the glitches.

[0199] In some embodiments, simulating burrs according to a burr simulation scenario includes:

[0200] When the glitch simulation scenario is a glitch scenario simulating system resource failure, the cloud requests the device to report status information under high-frequency request conditions and / or issues control commands to the device under high-frequency sending conditions, so that the device is in the business logic of high-frequency operation, thereby simulating glitch under system resource failure conditions.

[0201] The high-frequency request condition is that the number of requests sent from the cloud to the device within a preset request duration is greater than the request count threshold; the high-frequency delivery condition is that the number of instructions sent from the cloud to the device within a preset delivery duration is greater than the delivery count threshold.

[0202] In this embodiment of the application, the test scenario for constructing system glitch can be achieved by frequently requesting the device to report its status or frequently issuing control commands through the cloud, so that the device is in a high-frequency operation business logic, which in turn leads to high-frequency use of CPU, memory or disk space resources, thereby simulating the occurrence of system glitch.

[0203] While maintaining online status, smart products often require various system resources, such as CPU resources for instruction calculations, memory resources for caching data during communication, and disk space resources for storing log information at different levels during communication. Therefore, it is necessary to simulate glitches in these system resources to test whether the smart product responds to various unexpected system states to ensure continued online operation. In constructing system glitches in this embodiment, the cloud can frequently request the device to report its status or frequently issue control commands, causing the product to become engrossed in high-frequency business logic, leading to high-frequency use of CPU / memory / disk space resources, thus simulating glitches. Alternatively, a more refined approach can be taken: simulating only CPU resource consumption point-to-point. This can be achieved by triggering a device module to run a loop of code logic via a serial port tool, causing insufficient CPU resources due to high-frequency calculations. Similarly, code logic that continuously requests but does not release resources can be run, causing the product to fall into scenarios such as memory leaks / insufficient memory. These methods simulated the occurrence of glitches, and then tests were conducted according to a preset testing strategy. The online rate and other indicators of the product were then detected to determine and analyze the performance of the intelligent product in handling these system glitches.

[0204] 240. Test the smart products that have completed the puncture scenario test to obtain the test results of the smart products' online keeping capability. The online keeping capability is used to characterize the smart products' ability to normally handle unexpected signals.

[0205] After conducting glitch simulation scenario testing on the smart product, the test server checks the product's online performance, specifically determining whether the product handles glitch issues correctly, such as insufficient disk space. If the product handles glitch issues normally, it indicates that it first clears temporary files (e.g., log information older than a certain number of days) to free up space, or it caches data in memory and continuously attempts to write it to local system files until there is sufficient space, at which point it successfully writes the data locally and continues to the next connection operation. By detecting various parameters of the smart product, analyzing its status, and comparing the online rate and other metrics after the entire glitch test, the test results are obtained and a test report can be generated.

[0206] In some embodiments, testing is performed on smart products that have completed burr scene testing, including:

[0207] Obtain the stage identifier generated by the smart product during the burr scenario test, detect the stage identifier, and determine that the smart product's burr processing state is abnormal when a stage identifier is detected to be missing.

[0208] The system detects the status information of smart products. When it detects that the status information reported by the smart product to the cloud is missing, it determines that the status of the smart product in processing glitch is abnormal.

[0209] The system verifies the function commands of smart products. When it detects that the smart product does not respond according to the function commands, the system determines that the smart product's processing of glitches is abnormal.

[0210] In this embodiment, the stage identifier is used to characterize the various stages of the smart product's operation and the processing results of different stages.

[0211] In scenarios where smart products maintain normal online operation, identifiers for each operation can be obtained. These identifiers indicate the stages involved in maintaining online capability and the results of each stage. For example, a stage identifier of "dns-query:begin, dns-query:success" indicates a successful DNS (Domain Name Server) query, meaning the device can obtain the IP address corresponding to the cloud domain name. When a simulated glitch occurs, the presence of these stage identifiers can be used to determine whether the device has started the DNS query stage (i.e., whether the stage identifier contains "dns-query") and the results of these stages (i.e., whether the stage identifier contains "success"). Therefore, by judging the presence of these identifiers, if the smart product's handling of glitches is inadequate, specific identifiers may be missing. For example, the absence of "dns-query" indicates that the DNS query stage was not entered, or the absence of "success" indicates that the DNS query was unsuccessful.

[0212] The most basic requirement for judging the online capability of a smart product is that it communicates normally with the cloud and can report its status information. Therefore, this embodiment of the application can also determine whether the smart product has correctly handled glitches by detecting the status information reported by the smart product to the cloud.

[0213] For example, in a scenario without glitches, the smart product reported 10 logs within half an hour, each with a timestamp. However, after a glitch occurred, the smart product only reported 9 logs, with gaps in the connections. This indicates that the smart product's handling of the glitch was abnormal. Further calculation of the corresponding online rate metric reveals that the smart product's ability to remain online decreased while handling this glitch. Therefore, this state information of the smart product can be used as one of the criteria for judging whether the smart product's online performance is good.

[0214] Verifying the functional commands of smart products is crucial for testing their online capability, as it supports subsequent control operations and OTA (Over-the-Air) upgrades. Therefore, issuing functional commands can also serve as a basis for testing and judgment. For example, if issuing a control command allows the device to be successfully controlled, or issuing an OTA command allows the device to be successfully upgraded, the effectiveness of the device's online capability can be determined.

[0215] In this embodiment, after testing the device that has completed the burr scenario test, the method further includes calculating the device online rate index, which is the proportion of the smart product's online time within the test duration to the total test duration. After obtaining the device online rate index, a test result of the smart product's ability to remain online is generated. The test result includes the smart product's online rate in the corresponding burr simulation scenario and whether the smart product correctly processed the burr detection information, etc.

[0216] In this embodiment, the intelligent product is first triggered to go online. When the intelligent product reaches a specific stage during its online operation, i.e., when the status information meets the preset information conditions and the strategy timer meets the preset duration, a glitch simulation scenario is obtained from the glitch library and a glitch simulation is performed. After a random simulation period, the glitch is stopped, and the performance of the intelligent product's online operation capability is tested, i.e., whether the glitch is handled correctly, and an online rate test result is generated. Then, the simulation of the next glitch scenario continues, and finally, test report tables for various scenarios and various devices are generated.

[0217] As can be seen from the above, the present invention can automate the testing of equipment, accurately complete various burr scenario simulation tests, ensure the coverage of online rate scenario testing, and save labor costs while ensuring that the test coverage is large and detailed enough, and further improve the testing efficiency and user experience.

[0218] To better implement the above methods, this application also provides a device testing apparatus, which can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, or personal computer; the server can be a single server or a server cluster composed of multiple servers.

[0219] For example, in this embodiment, the method of this application embodiment will be described in detail by taking the example of the device testing device being specifically integrated into the test server.

[0220] For example, such as Figure 4As shown, the equipment testing device may include an information acquisition module 14, a burr scene acquisition module 15, a burr scene testing module 16, and an equipment detection module 17, as follows:

[0221] (I) Information Acquisition Module 14

[0222] The information acquisition module 14 is used to acquire the status information of the device.

[0223] In some embodiments, the apparatus further includes a device network configuration module, which is used to send a trigger command to the device so as to trigger the device to enter the network configuration mode and connect to the network.

[0224] In some embodiments, the device distribution module includes a distribution submodule, which is used for:

[0225] When the device is a remote control device, a remote control signal is sent to the device through a simulated remote control so that the device can enter the network distribution mode;

[0226] When the device is a panel-controlled device, the buttons on the device panel are operated by an automatic robot so that the device can enter the network distribution mode;

[0227] When the device is difficult to control via a panel, use a serial port tool to connect to the device's WiFi module and send a network configuration command to the device so that the device can execute the network configuration command through the debugging bridge service and thus enter the network configuration mode.

[0228] In some embodiments, the information acquisition module includes an information acquisition first module, which is used for:

[0229] The app associated with the device is opened using an automated testing framework;

[0230] The device interface of the APP is used to perform character recognition by a character recognition method to obtain offline character recognition information;

[0231] Read device status values ​​via XPath;

[0232] The offline character recognition information and the device status value are used as the status information of the device.

[0233] In some embodiments, the information acquisition module includes a second information acquisition module, which is used for:

[0234] Call the cloud interface to read the status information reported by the device to the cloud.

[0235] In some embodiments, the information acquisition module includes a third information acquisition module, which is used for:

[0236] Based on the log interface, and using serial port tools to capture device logs, the device status information is obtained from the device logs.

[0237] (II) Burr Scene Acquisition Module 15

[0238] The glitch scene acquisition module 15 is used to acquire glitch simulation scenes from the glitch library based on the device's status information. The glitch library stores glitch simulation scenes, and the glitch in the glitch simulation scene is an unexpected signal received by the device during signal processing.

[0239] In some embodiments, the puncture scene acquisition module includes a judgment submodule, which is used to acquire a puncture simulation scene from the puncture library when the state information meets a preset information condition and the strategy timer meets a preset duration.

[0240] In some embodiments, the glitch simulation scenarios include network glitch simulation scenarios, power glitch simulation scenarios, and / or system glitch simulation scenarios; the network glitch simulation scenario simulates glitch scenarios under network failure conditions; the power glitch simulation scenario simulates glitch scenarios under power supply failure conditions; and the system glitch simulation scenario simulates glitch scenarios under system resource failure conditions.

[0241] (III) Burr Scene Test Module 16

[0242] The burr scenario test module 16 is used to simulate burrs according to the burr simulation scenario and perform burr scenario test on the device.

[0243] In some embodiments, the glitch scenario testing module includes a network glitch simulation module, which is used for:

[0244] When the glitch simulation scenario is a glitch scenario simulating a network outage, an automated script connects to the router's backend and cancels the WAN port connection on the router's backend.

[0245] When the glitch simulation scenario is a glitch scenario where the simulated signal disappears, an automated script connects to the router's backend and turns off the WiFi hotspot signal on the router's backend.

[0246] When the glitch simulation scenario is a glitch scenario simulating a weak network condition, an automated script connects to the router backend and limits the network speed or sets a packet loss rate higher than the packet loss rate threshold on the router backend.

[0247] In some embodiments, the glitch scenario testing module includes a power glitch simulation module, which is used for:

[0248] When the glitch simulation scenario is a glitch scenario simulating a power failure, the controllable instrument simulates the equipment being in different voltage change states or a power failure state.

[0249] In some embodiments, the burr scenario testing module includes a system burr simulation module, which is used for:

[0250] When the glitch simulation scenario is a glitch scenario simulating system resource failure, the cloud requests the device to report status information under high-frequency request conditions and / or issues control commands to the device under high-frequency delivery conditions, so that the device is in a high-frequency operation business logic, thereby simulating glitch under system resource failure conditions; the high-frequency request condition is that the number of requests sent by the cloud to the device within a preset request duration is greater than a request count threshold; the high-frequency delivery condition is that the number of commands issued by the cloud to the device within a preset delivery duration is greater than a delivery count threshold.

[0251] (IV) Equipment Testing Module 17

[0252] The device detection module 17 is used to detect the device that has completed the burr scene test and obtain the test results of the device's online capability. The online capability is used to characterize the device's ability to normally process the unexpected signal.

[0253] In some embodiments, the device detection module includes a stage identifier detection module, a status information detection module, and an instruction verification detection module;

[0254] The stage identifier detection module is used to acquire stage identifiers generated by the device during burr scene testing, detect the stage identifiers, and determine that the device's burr processing state is abnormal when the stage identifier is detected to be missing. The stage identifiers are used to characterize the various stages of the device's operation and the processing results of different stages.

[0255] The status information detection module is used to detect the status information of the device. When it detects that the status information reported by the device to the cloud is missing, it determines that the status of the device in processing glitch is abnormal.

[0256] The instruction verification and detection module is used to verify the function instructions of the device. When it is detected that the device does not respond according to the function instructions, the device is determined to be in an abnormal state of glitch processing.

[0257] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0258] As can be seen from the above, the device testing apparatus of this embodiment can obtain the status information of the device through the information acquisition module, then obtain the burr simulation scenario from the burr library according to the status information of the device through the burr scenario acquisition module, then simulate burrs according to the burr simulation scenario through the burr scenario testing module, perform burr scenario testing on the device, and then use the device detection module to detect the device that has completed the burr scenario testing to obtain the test result of the device's ability to stay online.

[0259] Therefore, this invention, through automated testing of equipment, can accurately complete various burr scenario simulation tests, ensuring the coverage of online rate scenario testing. While ensuring sufficient and detailed test coverage, it saves labor costs and further improves testing efficiency. Moreover, when equipment fails later, the equipment testing method of this invention can find the corresponding fault point based on the corresponding burr simulation scenario. This invention not only achieves automated testing and reduces labor costs, but also improves testing efficiency and reduces the repair cost after equipment failure.

[0260] Accordingly, this application also provides an electronic device, which can be a terminal or a server. The terminal can be a smartphone, tablet computer, laptop computer, touch screen, game console, personal computer, personal digital assistant (PDA), or other terminal device. The server can be a single server or a server cluster composed of multiple servers.

[0261] like Figure 5 As shown, Figure 5 This is a schematic diagram of the electronic device structure provided in an embodiment of this application. The electronic device includes: a memory 18, a processor 19, and a communication module 20.

[0262] The memory 18 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), a hard disk, or a solid-state drive, etc. The memory 18 is used to store programs, and the processor 19 executes the programs after receiving execution instructions.

[0263] The processor 19 may be an integrated circuit chip with data processing capabilities. The processor 19 described above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0264] The communication module 20 is used to establish a communication connection between the test server and external devices via a network, enabling the transmission and reception of network signals and data. The network signals may include wireless or wired signals.

[0265] For details on the implementation of each module, please refer to the previous examples, which will not be repeated here.

[0266] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0267] As can be seen from the above, the electronic device provided in this embodiment can accurately complete various burr scenario simulation automatic tests, ensuring that the test coverage is large and detailed enough, while saving labor costs and further improving test efficiency. Moreover, when the equipment fails later, the corresponding fault point can be found according to the corresponding burr simulation scenario through the equipment testing method of this invention, which improves test efficiency and reduces the repair cost after the equipment fails.

[0268] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0269] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of computer programs that can be loaded by a processor to execute the steps in any of the device testing methods provided in embodiments of this application.

[0270] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0271] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0272] Since the computer program stored in the storage medium can execute the steps in any of the device testing methods provided in the embodiments of this application, the beneficial effects that any of the device testing methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0273] The above provides a detailed description of a device testing method, apparatus, electronic device, and storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A device testing method, characterized by, The method comprises: obtaining state information of the device; obtaining a glitch simulation scenario from a glitch library according to the state information of the device, the glitch library storing glitch simulation scenarios, glitches in the glitch simulation scenarios being unexpected signals received by the device during signal processing; simulating glitches according to the glitch simulation scenario, and performing a glitch scenario test on the device; detecting the device after the glitch scenario test to obtain a test result of the on-line capability of the device, the on-line capability being used to represent the ability of the device to normally process the unexpected signals; the glitch simulation scenario comprises a network glitch simulation scenario, a power glitch simulation scenario and / or a system glitch simulation scenario; the network glitch simulation scenario is a glitch scenario simulating a network failure condition; the power glitch simulation scenario is a glitch scenario simulating a power supply failure condition; and the system glitch simulation scenario is a glitch scenario simulating a system resource failure condition; the detection of the device after the glitch scenario test comprises: obtaining a stage identifier generated by the device during the glitch scenario test, detecting the stage identifier, and determining that the state of the device in processing glitches is abnormal when the stage identifier is detected to be missing; the stage identifier is used to represent various stages of the device in the running process and the processing results of different stages; detecting the state information of the device, and determining that the state of the device in processing glitches is abnormal when the state information reported to the cloud by the device is detected to be missing; and performing a function instruction verification on the device, and determining that the state of the device in processing glitches is abnormal when the device is detected to not respond according to the function instruction.

2. The device testing method of claim 1, wherein, Before the obtaining of the state information of the device, the method comprises: sending a trigger instruction to the device to trigger the device to enter a network configuration mode.

3. The device testing method of claim 2, wherein, The sending of the trigger instruction to the device comprises: when the device is a remote control device, sending a remote control signal to the device through a simulated remote controller to enable the device to enter the network configuration mode; when the device is a panel easy control device, automatically operating a key on a panel of the device through a robot to enable the device to enter the network configuration mode; when the device is a panel difficult control device, connecting a WiFi module of the device through a serial port tool and sending a network configuration instruction to the device to enable the device to execute the network configuration instruction through a debugging bridge service and enter the network configuration mode.

4. The apparatus testing method according to claim 1 or 3, wherein The obtaining of the state information of the device comprises: opening an APP associated with the device through an automated test framework; performing character recognition on a device interface of the APP through a character recognition method to obtain offline character recognition information; reading a device state value through an xpath path; taking the offline character recognition information and the device state value as the state information of the device.

5. The device testing method of claim 4, wherein, The obtaining of the state information of the device comprises: calling a cloud interface to read state information reported by the device to the cloud through the cloud interface.

6. The device testing method of claim 5, wherein, The obtaining of the state information of the device comprises: The device state information is obtained according to the device log based on a log interface and by capturing the device log by using a serial port tool.

7. The device testing method of claim 1, wherein, The spur simulation scenario is obtained from the spur library according to the device state information. The spur simulation scenario is obtained from the spur library when the device state information meets preset information conditions and a strategy timer meets a preset time length.

8. The device testing method of claim 7, wherein, The spur is simulated according to the spur simulation scenario. When the spur simulation scenario is a spur scenario simulating a network disconnection, a router background is connected through an automatic script, and a WAN port is disconnected on the router background. When the spur simulation scenario is a spur scenario simulating signal disappearance, a router background is connected through an automatic script, and a WiFi hotspot signal is turned off on the router background. When the spur simulation scenario is a spur scenario simulating a weak network, a router background is connected through an automatic script, and a network speed is limited or a packet loss rate higher than a packet loss rate threshold is set on the router background.

9. The device testing method of claim 7, wherein, The spur is simulated according to the spur simulation scenario. When the spur simulation scenario is a spur scenario simulating a power failure, a controllable instrument is used to simulate different voltage variation states or a power-off state of the device.

10. The device testing method of claim 7, wherein, The spur is simulated according to the spur simulation scenario. When the spur simulation scenario is a spur scenario simulating a system resource failure, a cloud end requests state information reporting of the device under a high-frequency request condition and / or issues a control instruction to the device under a high-frequency issue condition, so that the device is in a high-frequency operation business logic, thereby simulating a spur in a system resource failure condition; the high-frequency request condition is that a number of requests sent by the cloud end to the device within a preset request time length is greater than a request number threshold; and the high-frequency issue condition is that a number of instructions issued by the cloud end to the device within a preset issue time length is greater than an issue number threshold.

11. An apparatus testing device, characterized by The device includes an information obtaining module, a spur scenario obtaining module, a spur scenario testing module, and a device detecting module. The information obtaining module is configured to obtain device state information. The spur scenario obtaining module is configured to obtain a spur simulation scenario from a spur library according to the device state information, the spur library storing the spur simulation scenario, and a spur in the spur simulation scenario being an unexpected signal received by a device during signal processing. The spur scenario testing module is configured to simulate a spur according to the spur simulation scenario and perform a spur scenario test on the device. The device detecting module is configured to detect the device after the spur scenario test is completed, to obtain a test result of an online keeping capability of the device, the online keeping capability representing an ability of the device to normally process the unexpected signal. The glitch simulation scenario includes a network glitch simulation scenario, a power glitch simulation scenario, and / or a system glitch simulation scenario; the network glitch simulation scenario is a scenario simulating a network failure; the power glitch simulation scenario is a scenario simulating a power supply failure; and the system glitch simulation scenario is a scenario simulating a system resource failure; a stage identifier detection module configured to acquire a stage identifier generated by the device during the glitch scenario test, detect the stage identifier, and determine that the device is in an abnormal state of processing glitches when the stage identifier is missing; The stage identifier is used to represent the stages of the device during operation and the processing results of different stages. Detect the state information of the device, and determine that the device is in an abnormal state of processing glitches when the state information reported by the device to the cloud is missing. The function instruction of the device is verified, and it is determined that the device is in an abnormal state of processing glitches when the device does not respond according to the function instruction.

12. An electronic device, comprising: A processor and a memory are included, the memory stores a plurality of instructions, and the processor loads the instructions to execute the steps in the device test method of any one of claims 1 to 10.

13. A storage medium, characterized by The storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by the processor to execute the steps in the device test method of any one of claims 1 to 10.

Images