Internet of Things equipment performance test method and device, electronic equipment and storage medium

Abstract

The invention provides an Internet of Things equipment performance test method and device, electronic equipment and a storage medium, according to the embodiment of the invention, the Internet of Things equipment is controlled to change an operation load state according to an operation load condition for testing so as to pressurize an operation load of hardware resources of the Internet of Things equipment; injecting simulation fault parameters into an operating system kernel of the Internet of Things equipment, so that the Internet of Things equipment triggers a fault recovery process according to the simulation fault parameters; and finally, monitoring the fault recovery process of the Internet of Things equipment, and generating a performance test result corresponding to the Internet of Things equipment according to the monitored operation load index. The method can be compatible with operating systems of different Internet of Things devices, simulate the running state of the Internet of Things device in a complex service scene, and perform fault injection on the Internet of Things device in the running state to test the performance of the Internet of Things device.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, electronic device and storage medium for testing performance of an Internet of Things device. Background Art

[0002] The Internet of Things (IoT) is the "Internet of Everything Connected". It is an extension and expansion of the Internet. It combines various information sensing devices with the Internet to form a huge network, realizing the interconnection of people, machines and things at any time and any place. With the development of the Internet of Things, more IoT devices can be introduced to bring more convenience to people's lives.

[0003] Typical IoT devices may involve multiple fields such as industry, buildings, properties, transportation, and communities. There may be large industrial computers or miniature smart cameras. These IoT devices face different operating environments. In addition, due to the different operating systems of the IoT devices themselves (for example, different versions of the Linux operating system), the large differences in the load that can be run by hardware resources, and the large differences in the skills of the people who operate IoT devices, IoT devices are prone to encounter many unpredictable failures. However, there is currently no universal testing tool that can be compatible with different versions of operating systems for different IoT devices and simulate the operation of hardware with different hardware specifications when failures occur under multiple loads, which brings inconvenience to operation and maintenance personnel or R&D personnel in testing the performance of IoT devices. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, electronic device and storage medium for testing performance of an Internet of Things device to solve one or more of the above-mentioned technical problems.

[0005] In a first aspect, an embodiment of the present application provides a method for testing performance of an IoT device, including:

[0006] Controlling the IoT device to change the operating load state according to the operating load condition for testing, so as to increase the operating load of the hardware resources of the IoT device;

[0007] Injecting simulated fault parameters into the operating system kernel of the IoT device, so that the IoT device triggers a fault recovery process according to the simulated fault parameters;

[0008] The fault recovery process of the IoT device is monitored, and a performance test result corresponding to the IoT device is generated based on the monitored operating load index.

[0009] In a second aspect, an embodiment of the present application provides a method for testing performance of an IoT device, including:

[0010] Controlling the storage resources of the IoT device to change the operating load state according to the operating load condition for testing, so that the operating load of the storage resources of the IoT device increases;

[0011] Injecting simulated fault parameters into the operating system kernel of the IoT device, so as to cause an execution failure of the IO request in the IO scheduling layer of the operating system kernel in the IoT device, thereby triggering the IoT device to trigger a fault recovery process;

[0012] The fault recovery process of the IoT device is monitored, and a performance test result corresponding to the IoT device is generated based on the monitored operating load index.

[0013] In a third aspect, an embodiment of the present application provides an IoT device performance testing device, including:

[0014] A load state changing module, used to control the IoT device to change the operating load state according to the operating load condition for testing, so as to increase the operating load of the hardware resources of the IoT device;

[0015] A fault parameter injection module, used to inject simulated fault parameters into the operating system kernel of the IoT device, so that the IoT device triggers a fault recovery process according to the simulated fault parameters;

[0016] The status parameter monitoring module is used to monitor the fault recovery process of the IoT device and generate a performance test result corresponding to the IoT device based on the monitored operating load index.

[0017] In a fourth aspect, an embodiment of the present application provides an IoT device performance testing device, including:

[0018] A load state growth module, used for controlling the storage resources of the Internet of Things device to change the operation load state according to the operation load condition for testing, so that the operation load of the storage resources of the Internet of Things device increases;

[0019] A fault parameter injection module is used to inject simulated fault parameters into the operating system kernel of the IoT device, so as to cause an execution failure of the IO request in the IO scheduling layer of the operating system kernel in the IoT device, thereby triggering the IoT device to trigger a fault recovery process;

[0020] The status parameter monitoring module is used to monitor the fault recovery process of the IoT device and generate a performance test result corresponding to the IoT device based on the monitored operating load index.

[0021] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the methods described above when executing the computer program.

[0022] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the methods described above is implemented.

[0023] Compared with the related art, this application has the following advantages:

[0024] According to the embodiment of the present application, the IoT device is first controlled to change the operating load state according to the operating load conditions used for testing, so as to increase the operating load of the hardware resources of the IoT device; then, simulated fault parameters are injected into the operating system kernel of the IoT device, so that the IoT device triggers the fault recovery process according to the simulated fault parameters; finally, the fault recovery process of the IoT device is monitored, and the performance test results corresponding to the IoT device are generated according to the monitored operating load indicators. The above scheme can be compatible with the operating systems of different IoT devices (such as different versions of the Linux operating system), simulate the operating state of the IoT device in a complex service scenario, and inject faults into the IoT device in this operating state to test the performance of the IoT device.

[0025] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

[0027] Figure 1 A schematic diagram showing an application example of the IoT device performance testing solution provided in an embodiment of the present application;

[0028] Figure 2 A performance test page in an application example of the IoT device performance test solution provided in an embodiment of the present application is shown;

[0029] Figure 3 A flowchart of a method for testing performance of an IoT device provided in an embodiment of the present application is shown;

[0030] Figure 4 A flowchart of another IoT device performance testing method provided in an embodiment of the present application is shown;

[0031] Figure 5 A structural block diagram of an IoT device performance testing device provided in an embodiment of the present application is shown;

[0032] Figure 6 A structural block diagram of another IoT device performance testing device provided in an embodiment of the present application is shown; and

[0033] Figure 7 A block diagram of an electronic device used to implement an embodiment of the present application is shown. DETAILED DESCRIPTION

[0034] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the concept or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary in nature and not restrictive.

[0035] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0036] The Internet of Things is an important part of the new generation of information technology, also known as the pan-interconnection, which means that everything is connected and everything is connected. Therefore, "the Internet of Things is the Internet of Things connected to everything": first, the core and foundation of the Internet of Things is still the Internet, which is an extension and expansion of the Internet; second, its user end extends and expands to any object to exchange information and communicate. Therefore, the definition of the Internet of Things is a network that connects any object to the Internet through information sensing devices such as radio frequency identification, infrared sensors, global positioning systems, laser scanners, etc., according to the agreed protocol, to exchange information and communicate, so as to realize the intelligent identification, positioning, tracking, monitoring and management of objects.

[0037] With the development of the Internet of Things, more IoT devices can be introduced at the edge computing nodes close to the user side, bringing more convenience to people's lives. However, due to the large differences in the hardware of IoT devices themselves, different operating systems (different IoT devices may use different versions of the Linux operating system, such as Ubuntu, a Linux operating system mainly for desktop applications, and CentOS, a Linux operating system mainly for community enterprises), and different operating environments (for example, running in different environments such as industry, buildings, properties, transportation, and communities), there is currently no universal testing tool that can be compatible with different versions of the operating system of different IoT devices and simulate the operation of hardware with different hardware specifications when failures occur under multiple loads, which brings inconvenience to operation and maintenance personnel or R&D personnel in testing the performance of IoT devices.

[0038] In a related technology prior to the present application, a single load stress simulation test can be performed on an IoT device through the native tools of the Linux operating system, but it is not possible to perform a fault test on the IoT device while simulating the operating load state of the IoT device.

[0039] In another related technology prior to the present application, a single fault simulation test can be performed on an IoT device through native tools of the Linux operating system, but the operating load state of the IoT device cannot be simulated.

[0040] In view of this, the embodiments of the present application provide a method, apparatus, electronic device and storage medium for testing performance of an Internet of Things device to solve one or more of the above-mentioned technical problems.

[0041] Figure 1 It is a schematic diagram of an application example of the performance test scheme for the Internet of Things device provided in the embodiments of the present application. The Internet of Things device involved in the embodiments of the present application may be a device that can be connected to the Internet according to the agreed protocol to exchange information and communicate, such as some edge devices close to the user side. These edge devices can be deployed on computing nodes close to the user side to provide near-end services to the user side. Due to the diverse application scenarios and operating environments of edge devices, different operating systems and hardware resources, and large differences in operator skills, these edge devices are prone to encounter more unpredictable failures, so it is necessary to perform performance testing on these edge devices. Specifically, the Internet of Things devices involved in the embodiments of the present application may include but are not limited to some industrial gateways, routers, smart cameras, large industrial industrial computers, edge servers, etc. These Internet of Things devices can be deployed with Linux operating systems or other operating systems, and this application does not limit this.

[0042] The embodiment of the present application can simulate the operating load state of the IoT device by applying load pressure to the operating system kernel of the IoT device, and inject corresponding simulated fault parameters into the IO (input and output) process under the corresponding operating load state to simulate the situation where the IoT device fails under the operating load state, and then monitor the fault recovery process of the IoT device to judge the performance and stability of the IoT device. Among them, the fault recovery process of the IoT device can be the response process of the IoT device operating system to the fault injected by the IO external device, which can include fault injection and fault recovery processes.

[0043] First, the IoT device can be controlled to change the operating load state according to the operating load conditions used for testing, so as to increase the operating load of the hardware resources of the IoT device. The operating load conditions may include at least one of the operating load execution mode, the operating load amount, the load growth adjustment frequency, and the number of threads for controlling the load growth adjustment. For example, Figure 1 As shown, the running load execution mode (load pressure mode) adopted can be disk (including file system) pressure or memory pressure, wherein the disk (including file system) pressure mode can simulate the running load state of the external storage resources of the IoT device, and the memory pressure mode can simulate the running load state of the internal storage resources of the IoT device; the load pressure strategy can be to set the load amount, pressure frequency, number of concurrent pressure threads, multiple pressure modes, etc. of the IoT device; the load pressure action can be memory read and write load pressure, memory application & release load pressure state, disk (including file system) load pressure state, etc. Through the combination of these load pressure modes, strategies, and actions, the operating system kernel of the IoT device can be load-pressurized to simulate the running load state of the IoT device. For example, the disk (including file system) pressure mode, the concurrent pressure strategy of 6 threads, and the read and write load pressure action can simulate the running load state of the IoT device in the IO of concurrently reading and writing 6 threads with the external disk or file system.

[0044] Specifically, the IoT device is controlled to change the operating load state according to the operating load conditions for testing. The operating load conditions for testing input based on the performance test page can be received, and the operating load conditions for testing are provided to the IoT device, so that the IoT device changes the operating load state corresponding to the hardware resources according to the operating load conditions for testing. Among them, at least one operating load mode can be provided in the performance test page, and the operating load mode can record the combination of condition contents included in different operating load conditions (such as different load pressure methods, strategies, and combinations of actions), and then determine the operating load conditions for testing according to the selected and / or modified operating load mode.

[0045] Figure 2 The performance test page in an application example of the IoT device performance test solution provided in the embodiment of the present application is shown. The performance test page can be a page provided by an executable program (such as an exe program, i.e., a program in an executable file format, an executable program that can be floated in the operating system storage space). Figure 2 As shown, the operation and maintenance personnel or R&D personnel of the IoT device and other people who have the need to test the performance of the IoT device can connect to and communicate with the target IoT device by entering the corresponding remote connection tool parameters (ssh connection parameters), such as the URL (IP), user name (user), password (password), computer port number (port) of the target IoT device in the performance test page (stability test tool).

[0046] Before starting the test, you can set the corresponding operating load conditions (test parameters), such as test mode, number of files, size of a single file, file block size (default 16k), number of threads (default 1), frequency of writing kernel data to disk (fsync frequency), test time, whether cache is applicable, etc. You can also select preset scenarios for testing. The tester clicks the corresponding preset scenario button on the performance test page, and the program will automatically fill in the corresponding parameters (the corresponding parameters can be viewed through the parameter description). For example, scenario 1 can be a single large file single-thread sequential write; scenario 2 can be a single large file single-thread sequential read; scenario 3 can be multiple small files single-thread random write; scenario 4 can be multiple small files single-thread random read; scenario 5 can be multiple small files multi-thread random write; scenario 6 can be multiple small files multi-thread random read; scenario 7 can be a single large file de-cached write; scenario 8 can be a single large file de-cached read. In addition, you can also select a tool to inject simulated fault parameters into the target IoT device. You can select the corresponding tool installation through the tool installation directory on the performance test page.

[0047] After entering or selecting the above conditions, click the Start Test button in the performance test page to inject simulated fault parameters into the operating system kernel of the target IoT device under the set operating load conditions, so that the IoT device triggers the fault recovery process according to the simulated fault parameters. Among them, the fault recovery process of the IoT device can be the response process of the IoT device operating system to the fault injected by the IO external device, which can include fault injection and fault recovery processes. Furthermore, the fault recovery process of the target IoT device can be monitored, and the performance test results corresponding to the IoT device can be generated according to the monitored operating load indicators. The performance test results can include the performance and stability results of the IoT device.

[0048] Secondly, simulated fault parameters can be injected into the operating system kernel of the IoT device so that the IoT device triggers a fault recovery process according to the simulated fault parameters. There are two ways to inject simulated fault parameters into the operating system kernel of the IoT device: the first way is to inject simulated fault parameters in the IO request stage. After detecting the communication data sent by the external device to the IoT device, the fault injection tool can be called to inject the first simulated fault parameter into the operating system kernel, so that the IO scheduling layer of the operating system kernel does not initiate an IO request for the external device and the IoT device triggers a fault recovery process; the second way is to inject simulated fault parameters in the IO interruption stage. After detecting that the IO scheduling layer of the operating system kernel initiates an IO request for the external device, the fault injection tool can be called to inject the second simulated fault parameter into the operating system kernel, so that the IO scheduling layer of the operating system kernel does not respond to the IO request within the set time and the IoT device triggers a fault recovery process. Furthermore, before injecting simulated fault parameters into the operating system kernel of the IoT device so that the IoT device triggers a fault recovery process based on the simulated fault parameters, the number of injections and / or injection frequency of the simulated fault parameters into the operating system kernel can be determined based on the configured fault occurrence probability, and the simulated fault parameters can be injected into the operating system kernel of the IoT device according to the determined number of injections and / or injection frequency.

[0049] like Figure 1 As shown, in the embodiment of the present application, the fault injection tool used can be the fault-injection fault injection tool that comes with the Linux operating system; the fault injection strategy used can be a frequency / number control strategy, a random trigger probability strategy, and an allocation mode control strategy; the fault injection action used can be to use the method of returning the injection error by calling the generic make request function (generic computer program request function, or block device request processing function) in the IO request phase to perform IO request fault injection on the operating system kernel, or it can be to use the method of returning the injection error by calling the blk complete request function (complete request function) in the IO interrupt phase to perform IO timeout fault injection on the operating system kernel. Therefore, the combination of the above-mentioned fault injection tools, strategies, and actions can be used to inject faults into the operating system kernel of the Internet of Things device.

[0050] In the IO process of an IoT device using the Linux operating system, an external device can first initiate an IO request through a generic make request function. The IO scheduling layer in the operating system kernel of the IoT device responds to the IO request initiated by the external device and forwards the IO request to a block device (blk queue bio) with a request queue through a generic make request function. The above process is the IO request stage in the IO process. If an IO request fault is injected into the operating system kernel in this stage, the IO scheduling layer of the operating system kernel will not be able to initiate an IO request for the external device, and will not be able to forward the IO request to the corresponding request queue, thereby detecting the performance of the target IoT device under such a fault.

[0051] After the IO request stage, the operating system kernel will complete the hard interrupt according to the IO request, that is, start the driver layer (scsi) and hardware layer in the operating system kernel, call the scsi request fn function (driver request processing function) in the driver layer to scan the IO requests in the same request queue, and convert the specific IO request into a driver command (scsi command) that the driver layer can recognize. After the driver is completed (scsi done), that is, after the IO hard interrupt is completed, the IO completion information (completion request) returned by calling the blkcomplete request function triggers the IO completion soft interrupt (IO interrupt completion), and then transmits the relevant IO completion information back to the external device. In the above process, if the IO interrupt timeout fault is injected into the blkcompleterequest function, so that the IO scheduling layer of the operating system kernel cannot respond to the IO request within the set time, the performance of the target IoT device under such a fault can be detected.

[0052] Finally, the fault recovery process of the IoT device can be monitored, and the performance test results corresponding to the IoT device can be generated based on the monitored operation load indicators. The operation load indicators related to the operation status of the hardware resources of the target IoT device can be determined first. The operation load indicators can include the operation load indicators corresponding to the hardware resources, and the operation load indicators corresponding to the execution objects related to the hardware resources, such as the file system, host, and process-related operation load indicators. Then, the fault recovery process of the IoT device is monitored according to the predetermined operation load indicators.

[0053] like Figure 1As shown, the specific monitoring indicators may include: memory read and write execution time monitoring, memory read and write average delay monitoring, memory application release specified size, disk (including file) read and write throughput monitoring, core process status operation monitoring, host status operation monitoring, host self-recovery status monitoring, process self-recovery monitoring, host recovery memory performance comparison, etc. The above monitoring indicators can be adjusted according to actual service needs, and this application does not impose any restrictions on this. After obtaining the above monitoring indicators, you can use Figure 2 The performance test page shown provides the tester with performance test results (including stability test results) so that the tester can execute corresponding performance optimization strategies on the target IoT device based on the performance test results (including stability test results).

[0054] The execution subject of the embodiment of the present application may be an application, service, instance, functional module in software form, virtual machine (VM), container or cloud server, etc., or a hardware device (such as a server or terminal device) or hardware chip (such as CPU, GPU, FPGA, NPU, AI acceleration card or DPU) with data processing function, etc. The device for implementing the embodiment of the present application may be deployed at the terminal where the IoT device is located, or may be deployed at the control end where the pre-configured control device is located, or may be specifically executed by the terminal after the execution code is issued by the control end. The execution subjects of the three links of load pressure operation, fault injection operation and monitoring operation in the process of performance testing the IoT device can be deployed according to actual needs, and can be configured in the IoT device or in the control device respectively, and the present application does not impose any restrictions on this.

[0055] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0056] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0057] The present application embodiment provides a method for testing the performance of an IoT device. Figure 3The flowchart of a method 300 for testing performance of an IoT device according to an embodiment of the present application is shown. The method 300 may include:

[0058] In step S301, the IoT device is controlled to change its operating load state according to the operating load conditions for testing, so as to increase the operating load of the hardware resources of the IoT device. Since the IoT device is controlled to change its operating load state according to the operating load conditions for testing, the operating load conditions of the IoT device in different usage scenarios can be simulated, thereby performing performance monitoring on the IoT device.

[0059] In an embodiment of the present application, the operating load condition may be a parameter that can change the state of the IoT device and is used in the process of performing performance and stability testing on the IoT device, such as the block size of the write operation when writing a file to the memory of the IoT device, the number of threads for concurrent testing, the test time, etc.; the operating load state may be the load state of hardware resources such as memory, disk, and CPU of the IoT device under different parameter settings. The operating load state may be a simulated operating state of the IoT device in different environments and situations, such as the actual CPU, memory, disk, etc. occupancy state of the smart camera when the smart camera is operating at an intersection with high temperature or high-density traffic.

[0060] The tool for pressurizing the operating load of the hardware resources of the above-mentioned IoT devices can use Linux native test tools, such as sysbench (stress test tool) and hdparm (hard disk stress test tool), etc., by combining these native test tools into new test tools according to specific methods, sequences, etc., and setting different test strategies for the combined test tools, and then the combined test tools with specific strategies can be used to simulate the operating load state of different IoT devices in different scenarios. The way to combine these native tools can be to rewrite the computer code of the native tools, add new command lines to the computer code of the native tools, fit the computer codes of different native tools together, etc., and this application does not impose any restrictions on this. Similarly, setting a specific strategy can also be achieved by combining the above-mentioned native tools, and this application does not impose any restrictions on this. For example, a Linux native test tool only has the state of writing a paragraph of text to the memory of the simulated IoT device, and another Linux native test tool only has the state of simulating writing a paragraph of text to the disk of the IoT device. The functions of these two Linux native test tools can be combined in the above manner, so that the combined test tool can simulate the state of writing a paragraph of text to the memory and disk of the IoT device at the same time. In addition, by adding corresponding strategies to the combined test tool, such as random strategies, timing strategies, etc., for example, the corresponding strategies can be written into PIN (Patron Identification Number, personal identification number) file format or other formats of instructions, scripts, etc. using C language, and then combining the file or instruction or script with the aforementioned test tool, it is possible to simulate the state of randomly or timingly writing a piece of text into the memory and disk of the IoT device.

[0061] In one possible implementation, the above-mentioned method of controlling the IoT device to change the operating load state according to the operating load conditions for testing can be achieved by the server of the IoT device, which first receives the operating load conditions for testing input based on the performance test page, and then provides the operating load conditions for testing to the IoT device, so that the IoT device changes the operating load state corresponding to the hardware resources according to the operating load conditions for testing.

[0062] The performance test page involved in the embodiments of the present application can be a page provided by an executable program (e.g., an exe program, i.e., a program in an executable file format, an executable program that can be floated and located in the storage space of an operating system), or a client or program function plug-in for submitting a performance test request, etc. The operation and maintenance personnel, R&D personnel, or testers, etc., individuals, units, and other demanders who have test requirements for the corresponding IoT devices, can submit the conditions such as the IP address and test parameters of the IoT device to be tested based on the page, client, or program function plug-in, and the present application does not impose any restrictions on this. For example, the target IoT device's website address (IP), user name (user), password (password), computer port number (port), etc. can be entered in the performance test page to connect to and communicate with the target IoT device.

[0063] In some embodiments, at least one operating load mode is provided in the above-mentioned performance test page, and the operating load mode corresponds to the combination of conditional contents included in the above-mentioned operating load conditions, wherein the above-mentioned operating load conditions for testing received based on the input of the performance test page can determine the operating load conditions for testing based on the selected and / or modified operating load mode.

[0064] Optionally, the above-mentioned operating load conditions may include conditions such as test mode, number of files, size of a single file, file block size (default 16k), number of threads (default 1), frequency of writing kernel data to disk (fsync frequency), test time, whether cache is applicable, etc. After combining these conditions, a variety of operating load modes can be formed to simulate the operating load conditions of the target IoT device in different scenarios.

[0065] In addition, the operating load conditions for testing can also be determined based on the selected and / or modified operating load mode. For example, a preset operating load mode is selected to determine the operating load conditions for testing, and the preset operating load modes may include: single large file single thread sequential write, single large file single thread sequential read, multiple small files single thread random write, multiple small files single thread random read, multiple small files multi-thread random write, multiple small files multi-thread random read, single large file de-cached write, single large file de-cached read, etc., and this application does not impose any restrictions on this.

[0066] Correspondingly, the running load condition may include at least one of a running load execution mode, a running load amount, a load growth adjustment frequency, and a number of threads for controlling load growth adjustment.

[0067] The running load conditions can be set for different scenarios such as memory reading and writing, memory application release, file system or disk reading and writing or application release, and the test time, frequency, number of times, etc. can also be pre-set. Among them, the running load execution mode can be sequential reading, sequential writing, random reading, random writing, whether to use cache reading and writing, etc.; the running load volume can be memory block size, number of files, file size, file block size (block size of write operation when writing files), etc.; the load growth adjustment frequency can be the frequency of writing kernel data to disk (fsync frequency, that is, how many write operations are performed after an fsync operation); the number of threads that control the load growth adjustment can be the number of concurrent pressure threads and the number of concurrent test threads.

[0068] In step S302, simulated fault parameters are injected into the operating system kernel of the IoT device so that the IoT device triggers a fault recovery process according to the simulated fault parameters. The fault recovery process of the IoT device may be a response process of the IoT device operating system to a fault injected by an IO external device, which may include a fault injection and a fault recovery process. The simulated fault parameters may be parameters that cause the IoT device to fail during operation, such as a request that should be generated but is not generated, a request that should be responded to but is not responded to, etc. This application does not impose any restrictions on this.

[0069] That is, when a certain operating load condition is set for the target IoT through step S301, simulated fault parameters are injected into the operating system kernel of the target IoT device, so that the target IoT device fails according to the simulated fault parameters, thereby triggering the fault recovery process. Among them, the fault recovery process of the IoT device can be the response process of the IoT device operating system to the fault injected by the IO external device, which can include fault injection and fault recovery processes. In this way, the fault recovery process of the target IoT device can be monitored, and the performance test results (including stability test results) corresponding to the IoT device can be generated according to the monitored operating load indicators.

[0070] In some embodiments, simulated fault parameters are injected into the operating system kernel of the IoT device so that the IoT device triggers a fault recovery process based on the simulated fault parameters. After detecting communication data sent by an external device to the IoT device, a fault injection tool is called to inject a first simulated fault parameter into the operating system kernel so that the IO scheduling layer of the operating system kernel does not initiate an IO request for the external device and the IoT device triggers the fault recovery process.

[0071] The communication data sent by the external device to the IoT device may be IO (input and output) communication data, such as reading a file, writing a file, reading in data, writing out data, etc. Correspondingly, the first simulated fault parameter may be a fault parameter related to an IO request, for example, a specified frequency, number of times, random probability, etc. may be used to call a generic make request function in the IO request phase to perform IO request fault injection on the operating system kernel.

[0072] For example, an external device can first initiate an IO request to the operating system kernel of the IoT device through a generic make request function. The IO scheduling layer in the operating system kernel of the IoT device responds to the IO request initiated by the external device and forwards the IO request to a block device (blk queue bio) with a request queue through a generic make request function.

[0073] In IO requests, if the read operation does not hit the cache, the read page function in address space operations (used to operate the page mapped to the memory of the file, such as writing the changes in the memory back to the file, reading data from the file to the page buffer, etc.) can initiate a block device read request; the write operation initiates a block device request when the cache is replaced or PDFlush (used to synchronize the contents in the memory with the file system) wakes up. In the process of initiating a block device read request or write request, the bio structure (a core data structure of the general block layer in the Linux kernel, which describes the I / O operation of the block device and connects the memory buffer with the block device) can be constructed according to the requirements. The bio structure can contain information such as read and write addresses, lengths, target devices, and callback functions. After constructing the bio structure, the request can be forwarded to a specific block device through the generic make request function, and then make requestfn (a function used to process data requests for block devices, which will obtain the request queue pointed to by bio and send bio through the request queue) is called to process bio. During the processing, a new bio may be generated, that is, make request fn will recursively call the generic make request function. Since recursive calls are a special kind of nested calls, that is, a function calls itself or calls other functions and then calls itself again, and functions call each other to generate a loop, in order to prevent recursive call errors that may occur when block devices execute requests, only one generic make request is allowed to be called in the process at a time. For this reason, a bio waiting list is defined in the process structure: bio list, which is the request queue in the block device mentioned above. At the same time, the bios in the list will be divided into "active" and "inactive" states. The active state means that the process is already calling generic make request. In the active state, the generic make request function will first complete the current bio, that is, the IO scheduling layer in the operating system kernel initiates an IO request for the current bio of the external device to the driver layer in the operating system kernel, and then processes the bios in the same request queue one by one. Until the generic make request completes the processing of all bios in the same request queue (clears to zero), it means that the "current execution cycle" of the generic make request has been completed, that is, the current IO request phase is completed.

[0074] The above process is the IO request stage in the IO process. If an IO request fault is injected into the operating system kernel during this stage, the IO scheduling layer of the operating system kernel will not be able to initiate an IO request for the external device, and will not be able to forward the IO request to the corresponding request queue, thereby detecting the performance of the target IoT device under such a fault.

[0075] Among them, the fault injection tool used can be the fault-injection fault injection tool that comes with the Linux operating system, or other tools such as executable programs with the function of injecting simulated fault parameters. The tester can connect to the target IoT device and communicate by entering the parameters of the corresponding remote connection tool (ssh connection parameters) in the performance test page, such as the website address (ip), user name (user), password (password), computer port number (port), etc. of the target IoT device, call the fault injection tool that comes with the operating system of the target IoT device, and then enter the corresponding script and instruction to the operating system kernel of the target IoT device through CMD (command prompt, a work prompt that prompts for command input in the operating system), such as inputting an instruction that cannot call the readpage function to initiate a block device read request, or inputting a script that cannot build a bio structure, or inputting an instruction that the generic make request function cannot forward the request to a specific block device, or inputting a script that does not call the make request fn function to process bio and other fault parameters, and this application does not impose any restrictions on this.

[0076] In terms of fault injection strategy, the probability of fault occurrence, duration of fault, number of faults, etc. can be set. For example, under a probability of 50%, an exception is injected into the generic make request at a frequency of 10 times per second, and the total number of injections is 1000 times. This is to monitor whether the target IoT device will not have a request corresponding to the input of the external device when injecting simulated fault parameters, whether it can return to normal after stopping injecting simulated fault parameters, and whether the system will not crash when injecting simulated fault parameters multiple times or at a high frequency, so as to judge the robustness and stability of the target IoT device system. If the judgment result is that the robustness and stability of the target IoT device system are not high, it is necessary to further transform or optimize the system.

[0077] In other embodiments, simulated fault parameters are injected into the operating system kernel of the IoT device so that the IoT device triggers a fault recovery process based on the simulated fault parameters. After detecting that the IO scheduling layer of the operating system kernel initiates an IO request for an external device, a fault injection tool is called to inject a second simulated fault parameter into the operating system kernel, so that the IO scheduling layer of the operating system kernel fails to respond to the IO request within a set time and the IoT device triggers a fault recovery process.

[0078] Correspondingly, the second simulated fault parameter may be a fault parameter related to the IO request. For example, a specified frequency, number of times, random probability, etc. may be used to call the blk complete request function during the IO interrupt phase to perform IO timeout fault injection on the operating system kernel.

[0079] For example, after the IO request stage, the operating system kernel will complete the hard interrupt according to the IO request, that is, start the driver layer (scsi) and the hardware layer in the operating system kernel, call the scsi request fn function (driver request processing function) in the driver layer to scan the IO requests in the same request queue, and convert the specific IO request into a driver command (scsi command) that the driver layer can recognize, and then hang the received scsi command in the queue maintained by the driver layer itself, and then start the DMA process (Direct Memory Access, grouped data transfer mode, or direct memory operation) to send the data in the scsi command to the specific hardware layer. After the DMA is completed, the DMA controller interrupts the CPU and tells the CPU that the DMA process is over, that is, the IO hard interrupt is completed. After that, the lower half of the interrupt (soft interrupt) for DMA end can be set in the interrupt context, that is, by calling the callback function for scsi command end, the callback function for driver completion can be called (it can be the driver completion function scsi done), and the scsi done function is used to call the IO completion information returned by the blk complete request function to end the IO request. Each IO request here can correspond to a bio request queue. Therefore, in the process of ending the request, it is also necessary to call back the bio callback function in each request to end the specific bio. When all bios in the same bio request queue are ended, the IO completes the soft interrupt, which means that the current IO is completed, and then the relevant IO completion information is returned to the external device.

[0080] The above process is the IO interruption stage in the IO process. In the above process, if an IO interruption timeout fault is injected into the blk completerequest function so that the IO scheduling layer of the operating system kernel cannot respond to the IO request within the set time, the performance of the target IoT device under this fault condition can be detected.

[0081] Among them, the fault injection tool used can be the fault-injection fault injection tool that comes with the Linux operating system, or other tools such as executable programs with the function of injecting simulated fault parameters. The tester can also connect to the target IoT device and communicate by entering the corresponding remote connection tool parameters (ssh connection parameters) in the performance test page, and then enter the corresponding scripts and instructions to the operating system kernel of the target IoT device through CMD, such as setting the blk_complete_request abnormal return, so that all processes' IO requests respond to timeouts and cannot trigger soft interrupts. The type of the input simulated fault parameter can be the input of the IO request conversion driver command that cannot call the scsi request fn function in the driver layer, or the input of the callback function instruction that cannot call the end of the scsi command, or the input of the script that cannot call the IO completion information returned by the blk complete request function using the scsi done function, and this application does not impose any restrictions on this.

[0082] In terms of fault injection strategy, the number of fault injections, the frequency of fault injections, the probability of fault injections, etc. can be set. For example, under a probability of 20%, an exception is injected into the blk complete request exception at a frequency of 10 times per second, and the total number of injections is 1000 times. This is to monitor whether the target IoT device will not have a request corresponding to the input of the external device when injecting simulated fault parameters, whether it can return to normal after stopping injecting simulated fault parameters, and whether the system will not crash when injecting simulated fault parameters multiple times or at a high frequency, so as to judge the robustness and stability of the target IoT device system. If the judgment result is that the robustness and stability of the target IoT device system are not high, it is necessary to further transform or optimize the system.

[0083] In one possible implementation, before injecting simulated fault parameters into the operating system kernel of the IoT device so that the IoT device triggers a fault recovery process according to the simulated fault parameters, the number of injections and / or injection frequency of the simulated fault parameters into the operating system kernel can be determined based on the configured fault occurrence probability, and then the simulated fault parameters are injected into the operating system kernel of the IoT device according to the determined number of injections and / or injection frequency, so that the IoT device triggers a fault recovery process according to the simulated fault parameters.

[0084] In step S303, the fault recovery process of the IoT device is monitored, and a performance test result corresponding to the IoT device is generated according to the monitored operating load index. The performance test result may be a test result that can characterize the performance and stability of the IoT device, such as a performance and stability test result generated according to the monitored parameters related to the performance and stability of the IoT device.

[0085] Among them, the execution subject that monitors the fault recovery process of the IoT device can be an execution module configured in the target IoT device, or an execution module configured in the control device. For example, the control device can formulate a monitoring execution code and send it to the target IoT device. The execution module in the target IoT device monitors according to the execution code and feeds back the monitoring results to the control device, and then displays the monitoring results on the performance test page in the control device; or after the control device formulates the monitoring execution code, the execution module in the control device directly monitors the target IoT device, and then displays the monitoring results on the performance test page. The application does not impose any restrictions on the specific monitoring method, monitoring execution subject, and monitoring display effect.

[0086] In addition, in the embodiments of the present application, the execution entities of the three links of load pressure operation, fault injection operation and monitoring operation can be deployed according to actual needs. For example, the load pressure operation can be performed by the execution module A in the control device, the fault injection operation can be performed by the execution module B in the Internet of Things device, and the monitoring operation can be performed by the execution module C in the control device. The present application does not impose any restrictions on this.

[0087] In one possible implementation, before monitoring the fault recovery process of an IoT device, an operating load indicator related to the operating status of the hardware resources of the IoT device can be determined first. The operating load indicator may include an operating load indicator corresponding to the hardware resources, and an operating load indicator corresponding to an execution object related to the hardware resources, wherein the execution object may include at least one of a file system, a host, and a process; and then the fault recovery process of the IoT device is monitored according to the predetermined operating load indicator.

[0088] The operating load indicators involved in the embodiments of the present application may be parameters that can reflect the performance of the IoT device, including memory performance parameters, host performance parameters, process performance parameters, etc. Testers can judge the stability or robustness of the IoT device based on these parameters, and the present application does not impose any restrictions on this.

[0089] The operation load index corresponding to the hardware resource may be a memory operation load index or a disk operation load index, such as memory read and write execution time, memory read and write average latency, memory request release specified size, disk read and write average latency, disk throughput, etc. These indicators or parameters are mainly used to record the performance changes of hardware resources such as memory and disk during the process of load pressure and / or injection of simulated fault parameters on the target IoT device.

[0090] The operating load indicators corresponding to the execution objects related to hardware resources can be operating load indicators of file systems, hosts, processes, etc., such as average file read and write latency, file throughput, core process status operating indicators, host status operating indicators, host self-recovery status indicators, process self-recovery status indicators, host memory performance comparison parameters after recovery, etc. These indicators or parameters are mainly used to record the performance changes of file systems, hosts or processes, etc. during the process of load pressure and / or injection of simulated fault parameters to the target IoT device. Through these changes, it is possible to monitor whether the host is healthy and stable during operation, or monitor the operation of the core processes in the target IoT device, the degree of impact during fault injection, and whether the file system, host or process can self-recover in the event of a fault, and whether the state after recovery is stable. The above monitoring indicators can be adjusted according to actual service needs, and this application does not impose any restrictions on this.

[0091] In some embodiments, the performance test results of the IoT device after the above-mentioned solution test can also be provided, so as to execute the corresponding performance optimization strategy according to the performance test results. That is, after obtaining the above-mentioned monitoring indicators, the corresponding performance test results can be displayed to the tester through the aforementioned performance test page, so that the tester can execute the corresponding strategy on the target IoT device according to the performance test results, such as shutting down, replacing or optimizing the performance of the IoT device. Among them, the performance test results include the performance and stability test results of the IoT device.

[0092] Accordingly, the hardware resources may include storage resources, which may include memory and / or disk, and the operation load state corresponding to the storage resources may include the operation load state corresponding to at least one of memory reading and writing, memory application, memory release, and disk reading and writing. That is, pressurizing the operation load of the hardware resources of the IoT device may be to increase the load pressure of the storage resources in the IoT device by load pressurization.

[0093] The present application also provides a method for testing the performance of an IoT device. Figure 4 The flowchart of the IoT device performance testing method 400 according to an embodiment of the present application is shown. The method 400 may include:

[0094] In step S401, the storage resources of the Internet of Things device are controlled to change the operating load state according to the operating load condition for testing, so that the operating load of the storage resources of the Internet of Things device increases.

[0095] In step S402, simulated fault parameters are injected into the operating system kernel of the IoT device, so that an IO scheduling layer of the operating system kernel in the IoT device fails to execute an IO request, thereby triggering the IoT device to trigger a fault recovery process.

[0096] In step S403, the fault recovery process of the IoT device is monitored, and a performance test result corresponding to the IoT device is generated according to the monitored operation load index, wherein the performance test result may include performance and stability test results of the IoT device.

[0097] The embodiment of the present application can control the storage resources of the IoT device to change the operating load state according to the operating load conditions used for testing, so that the operating load of the storage resources of the IoT device increases. The storage resources of the IoT device may include internal storage resources (such as memory, cache and other storage resources) or external storage resources (such as disk, file system and other storage resources). By increasing the operating load of the storage resources of the IoT device, the operating load state of the storage resources of the IoT device in a complex application scenario is simulated.

[0098] In addition, by simulating the load state of storage resources, such as the continuous reading and writing state of the simulated memory, or the continuous reading and writing state of the file system, it is also possible to simulate and increase the pressure on other hardware resources of the target IoT device, such as the CPU load, and then test the performance of the target IoT device by monitoring the CPU-related operating status parameters.

[0099] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides an IoT device performance testing device. Figure 5 The structure diagram of an IoT device performance testing device 500 according to an embodiment of the present application is shown. The device 500 may include:

[0100] A load state changing module 501, configured to control the IoT device to change its operating load state according to the operating load condition for testing, so as to increase the operating load of the hardware resources of the IoT device;

[0101] A fault parameter injection module 502 is used to inject simulated fault parameters into the operating system kernel of the IoT device, so that the IoT device triggers a fault recovery process according to the simulated fault parameters;

[0102] The state parameter monitoring module 503 is used to monitor the fault recovery process of the IoT device and generate a performance test result corresponding to the IoT device according to the monitored operating load index.

[0103] In a possible implementation, the load state changing module 501 may include:

[0104] A load condition receiving submodule, used for receiving the operating load condition for testing input based on the performance test page;

[0105] The load state changing submodule is used to provide the operating load condition for testing to the Internet of Things device, so that the Internet of Things device changes the operating load state corresponding to the hardware resource according to the operating load condition for testing.

[0106] In some embodiments, the performance test page provides at least one operation load mode, and the operation load mode corresponds to recording a combination of condition contents included in the operation load condition. The load condition receiving submodule may include:

[0107] The load condition determination unit determines the operating load condition for testing according to the selected and / or modified operating load mode.

[0108] In a possible implementation, the running load condition includes at least one of a running load execution mode, a running load amount, a load growth adjustment frequency, and a number of threads for controlling load growth adjustment.

[0109] In a possible implementation, the fault parameter injection module 502 may include:

[0110] The first simulated fault parameter injection submodule is used to call a fault injection tool to inject a first simulated fault parameter into the operating system kernel after detecting communication data sent by an external device to the Internet of Things device, so that the IO scheduling layer of the operating system kernel does not initiate an IO request for the external device and the Internet of Things device triggers a fault recovery process.

[0111] In a possible implementation, the fault parameter injection module 502 may include:

[0112] The second simulated fault parameter injection submodule is used to call the fault injection tool to inject the second simulated fault parameter into the operating system kernel after detecting that the IO scheduling layer of the operating system kernel initiates an IO request for an external device, so that the IO scheduling layer of the operating system kernel fails to respond to the IO request within a set time and the Internet of Things device triggers a fault recovery process.

[0113] In a possible implementation, before injecting simulated fault parameters into the operating system kernel of the IoT device so that the IoT device triggers a fault recovery process according to the simulated fault parameters, the apparatus may further include:

[0114] A fault parameter injection determination module, used to determine the number of injections and / or injection frequency of injecting simulated fault parameters into the operating system kernel according to the configured fault occurrence probability;

[0115] The above-mentioned fault parameter injection module 502 may include:

[0116] The simulated fault parameter injection submodule is used to inject simulated fault parameters into the operating system kernel of the Internet of Things device according to the determined injection number and / or injection frequency, so that the Internet of Things device triggers a fault recovery process according to the simulated fault parameters.

[0117] In a possible implementation, before monitoring the fault recovery process of the IoT device, the apparatus may further include:

[0118] An operation load indicator determination module, used to determine an operation load indicator related to the operation state of the hardware resources of the Internet of Things device, wherein the operation load indicator includes an operation load indicator corresponding to the hardware resources and an operation load indicator corresponding to an execution object related to the hardware resources, wherein the execution object includes at least one of a file system, a host, and a process;

[0119] The above-mentioned state parameter monitoring module 503 may include:

[0120] The fault recovery process monitoring submodule is used to monitor the fault recovery process of the Internet of Things device according to a predetermined operating load indicator.

[0121] In a possible implementation, the above device may further include:

[0122] The test result providing module is used to provide the performance test result so as to execute the corresponding performance optimization strategy according to the performance test result.

[0123] In one possible implementation, the hardware resources include storage resources, the storage resources include memory and / or disk, and the operating load status corresponding to the storage resources includes the operating load status corresponding to at least one process of memory reading and writing, memory application, memory release, and disk reading and writing.

[0124] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides an IoT device performance testing device. Figure 6 The structure block diagram of an IoT device performance testing device 600 according to an embodiment of the present application is shown. The device 600 may include:

[0125] A load state growth module 601, used for controlling the storage resources of the IoT device to change the operation load state according to the operation load condition for testing, so that the operation load of the storage resources of the IoT device increases;

[0126] A fault parameter injection module 602 is used to inject simulated fault parameters into the operating system kernel of the IoT device, so as to cause an execution fault of an IO request in the IO scheduling layer of the operating system kernel in the IoT device, thereby triggering the IoT device to trigger a fault recovery process;

[0127] The state parameter monitoring module 603 is used to monitor the fault recovery process of the IoT device and generate a performance test result corresponding to the IoT device according to the monitored operating load index.

[0128] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0129] Figure 7 FIG. 1 is a block diagram of an electronic device used to implement an embodiment of the present application. Figure 7 As shown, the electronic device includes: a memory 701 and a processor 702. The memory 701 stores a computer program that can be run on the processor 702. When the processor 702 executes the computer program, the method in the above embodiment is implemented. The number of the memory 701 and the processor 702 can be one or more.

[0130] The electronic device also includes:

[0131] The communication interface 703 is used to communicate with external devices and perform data exchange transmission.

[0132] If the memory 701, the processor 702 and the communication interface 703 are implemented independently, the memory 701, the processor 702 and the communication interface 703 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0133] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0134] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0135] An embodiment of the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method provided in any embodiment of the present application is implemented.

[0136] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0137] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the present application.

[0138] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0139] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0140] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0141] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0142] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0143] Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0144] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or used in combination with these instruction execution systems, devices or apparatuses.

[0145] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0146] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0147] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for testing performance of an IoT device, comprising: Controlling the IoT device to change the operating load state according to the operating load condition for testing, so as to increase the operating load of the hardware resources of the IoT device; Injecting simulated fault parameters into the operating system kernel of the IoT device, so that the IoT device triggers a fault recovery process according to the simulated fault parameters; The fault recovery process of the IoT device is monitored, and a performance test result corresponding to the IoT device is generated based on the monitored operating load index.

2. The method according to claim 1, wherein: The step of controlling the IoT device to change the operating load state according to the operating load condition for testing includes: receiving an operating load condition for testing based on an input from a performance test page; The operating load condition for testing is provided to the Internet of Things device, so that the Internet of Things device changes the operating load state corresponding to the hardware resource according to the operating load condition for testing.

3. The method according to claim 2, wherein: The performance test page provides at least one operation load mode, and the operation load mode records the condition content combination included in the operation load condition. The receiving of the operation load condition for testing input based on the performance test page includes: Based on the selected and / or modified operating load pattern, the operating load conditions for the test are determined.

4. The method according to claim 1, wherein: The running load condition includes at least one of a running load execution mode, a running load amount, a load growth adjustment frequency, and a number of threads for controlling load growth adjustment.

5. The method according to claim 1, wherein: The injecting of simulated fault parameters into the operating system kernel of the IoT device so that the IoT device triggers a fault recovery process according to the simulated fault parameters comprises: After detecting the communication data sent by the external device to the Internet of Things device, a fault injection tool is called to inject a first simulated fault parameter into the operating system kernel so that the IO scheduling layer of the operating system kernel does not initiate an IO request for the external device and the Internet of Things device triggers a fault recovery process.

6. The method according to claim 1, wherein: The injecting of simulated fault parameters into the operating system kernel of the IoT device so that the IoT device triggers a fault recovery process according to the simulated fault parameters comprises: After detecting that the IO scheduling layer of the operating system kernel initiates an IO request for an external device, a fault injection tool is called to inject a second simulated fault parameter into the operating system kernel, so that the IO scheduling layer of the operating system kernel fails to respond to the IO request within a set time and the Internet of Things device triggers a fault recovery process.

7. The method according to claim 1, wherein: Before injecting simulated fault parameters into the operating system kernel of the IoT device so that the IoT device triggers a fault recovery process according to the simulated fault parameters, the method further includes: Determining the number of injections and / or the injection frequency of injecting simulated fault parameters into the operating system kernel according to the configured fault occurrence probability; The injecting of simulated fault parameters into the operating system kernel of the IoT device so that the IoT device triggers a fault recovery process according to the simulated fault parameters comprises: According to the determined injection times and / or injection frequency, simulated fault parameters are injected into the operating system kernel of the IoT device, so that the IoT device triggers a fault recovery process according to the simulated fault parameters.

8. The method according to claim 1, wherein: Before monitoring the fault recovery process of the IoT device, the method further includes: Determine an operation load indicator related to the operation state of the hardware resource of the Internet of Things device, wherein the operation load indicator includes an operation load indicator corresponding to the hardware resource and an operation load indicator corresponding to an execution object related to the hardware resource, wherein the execution object includes at least one of a file system, a host, and a process; The monitoring of the fault recovery process of the IoT device includes: The fault recovery process of the IoT device is monitored according to a predetermined operating load indicator.

9. The method according to claim 1, wherein: The method further comprises: The performance test results are provided so as to execute corresponding performance optimization strategies according to the performance test results.

10. The method according to any one of claims 1 to 9, wherein: The hardware resources include storage resources, and the storage resources include memory and / or disk. The operating load status corresponding to the storage resources includes the operating load status corresponding to at least one process of memory reading and writing, memory application, memory release, and disk reading and writing.

11. A method for testing performance of an IoT device, comprising: Controlling the storage resources of the IoT device to change the operating load state according to the operating load condition for testing, so that the operating load of the storage resources of the IoT device increases; Injecting simulated fault parameters into the operating system kernel of the IoT device, so as to cause an execution failure of the IO request in the IO scheduling layer of the operating system kernel in the IoT device, thereby triggering the IoT device to trigger a fault recovery process; The fault recovery process of the IoT device is monitored, and a performance test result corresponding to the IoT device is generated based on the monitored operating load index.

12. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 11 when executing the computer program.

13. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images