Test method for embedded device combination

Abstract

The invention relates to the technical field of embedded device testing, in particular to a testing method of an embedded device combination. The method has the advantages that aiming at the problem that the whole test result is invalid due to a single-point fault in the prior art, the test plan is executed in a sandbox mode, each test task is ensured to be mutually independent and have reusability, the test tasks are sent to a plurality of devices for operation, the influence of the single device fault on the whole test is avoided, and the test efficiency is improved. Test results are uniformly synchronized and integrated in the later period, meanwhile, through a real-time monitoring and fault isolation mechanism, the reliability and flexibility of the system are improved, the framework supports multi-task parallel execution and instant equipment resource expansion, adapts to test requirements of different scales and types, remarkably improves the test efficiency, optimizes the resource utilization rate, and improves the test efficiency. And the comprehensiveness and the accuracy of a test result are ensured, and an efficient, reliable and extensible equipment test solution is provided.

Description

Technical Field

[0001] This invention relates to the field of embedded device testing technology, and in particular to a testing method for an embedded device assembly. Background Technology

[0002] In existing technologies, testing multiple embedded devices faces severe constraints due to system environment and peripheral conditions, making independent testing of specific devices impossible. This limitation makes the testing process extremely vulnerable to single points of failure; if a device malfunctions, it not only directly contaminates the test results but may even cause the entire testing process to fail completely. Worse still, significant human and material resources are wasted on ineffective test attempts without obtaining any reliable test data, severely hindering development progress and significantly increasing costs. This situation exposes the high risk and inefficiency of existing testing methods in complex environments. Summary of the Invention

[0003] The present invention aims to solve the problem that in the prior art, the testing of multiple embedded devices is severely constrained by the system environment and peripheral conditions, making it impossible to test specific devices independently. This results in the testing process being extremely vulnerable to single-point failures. Once a device malfunctions, it not only directly contaminates the test results but may even cause the entire testing process to fail completely.

[0004] To achieve the above objectives, the present invention provides a testing method for embedded device assemblies, comprising the following steps: S1: Task Reception and Parsing: The test task is pushed to the local auto_test_server through the third-party service of Airflow. After receiving the task, the auto_test_server parses the test task content and queries the local test case cache to confirm whether the relevant test cases have been cached. S2: Test Case Query and Device Allocation: If the local cache exists and has not expired, the test case and its related information are read directly; if the cache does not exist or has expired, the test case library is queried through the intranet workbench to obtain detailed test case information, and the types and models of devices required for test plan execution are also obtained; ATS calls the device service to query the availability of devices required for the current test. If there are enough available devices, ATS will occupy device resources and continue to execute the task; if there are insufficient devices, ATS will register a task retry mechanism with Airflow and re-add the task to the queue to wait for the next execution. S3: Sandboxed Execution: After successful device acquisition, ATS encapsulates the test plan into an independent sandbox environment for execution, ensuring the isolation, independence, and reusability of task execution; the test task execution process is completed within the sandbox, and test results are generated after the task is completed; S4: Result Upload and Classification: The sandbox pushes test results to ATS via RabbitMQ. ATS parses and classifies the test results, including the results of individual test cases and the results of the overall test plan. If a task is suspended due to insufficient device resources or other reasons, ATS re-registers it to the Airflow queue to trigger a retry. After the test is completed, the system automatically releases the occupied device resources.

[0005] Preferably, in any of the above solutions, the interaction logic between the ATS and the workbench includes: 1) Login state update and backup: The ATS service receives the login state update command issued by Airflow and immediately backs up the received login state information to Redis; 2) Test Plan and Test Case Acquisition: After confirming the login status is correct, ATS obtains the relevant test cases under the test plan by calling relevant interfaces or GraphQL query methods, and extracts and retains the key information related to execution in the test cases; 3) Sandbox Execution Environment Preparation: Based on complete test plan and test case information, ATS prepares a brand new sandbox execution environment for each test plan. During initialization, the sandbox environment reads the aforementioned reserved equipment requirements and other test-related information from ATS's internal storage. Preferably, as described in any of the above schemes, the execution logic within the sandbox includes: 1) Test plan reception and test case synchronization: ATS receives test plan requests, starts the task scheduling process, and extracts relevant test cases from the test plan and synchronizes them to Redis; 2) Creation of the sandbox runtime environment: According to the requirements of the test plan, create an isolated sandbox runtime environment and execute each test case in sequence within the sandbox; 3) Device usage request and task queuing: When a test case needs to use a peripheral device, the sandbox sends a device usage request to the device service. The device service ensures that the device is not occupied by other plans before the current test plan is completed, based on the policy of whether the device is exclusive. It also allows the device's tasks to be queued for execution by detecting the device's idle status. The device usage result is sent back to the test sandbox via WebSocket. 4) Test execution without peripherals: If no peripheral support is required, the test method associated with the sandbox creation is called directly to execute the test, and the execution result is sent back to ATS through RabbitMQ message queue; 5) Test Result Summary and Display: After all test cases have been executed, the results of each test case execution are summarized to form the final test plan report, and the summary results are sent back to ATS.

[0006] Preferably, in any of the above schemes, the third-party service of Airflow in the task receiving and parsing step is used to push the test task to ATS according to a preset scheduling strategy, the preset scheduling strategy including parameters such as task priority and execution time window.

[0007] Preferably, in the test case query and device allocation step, the ATS uses a hash algorithm to quickly locate and retrieve test cases when querying the local test case cache.

[0008] Preferably, in the sandboxing execution step, the sandbox environment is constructed using container technology, and each sandbox environment has an independent operating system kernel and file system to ensure the isolation of task execution.

[0009] Preferably, in the result uploading and classification processing step, ATS uses machine learning algorithms to automatically classify and analyze the test results when parsing them.

[0010] Preferably, in the login status update and backup step, ATS uses an encryption algorithm to encrypt and store the login status information.

[0011] Preferably, in the device usage request and task queuing step, the device service uses a load balancing algorithm to queue and allocate tasks for the device.

[0012] Preferably, any of the above solutions includes monitoring and handling abnormal situations during the testing process. When an abnormal situation is detected, a corresponding alarm mechanism is automatically triggered, and the abnormal information is recorded for subsequent analysis and processing.

[0013] The present invention has the following advantages: 1. The testing method for this embedded device combination, through sandbox execution, ensures that each test task is independent and reusable, avoiding overall test failure caused by a single point of failure.

[0014] 2. The testing method for this embedded device combination distributes the test tasks to multiple devices for execution through distributed testing, thereby mitigating the failure risk of individual devices and improving the reliability of the testing process.

[0015] 3. The testing method for this embedded device combination adopts a real-time monitoring and fault isolation mechanism through result synchronization and integration to ensure the comprehensiveness and accuracy of the test results.

[0016] 4. The testing method for this embedded device combination supports multi-task parallel execution and real-time expansion of device resources, adapting to different scales and types of testing needs and optimizing resource utilization.

[0017] 5. The testing method for this embedded device combination significantly improves testing efficiency and ensures the reliability and flexibility of the system in various environments. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the main operation of the present invention; Figure 2 This is a diagram of the execution logic inside the sandbox of the present invention. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited thereto.

[0020] like Figures 1 to 2 As shown, a testing method for an embedded device assembly includes the following steps: S1: Task Reception and Parsing: Test tasks are pushed to the local auto_test_server via Airflow's third-party service. After receiving the task, the auto_test_server parses the test task content and queries the local test case cache to confirm whether the relevant test cases have been cached. S2: Test Case Query and Device Allocation: If the local cache exists and has not expired, the test case and its related information are read directly; if the cache does not exist or has expired, the test case library is queried through the intranet workbench to obtain detailed test case information, and the types and models of devices required for test plan execution are also obtained; ATS calls the device service to query the availability of devices required for the current test. If there are enough available devices, ATS will occupy device resources and continue to execute the task; if there are insufficient devices, ATS will register a task retry mechanism with Airflow and re-add the task to the queue to wait for the next execution. S3: Sandboxed Execution: After successful device acquisition, ATS encapsulates the test plan into an independent sandbox environment for execution, ensuring the isolation, independence, and reusability of task execution; the test task execution process is completed within the sandbox, and test results are generated after the task is completed; S4: Result Upload and Classification: The sandbox pushes test results to ATS via RabbitMQ. ATS parses and classifies the test results, including the results of individual test cases and the results of the overall test plan. If a task is suspended due to insufficient device resources or other reasons, ATS re-registers it to the Airflow queue to trigger a retry. After the test is completed, the system automatically releases the occupied device resources.

[0021] As an optional technical solution of the present invention, the interaction logic between ATS and workbench includes: 1) Login state update and backup: The ATS service receives the login state update command issued by Airflow and immediately backs up the received login state information to Redis; 2) Test Plan and Test Case Acquisition: After confirming the login status is correct, ATS obtains the relevant test cases under the test plan by calling relevant interfaces or GraphQL query methods, and extracts and retains the key information related to execution in the test cases; 3) Preparation of the sandbox execution environment: Based on the complete test plan and test case information, ATS prepares a brand new sandbox execution environment for each test plan. During initialization, the sandbox environment reads the aforementioned reserved equipment requirements and other test-related information from ATS's internal storage.

[0022] As an optional technical solution of the present invention, the execution logic inside the sandbox includes: 1) Test plan reception and test case synchronization: ATS receives test plan requests, starts the task scheduling process, and extracts relevant test cases from the test plan and synchronizes them to Redis; 2) Creation of the sandbox runtime environment: According to the requirements of the test plan, create an isolated sandbox runtime environment and execute each test case in sequence within the sandbox; 3) Device usage request and task queuing: When a test case needs to use a peripheral device, the sandbox sends a device usage request to the device service. The device service ensures that the device is not occupied by other plans before the current test plan is completed, based on the policy of whether the device is exclusive. It also allows the device's tasks to be queued for execution by detecting the device's idle status. The device usage result is sent back to the test sandbox via WebSocket. 4) Test execution without peripherals: If no peripheral support is required, the test method associated with the sandbox creation is called directly to execute the test, and the execution result is sent back to ATS through RabbitMQ message queue; 5) Test Result Summary and Display: After all test cases have been executed, the results of each test case execution are summarized to form the final test plan report, and the summary results are sent back to ATS.

[0023] As an optional technical solution of the present invention, in the task receiving and parsing step, the third-party service of Airflow is used to push the test task to ATS according to the preset scheduling strategy. The preset scheduling strategy includes parameters such as task priority and execution time window.

[0024] As an optional technical solution of the present invention, in the test case query and device allocation step, the ATS uses a hash algorithm to quickly locate and retrieve test cases when querying the local test case cache, so as to improve query efficiency.

[0025] As an optional technical solution of the present invention, in the sandboxing execution step, the sandbox environment is constructed using container technology, and each sandbox environment has an independent operating system kernel and file system to ensure the isolation of task execution.

[0026] As an optional technical solution of the present invention, in the result uploading and classification processing steps, ATS uses machine learning algorithms to automatically classify and analyze the test results when parsing the test results, so as to improve the accuracy and efficiency of result processing.

[0027] As an optional technical solution of the present invention, in the login state update and backup step, ATS uses an encryption algorithm to encrypt and store the login state information to ensure the security of the login state.

[0028] As an optional technical solution of the present invention, in the device usage request and task queuing step, the device service uses a load balancing algorithm to queue and allocate the tasks of the device in order to optimize the utilization of device resources.

[0029] As an optional technical solution of the present invention, the testing scheme also includes monitoring and handling abnormal situations during the testing process. When an abnormal situation is detected, the corresponding alarm mechanism is automatically triggered, and the abnormal information is recorded for subsequent analysis and processing.

[0030] Example 1: Smart Home Device Combination Test Process: 1. Task Reception and Parsing: Test tasks are pushed to the local auto_test_server (ATS) via Airflow's third-party service; the task content includes scenarios for testing smart home device combinations, such as "when the user leaves home, the door lock automatically locks, the lights automatically turn off, the thermostat switches to energy-saving mode, and the camera enters monitoring mode"; after receiving the task, the ATS parses the task content and queries the local test case cache; the ATS queries the test case library through the intranet workbench to obtain detailed test case information; at the same time, it obtains the types and models of equipment required for the test plan (such as smart door lock model A, smart lighting control device model B, etc.).

[0031] 2. Use Case Query and Device Allocation: The ATS calls the device service (device_service) to query the available smart device reserves. The system detects that there are enough smart door locks, smart lighting control devices, and smart temperature control devices, but the number of smart cameras is insufficient. The ATS registers a task retry mechanism with Airflow and re-adds the task to the queue. After a period of time, device resources become available, and the ATS uses the device resources to continue executing the task.

[0032] 3. Sandboxed Execution: ATS encapsulates the test plan into an independent sandbox environment to ensure task isolation. The sandbox environment simulates a user leaving home, executing the following test cases sequentially: automatic door locking test; light turning off test; thermostat energy-saving mode test; and camera monitoring mode test. The results of each test case are sent back to ATS via RabbitMQ.

[0033] 4. Result Upload and Classification: The sandbox summarizes the test results to form an overall test plan report. After receiving the report, ATS parses and classifies the results. For example, the smart door lock test may be successful, but the smart lighting control device may experience a delay when turning off the lights. ATS saves the results of the failed test cases to the workbench and automatically generates a bug report. After the test is completed, ATS releases the occupied device resources to ensure the dynamic availability of the device pool.

[0034] Results: Sandbox testing successfully identified latency issues in the intelligent lighting control device and isolated test results from other devices. This testing method avoids the failure of the entire test due to a single device malfunction, significantly improving testing efficiency and reliability.

[0035] Example 2: Industrial Automation Equipment Test Procedure: 1. Task Reception and Parsing: Test tasks are pushed to the ATS via Airflow. The task content includes testing the robotic arm's grasping accuracy, sensor detection sensitivity, controller command response speed, and conveyor belt operational stability. After parsing the task, the ATS queries its local test case cache. Since similar test cases have been cached previously and the cache has not expired, the ATS directly reads the test cases and their related information.

[0036] 2. Use Case Query and Device Allocation: ATS calls the device service to query the currently available device resources. If the system detects that all device resources are available, ATS acquires the device resources and continues executing the task.

[0037] 3. Sandboxed Execution: ATS encapsulates the test plan into an independent sandbox environment. The sandbox environment simulates a factory production scenario, executing the following test cases sequentially: robotic arm grasping accuracy test; sensor detection sensitivity test; controller command response speed test; and conveyor belt operation stability test. The results of each test case are sent back to ATS via RabbitMQ.

[0038] 4. Result Upload and Classification: The sandbox summarizes the test results to form an overall test plan report. After receiving the report, ATS parses and classifies the results. For example, the robotic arm's grasping accuracy test may be successful, but the sensor's detection sensitivity may be insufficient under low light conditions. ATS saves the results of the failed test cases to the workbench and automatically generates a bug report. After the test is completed, ATS releases the occupied device resources to ensure the dynamic availability of the device pool.

[0039] Results: Sandbox testing successfully identified sensor performance issues under specific conditions and isolated test results from other devices. This testing method avoids the failure of the entire test due to a single device malfunction, significantly improving testing efficiency and reliability.

[0040] In summary, this invention ensures that each test task is independent and reusable through sandbox execution, avoiding overall test failure caused by single point of failure. Distributed testing distributes test tasks to multiple devices, mitigating the risk of individual device failures and improving the reliability of the testing process. Real-time monitoring and fault isolation mechanisms ensure the comprehensiveness and accuracy of test results through result synchronization and integration. It supports parallel execution of multiple tasks and instant expansion of device resources, adapting to different scales and types of testing needs, optimizing resource utilization, significantly improving testing efficiency, and ensuring the system's reliability and flexibility in various environments.

[0041] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A testing method for an embedded device assembly, characterized in that: Includes the following steps: S1: Task Reception and Parsing: The test task is pushed to the local auto_test_server through the third-party service of Airflow. After receiving the task, the auto_test_server parses the test task content and queries the local test case cache to confirm whether the relevant test cases have been cached. S2: Test Case Query and Device Allocation: If the local cache already exists and has not expired, directly read the test cases and their related information; If the cache does not exist or has expired, the test case library is queried through the intranet workbench to obtain detailed test case information, as well as the types and models of equipment required for test plan execution. ATS calls the device service to query the availability of equipment required for the current test. If there are enough available devices, ATS will occupy the device resources and continue to execute the task. If there are not enough devices, ATS will register the task retry mechanism with Airflow and add the task back to the queue to wait for the next execution. S3: Sandboxed Execution: After successful device acquisition, ATS encapsulates the test plan into an independent sandbox environment for execution, ensuring the isolation, independence, and reusability of task execution; the test task execution process is completed within the sandbox, and test results are generated after the task is completed; S4: Result Upload and Classification Processing: The sandbox pushes the test results to ATS via RabbitMQ. ATS parses and classifies the test results, including the results of individual test cases and the results of the overall test plan. If a task is suspended due to insufficient device resources or other reasons, ATS will re-register it with the Airflow queue to trigger a retry. After the test is completed, the system will automatically release the device resources it occupies.

2. The test method for embedded device assemblies according to claim 1, characterized in that: The interaction logic between the ATS and the workbench includes: 1) Login state update and backup: The ATS service receives the login state update command issued by Airflow and immediately backs up the received login state information to Redis; 2) Test Plan and Test Case Acquisition: After confirming the login status is correct, ATS obtains the relevant test cases under the test plan by calling relevant interfaces or GraphQL query methods, and extracts and retains the key information related to execution in the test cases; 3) Preparation of the sandbox execution environment: Based on the complete test plan and test case information, ATS prepares a brand new sandbox execution environment for each test plan. During initialization, the sandbox environment reads the aforementioned reserved equipment requirements and other test-related information from ATS's internal storage.

3. The test method for embedded device assemblies according to claim 1, characterized in that: The execution logic within the sandbox includes: 1) Test plan reception and test case synchronization: ATS receives test plan requests, starts the task scheduling process, and extracts relevant test cases from the test plan and synchronizes them to Redis; 2) Creation of the sandbox runtime environment: According to the requirements of the test plan, create an isolated sandbox runtime environment and execute each test case in sequence within the sandbox; 3) Device usage request and task queuing: When a test case needs to use a peripheral device, the sandbox sends a device usage request to the device service. The device service ensures that the device is not occupied by other plans before the current test plan is completed, based on the policy of whether the device is exclusive. It also allows the device's tasks to be queued for execution by detecting the device's idle status. The device usage result is sent back to the test sandbox via WebSocket. 4) Test execution without peripherals: If no peripheral support is required, the test method associated with the sandbox creation is called directly to execute the test, and the execution result is sent back to ATS through RabbitMQ message queue; 5) Test Result Summary and Display: After all test cases have been executed, the results of each test case execution are summarized to form the final test plan report, and the summary results are sent back to ATS.

4. The test method for embedded device assemblies according to claim 1, characterized in that: In the task receiving and parsing step, Airflow's third-party service is used to push test tasks to ATS according to a preset scheduling strategy. The preset scheduling strategy includes parameters such as task priority and execution time window.

5. The test method for embedded device assemblies according to claim 1, characterized in that: In the test case query and device allocation steps, ATS uses a hash algorithm to quickly locate and retrieve test cases when querying the local test case cache.

6. The test method for embedded device assemblies according to claim 1, characterized in that: In the sandboxing execution step, the sandbox environment is built using container technology. Each sandbox environment has an independent operating system kernel and file system to ensure the isolation of task execution.

7. The test method for embedded device assemblies according to claim 1, characterized in that: In the result uploading and classification processing steps, ATS uses machine learning algorithms to automatically classify and analyze the test results when parsing them.

8. The test method for embedded device combinations according to claim 2, characterized in that: In the login status update and backup step, ATS uses an encryption algorithm to encrypt and store the login status information.

9. The test method for embedded device combinations according to claim 3, characterized in that: In the device usage request and task queuing steps, the device service uses a load balancing algorithm to queue and allocate tasks for the device.

10. The test method for embedded device assemblies according to claim 1, characterized in that: The testing plan also includes monitoring and handling abnormal situations during the testing process. When an abnormal situation is detected, the corresponding alarm mechanism is automatically triggered, and the abnormal information is recorded for subsequent analysis and processing.

Images