Cyber physical integrated testbed architecture based on mecha master

Abstract

This invention relates to a cyber-physical testbed architecture based on the DJI RoboMaster, comprising an application support framework and an environment synchronization execution framework, which respectively provide execution support for the software support components and the runtime environment in a cyber-physical fusion system. This invention combines the software-in-the-loop (Software-in-the-Loop) and hardware-in-the-loop (HIL) runtime environments during CPS application development. While preserving the real hardware-in-the-loop software-hardware interaction process, it leverages the low cost and secure operation of the software-in-the-loop virtual environment, ensuring the security, observability, and operability of the software execution process during CPS application development.

Description

Technical Field

[0001] This invention relates to the field of cyber-physical system development and application, specifically to a cyber-physical integrated testbed architecture based on DJI RoboMaster. Background Technology

[0002] Cyber-Physical Systems (CPS) are engineering systems composed of a set of highly integrated software components and a physical environment. Unlike traditional software, CPS applications, in addition to functional tasks, also undertake two types of interactive tasks connecting information and the physical space: "sensing" and "execution." Sensing tasks can be viewed as the CPS application reading from the environment, while execution tasks can be viewed as the CPS application outputting to the environment. Due to the highly dynamic and uncertain nature of the physical space, the interaction between CPS applications and the environment is generally uncertain, posing challenges to the development process of CPS applications.

[0003] To develop CPS applications efficiently and securely, the existing CPS software development process can be divided into two phases based on its development environment: Software-in-the-Loop (SiL) and Hardware-in-the-Loop (HiL). Their characteristics are as follows: The SiL phase typically uses simulation or emulation to build the CPS application's runtime environment, thereby reducing the cost of setting up the runtime scenario. However, due to the significant differences between simulators and real hardware, using only a virtual environment for development cannot guarantee software reliability. The HiL phase runs the software in the final physical deployment environment, thus ensuring interaction between the software and real hardware. However, using a HiL physical development environment often incurs higher setup costs and debugging risks.

[0004] Since both environments have their advantages and disadvantages, some work has attempted to combine the two to design a comprehensive development framework that lies between software-in-the-loop and hardware-in-the-loop, but these frameworks all suffer from the following problems:

[0005] (1) During the development process, CPS applications lack observability and operability at runtime.

[0006] (2) The development environment for hardware-in-the-loop is expensive to set up and cannot be quickly built into a running scenario.

[0007] (3) When combining the software-in-the-loop virtual environment and the hardware-in-the-loop physical environment, the virtual simulation scene set is limited by the current physical environment. Summary of the Invention

[0008] The technical problem to be solved by this invention is that it provides a cyber-physical integrated testbed architecture based on DJI RoboMaster. This is a tool to assist the software development process in a cyber-physical fusion system (CPS). It retains the advantages of high observability and operability, low operational risk and low construction cost in software-in-the-loop development, while using real physical hardware to avoid the problem of insufficient understanding of real software and hardware interaction behavior in software-in-the-loop development.

[0009] To achieve the above objectives, the technical solution provided by the present invention is as follows:

[0010] Based on the DJI RoboMaster cyber-physical testbed architecture, this system includes an application support framework and an environment synchronization execution framework, which provide execution support for the software support components and runtime environment of the cyber-physical fusion system (CPS), respectively. The application support framework accepts CPS applications uploaded by users, provides the software support components for application execution, and simulates execution. During simulation, the software support components categorize execution tasks into perception tasks and action execution tasks. Perception tasks are passed to the software-in-the-loop (MIL) execution framework for processing, while action execution tasks are passed to the hardware-in-the-loop (HIL) execution framework for processing. The environment synchronization execution framework maintains both the virtual environment for MIL execution and the real physical environment for HIL execution. It uses the execution scene in the virtual environment for MIL execution as input to the CPS application and uses the real hardware execution results of action execution tasks in the real physical environment for HIL execution as output to the CPS application. By synchronizing the scenes in the virtual and real physical environments, a comprehensive runtime environment between HIL and MIL is generated.

[0011] To optimize the above technical solution, the specific measures also include:

[0012] The application support framework includes a simulation operation module, a perception processing module, and an execution processing module. The simulation operation module is responsible for loading the CPS application and the application execution support environment SDK during initialization configuration, simulating the execution of the user-uploaded CPS application in conjunction with the application execution support environment SDK, and recording decisions locally. The perception processing module is responsible for encapsulating perception-type task call requests and sending them to the software-in-the-loop virtual environment. The execution processing module is responsible for sending action execution-type task call requests to the hardware-in-the-loop real physical environment and parsing and encapsulating the return value information of the real physical environment as the execution result.

[0013] The environment synchronization execution framework includes a software-in-the-loop (Software-in-the-loop) module, a hardware-in-the-loop (HIL) module, and a scene synchronization module. The Software-in-the-loop module maintains the virtual environment of the software and, upon receiving a perception-type task, uses the execution scene within the virtual environment as input to the CPS application. The Hardware-in-the-loop module maintains the real physical environment of the hardware and, upon receiving an action-execution task, uses the actual hardware execution result of the action-execution task within the real physical environment as output to the CPS application. The scene synchronization module calculates the actual hardware execution result in the real physical environment and synchronizes the scene generation between the virtual and real physical environments to create a comprehensive operating environment that bridges the gap between the hardware-in-the-loop and software-in-the-loop environments.

[0014] Furthermore, the user CPS application execution process includes the following specific steps:

[0015] (1) Initialization configuration: Users upload the CPS application to be executed through the application support framework;

[0016] (2) Execution environment synchronization: Synchronize the virtual environment and the real physical environment in the execution framework;

[0017] (3) Software-in-the-loop execution: The virtual environment performs perception-related tasks and obtains input information from CPS applications;

[0018] (4) Application execution simulation: Provide input information to the application support framework CPS application, parse and generate action execution class tasks as the output of CPS application;

[0019] (5) Hardware-in-the-loop execution: Execute CPS application output in a real physical environment.

[0020] Among them, step (1) initialization configuration and step (4) application execution simulation are executed by the application support framework; step (2) execution environment synchronization, (3) software-in-the-loop execution and (5) hardware-in-the-loop execution are executed by the environment synchronization execution framework.

[0021] Furthermore, the perception-type task is a perception-type API, and the action execution-type task is an action execution-type API.

[0022] Furthermore, the software support component is an extended SDK that performs secondary development on the DJI RoboMaster native SDK. It modifies and encapsulates the key APIs for CPS applications to interact with the external environment, separates the API call request and execution process, and enables seamless switching of API call requests of CPS applications between the virtual environment and the real physical environment.

[0023] As a preferred approach, the execution process of the application support framework is as follows:

[0024] Loading CPS application and execution environment: The application support framework initializes the execution environment and uses the Python module dynamic loading function to load the latest CPS application submitted by the user into the execution environment. The execution environment is the extended SDK after introducing the patch library for the DJI RoboMaster native SDK.

[0025] Execute CPS application: The application support framework executes the statements in the CPS application sequentially. If the statement is a perception API or an action execution API, it jumps to the extension SDK for execution and processing.

[0026] Execute the perception class API in the CPS application: Call the perception class API code in the extended SDK. This code decouples the data subscription of specific sensors and specific hardware in the physical world hardware components, and instead reads the results of virtual sensors from the virtual world. It provides a unified interface for interaction with the external environment, ensuring flexible virtual environment configuration. The unified interface for interaction with the external environment is the virtual sensor modeling interface configured in the virtual environment, which will be referred to as the "data reading interface" below.

[0027] Execute the action execution class API in the CPS application: Use the action execution class API code in the extended SDK. This code provides task splitting and execution functionality to handle API context switching or interruption triggering during hardware execution, ensuring a safe hardware interaction process.

[0028] Local records: Record context data of the interaction between the CPS application and the integrated runtime environment that combines virtual and real physical environments in the local database. For API calls in the extended SDK, record the call parameters, execution data and return values.

[0029] As a preferred solution, the execution process of the environment synchronization execution framework is as follows:

[0030] Maintain the virtual runtime environment of the software in the loop: build virtual scenes and Mechmaster vehicle models in a third-party virtual engine, and provide scene setting interfaces for scene updates, such as adding obstacles and moving vehicle models;

[0031] Maintaining the physical operating environment of hardware in the loop: Setting up an experimental site in the physical world and connecting it to DJI RoboMaster to obtain operable hardware components.

[0032] Processing perception APIs: When a perception request is received, the current scene data is read from the virtual scene through the data reading interface to generate CPS application input from the virtual environment;

[0033] Handling Action Execution APIs: When an action execution API is received, control the DJI RoboMaster to safely execute actions in the experimental field;

[0034] Numerical measurement of changes in the physical scene: Generate real-time positioning data of DJI RoboMaster in the physical scene. Here, a four-way sensing positioning algorithm is used to calculate the relative position of DJI RoboMaster in the experimental field in real time.

[0035] Location result synchronization and scene synthesis: The execution results of CPS applications in the hardware-in-the-loop scene are synchronized to the virtual scene and the comprehensive operating environment is displayed. The position change data of DJI RoboMaster during the operation of action execution APIs are mapped to the virtual scene to generate a comprehensive operating environment that includes hardware-in-the-loop action execution API results and software-in-the-loop perception API results, so as to show users the behavior of CPS applications in the current scene.

[0036] The Hardware-in-the-Loop (HIL) module employs a boundary detection mechanism combined with an attitude adjustment mechanism when processing action execution APIs. The HIL module consists of a data stream processing module, an instruction parsing module, an execution state control module, a boundary monitoring module, an attitude scheduling module, and a positioning computer module. The data stream processing module receives incoming action execution APIs and forwards the requests to the instruction parsing module. The instruction parsing module receives action execution API calls from the data stream processing module, the boundary monitoring module, and the attitude scheduling module, ultimately parsing them into low-level instructions for the DJI RoboMaster and interacting with it. The execution state control module controls the overall execution state of the HIL module to either an execution state or a scheduling state. The boundary monitoring module implements the boundary monitoring mechanism. The attitude scheduling module implements the attitude adjustment mechanism. The positioning computer module supports numerical measurement of changes in the physical scene.

[0037] The scene synchronization module consists of a scene synchronization data filter and a comprehensive scene update module. The scene synchronization data filter interacts with the hardware-in-the-loop execution module and is responsible for statistically analyzing the output results of CPS application execution in the real physical environment. The comprehensive scene update module interacts with the software-in-the-loop execution module and maps the execution results of CPS application actions to the virtual environment built by the virtual engine.

[0038] Compared with the prior art, the beneficial effects of the present invention are:

[0039] The interaction process between the DJI RoboMaster-based cyber-physical testbed architecture and CPS applications in this invention is like software-in-the-loop, acquiring environmental input in a virtual environment, and like hardware-in-the-loop, executing output on real hardware. It is an execution process between the software-in-the-loop and hardware-in-the-loop stages.

[0040] The cyber-physical integrated testbed architecture based on DJI RoboMaster of this invention is divided into an application support framework and an environment synchronization execution framework, which correspond to the software components and the operating environment in a cyber-physical fusion system (CPS), respectively.

[0041] The application support framework is a software framework that supports the uploading, simulated execution, and configuration of the execution environment for CPS applications. The framework accepts CPS applications uploaded by users, provides software support components (Software Development Kits, SDKs) for application execution, and simulates execution. During execution, the software support components break down the CPS application into tasks that interact with the environment. These tasks typically involve API calls within the SDK, namely awareness APIs and execution APIs. These awareness and execution APIs are then passed to the corresponding runtime environment for processing. Through these steps, users do not need to configure the software support environment for CPS application execution, reducing the cost of building the software components.

[0042] The Environment Synchronization Execution Framework is a software framework that connects the cyber-physical testbed architecture based on DJI RoboMaster with the runtime environment used for CPS application development. It maintains a virtual runtime environment built on the Unity virtual engine and a physical runtime environment built on the hardware of the DJI RoboMaster EP autonomous vehicle. In the virtual runtime environment, it generates inputs for the CPS application; in the physical runtime environment, it executes the CPS application's outputs and synchronizes the CPS application's execution data, showcasing it in conjunction with the runtime scenario. Through these steps, this module generates a comprehensive runtime environment that integrates virtual runtime scenarios and real hardware behavior. This ensures rapid scenario iteration and real hardware feedback while reducing the cost and risk of building physical scenarios and improving observability.

[0043] Compared with the prior art, the significant advantages of this invention are:

[0044] 1. The cyber-physical integrated testbed architecture based on DJI RoboMaster of this invention divides the execution process of the cyber-physical fusion system into an application support framework that maintains the execution of CPS applications and an environment synchronous execution framework that maintains the execution scenarios in the environment; wherein the application support framework has API-level behavior recording, and the environment synchronous execution framework monitors the hardware status in real time; therefore, the overall architecture has good observability and operability;

[0045] 2. The environment synchronous execution framework based on the DJI RoboMaster cyber-physical integrated testbed architecture of the present invention still provides the execution results of CPS applications on real hardware; at the same time, it reduces the cost of setting up scenes in the physical world by using the mode of setting up simple experimental sites and virtual scenes.

[0046] 3. Regarding the issue of virtual scene construction being limited by physical experimental sites, the cyber-physical integrated testbed architecture based on DJI RoboMaster in this invention supports the execution of action execution APIs with arbitrary parameters through an extended SDK in the application support framework. At the same time, the environment synchronization execution framework maintains a virtual-physical combined operating environment, builds an experimental site in the physical scene for execution, and displays the complete execution results in the virtual scene, thus solving the scene limitations of action execution APIs. Attached Figure Description

[0047] Figure 1 This is a structural diagram of the cyber-physical integrated testbed architecture based on DJI RoboMaster.

[0048] Figure 2 This is a structural diagram of the module design of the present invention.

[0049] Figure 3 It is the execution flowchart of the application support framework.

[0050] Figure 4 This is the general pattern of CPS application code.

[0051] Figure 5 It uses the Patch library in the native software development kit.

[0052] Figure 6 This refers to the execution process of the extended perception class API.

[0053] Figure 7 It is the execution process of the extended action execution class API.

[0054] Figure 8 This is the execution flowchart of the environment synchronous execution framework.

[0055] Figure 9 This is a software-in-the-loop module design structure diagram based on the DJI RoboMaster cyber-physical integrated testbed architecture.

[0056] Figure 10 This is an example of a software-in-the-loop virtual runtime environment.

[0057] Figure 11 This is a hardware-in-the-loop module design structure diagram based on the DJI RoboMaster cyber-physical integrated testbed architecture.

[0058] Figure 12 This is an example of a hardware-in-the-loop physical operating environment.

[0059] Figure 13 It is the execution process of action-type APIs that combines boundary detection and attitude adjustment mechanisms.

[0060] Figure 14 This is a diagram illustrating the data format of four-way sensing technology.

[0061] Figure 15 This is a structural diagram of the scene synchronization module in the cyber-physical integrated testbed architecture based on DJI RoboMaster. Detailed Implementation

[0062] The present invention will be further described in detail below through embodiments, but it should not be construed as limiting the scope of the subject matter of the present invention to the following embodiments. All technologies implemented based on the above content of the present invention fall within the scope of the present invention.

[0063] like Figure 1 As shown, this invention provides a cyber-physical integrated testbed architecture based on DJI RoboMaster, including an application support framework and an environment synchronous execution framework, which respectively provide execution support for the software support components and the operating environment in a cyber-physical fusion system (CPS). The application support framework is used to accept CPS applications uploaded by users, provide software support components for application execution, and simulate execution. During the simulation execution, the software support components divide the execution tasks into two categories: perception tasks and action execution tasks. Perception tasks are passed to the software-in-the-loop (MIL) operation in the environment synchronous execution framework for processing, and action execution tasks are passed to the hardware-in-the-loop (HIL) operation in the environment synchronous execution framework for processing. The environment synchronous execution framework is used to maintain the virtual environment for software-in-the-loop operation and the real physical environment for hardware-in-the-loop operation. The execution scene in the virtual environment for software-in-the-loop operation is used to execute perception tasks as input to the CPS application, and the real hardware execution results of action execution tasks in the real physical environment for hardware-in-the-loop operation are used as output to the CPS application. By synchronizing the scenes in the virtual environment and the real physical environment, a comprehensive operating environment between hardware-in-the-loop and software-in-the-loop is generated.

[0064] This invention provides support for both the software components and the execution environment in a cyber-physical system, combining the software-in-the-loop (Software-in-the-loop) and hardware-in-the-loop (HIL) environments during CPS application development. This preserves the real hardware-in-the-loop interaction process while leveraging the low cost and secure operation of the software-in-the-loop virtual environment, ensuring the security, observability, and operability of the software execution process during CPS application development.

[0065] The application support framework includes a simulation operation module, a perception processing module, and an execution processing module. The simulation operation module is responsible for loading the CPS application and the application execution support environment SDK during initialization configuration, simulating the execution of the user-uploaded CPS application in conjunction with the application execution support environment SDK, and recording decisions locally. The perception processing module is responsible for encapsulating perception-type task call requests and sending them to the software-in-the-loop virtual environment. The execution processing module is responsible for sending action execution-type task call requests to the hardware-in-the-loop real physical environment, parsing and encapsulating the return value information of the real physical environment as the execution result.

[0066] The environment synchronization execution framework includes a software-in-the-loop (Software-in-the-loop) execution module, a hardware-in-the-loop (HIL) execution module, and a scene synchronization module. The software-in-the-loop (Software-in-the-loop) execution module is responsible for maintaining the virtual environment of the software-in-the-loop and, after receiving a perception-type task, uses the execution scene in the virtual environment of the software-in-the-loop to execute the perception-type task as input to the CPS application. The hardware-in-the-loop (HIL) execution module is responsible for maintaining the real physical environment of the hardware-in-the-loop and, after receiving an action execution task, uses the real hardware execution result of the action execution task in the real physical environment as output to the CPS application. The scene synchronization module is responsible for calculating the real hardware execution result in the real physical environment and synchronizing the scene generation between the virtual environment and the real physical environment to form a comprehensive operating environment between the hardware-in-the-loop and software-in-the-loop.

[0067] In this embodiment, perception-type tasks are perception-type APIs, and action execution-type tasks are action execution-type APIs.

[0068] In this embodiment, the software support component is an extended SDK that performs secondary development on the DJI RoboMaster native SDK. It modifies and encapsulates the key APIs for CPS applications to interact with the external environment, separates the API call request and execution process, and enables seamless switching of API call requests of CPS applications between the virtual environment and the real physical environment.

[0069] Specifically, within the application support framework, the simulation execution module is responsible for loading the user's CPS application, configuring the application execution environment, and simulating execution. It configures the extended SDK environment, accepts the CPS application and executes it statement by statement, recording the execution results. If the currently executed statement is a perception-based API or an action execution API, it jumps to the corresponding module for processing. The perception processing module is responsible for executing perception-based APIs in the CPS application, generating requests for these APIs, and processing the execution results returned by the environment. The execution processing module is responsible for executing action execution APIs in the CPS application, sending execution commands for these APIs, and waiting for the execution results from the environment.

[0070] Specifically, in the environment synchronization execution framework, the software-in-the-loop (Software-in-the-loop) module is responsible for maintaining the virtual runtime environment of the software in the loop and processing perception-related APIs, while the hardware-in-the-loop (HIL) module is responsible for maintaining the real physical environment of the hardware in the loop and processing action execution APIs. The scene synchronization module is responsible for calculating the action execution results and generating the comprehensive runtime environment.

[0071] The user CPS application execution process includes the following specific steps:

[0072] (1) Initialization configuration: Users upload the CPS application to be executed through the application support framework;

[0073] (2) Execution environment synchronization: Synchronize the virtual environment and the real physical environment in the execution framework;

[0074] (3) Software-in-the-loop execution: The virtual environment performs perception-related tasks and obtains input information from CPS applications;

[0075] (4) Application execution simulation: Provide input information to the application support framework CPS application, parse and generate action execution class tasks as the output of CPS application;

[0076] (5) Hardware-in-the-loop execution: Execute CPS application output in a real physical environment.

[0077] Among them, step (1) initialization configuration and step (4) application execution simulation are executed by the application support framework; step (2) execution environment synchronization, (3) software-in-the-loop execution and (5) hardware-in-the-loop execution are executed by the environment synchronization execution framework.

[0078] As a preferred approach, the execution process of the application support framework is as follows:

[0079] (1a) Loading CPS application and execution environment: The application support framework initializes the execution environment and uses the Python module dynamic loading function to load the latest CPS application submitted by the user into the execution environment. The execution environment is the extended SDK after introducing the patch library for the DJI RoboMaster native SDK.

[0080] (4a) Execute CPS application: The application support framework executes the statements in the CPS application in sequence. If the statement is a perception API or an action execution API, it jumps to the extension SDK for execution and processing.

[0081] (4b) Execute the perception class API in the CPS application: Call the perception class API code in the extended SDK. This code reads the results of virtual sensors from the virtual world and provides a unified interface for interacting with the virtual environment, namely the data reading interface, to ensure flexible virtual environment configuration.

[0082] (4c) Execute the action execution class API in the CPS application: Use the action execution class API code in the extended SDK, which provides task splitting and execution functionality to deal with API context switching or interruption during hardware execution and ensure a safe hardware interaction process.

[0083] (1b) Local records: Record the context data of the interaction between the CPS application and the integrated operating environment that combines the virtual environment and the real physical environment in the local database. For API calls in the extended SDK, record the call parameters, execution data and return values.

[0084] The simulation execution module is responsible for (1a) loading the user's CPS application and execution environment, (4a) executing the CPS application, and (1b) local recording function. This module executes the program file uploaded by the user in conjunction with the extended SDK, and records the decision locally. The simulation execution module quickly configures the extended SDK execution environment by introducing a patch library and modifying function pointers without modifying the official SDK source code.

[0085] The simulation module is responsible for (1b) the local recording function. Since the parameters, callback process and return value are consistent between the extended SDK code and the official SDK code, the local recording function of DJI RoboMaster is reused. The data recorded by the simulation module is in the same format as DJI's local recording function, but the recorded data only includes the perception and execution results related to the CPS application.

[0086] Specifically, the simulation running module is responsible for (4a) loading the user's CPS application and execution environment. Within the extended SDK code, the extended perception class API code and the extended action class API code will jump to other modules of the application support framework for execution.

[0087] The perception processing module is responsible for (4b) executing the perception-type API in the CPS application, sending the API call request to the software-in-the-loop virtual runtime environment, and waiting for the execution result; the perception processing module adds external data synchronization encoding by rewriting the pub-sub mechanism of data subscription, and accepts perception data from the outside.

[0088] The execution processing module is responsible for (4c) executing the action execution class APIs in the CPS application, sending API call requests to the hardware-in-the-loop physical runtime environment, and processing execution records. The execution processing module adds context data and platform timers to the action execution class APIs through an action execution segmentation mechanism, and ensures the complete execution of the action execution class APIs by supporting context switching.

[0089] refer to Figure 4The user CPS application loaded by the simulation operation module can be written as the following decision program, which has the general pattern of "the CPS application periodically checks the status of the vehicle (Status = Call perception API) and executes the specified action (Call action execution API) when the vehicle meets the corresponding conditions".

[0090] refer to Figure 5 The application execution environment configured for the simulation module is as follows. DJI RoboMaster provides a Software Development Kit (SDK) to support the interaction between the application and the environment; this SDK is referred to as the native SDK. The simulation module introduces a patch library to modify the function pointers of the perception API and the action execution API. When the CPS application encounters perception API and action execution API statements, the extended SDK code is called in the patch library. This module separates the API call request and execution process, enabling seamless switching of CPS application API call requests between the virtual and real environments.

[0091] refer to Figure 6 After expansion, the specific process of the Perception API execution is described as follows:

[0092] The perception processing module accepts extended perception class API requests from the simulation execution module and performs periodic callbacks based on the user perception frequency set in the perception class API parameters.

[0093] If this is the first subscription to the perception API, initialize the perception processing module: send a perception request, subscribe to the maximum perception frequency from the software-in-the-loop module, and receive response data; here, the sent perception request and the received response data are called the perception data stream.

[0094] The execution processing module of the application support framework is responsible for (4c) executing the action execution class APIs in the CPS application, sending API call requests to the hardware-in-the-loop physical runtime environment, and processing execution records. The execution processing module adds context data and platform timers to the action execution class APIs through an action execution segmentation mechanism, and ensures the complete execution of the action execution class APIs by supporting context switching.

[0095] refer to Figure 7 After expansion, the specific process of executing the action execution class API is divided into several parts, as described below:

[0096] The execution processing module accepts action execution class API calls from the simulation execution module, records them as complete actions, sets the action context, and monitors the execution progress.

[0097] Generate sub-actions (which are also action execution APIs), send them to the hardware-in-the-loop module, invoke the physical hardware to run them, and simultaneously receive the sub-action execution results from the hardware-in-the-loop module to update the execution progress of the complete action; here, the sent sub-actions and the received sub-action execution results are called action execution data streams.

[0098] If the current execution progress is incomplete, the execution processing module continues to generate new sub-actions and send them to the hardware-in-the-loop module. If the current execution progress is complete, the execution processing module returns the final execution result to the simulation execution module.

[0099] As a preferred solution, the execution process of the environment synchronization execution framework is as follows:

[0100] (2a) Maintain the virtual runtime environment of the software in the loop: build virtual scenes and Mechmaster vehicle models in a third-party virtual engine, and provide scene setting interfaces for scene updates;

[0101] (2b) Maintain the physical operating environment of hardware in the loop: Set up an experimental site in the physical world and connect it to DJI RoboMaster to obtain operable hardware components.

[0102] (3a) Processing perception APIs: When a perception request is received, the current scene data is read in the virtual scene through the data reading interface to generate CPS application input from the virtual environment;

[0103] (5a) Handling Action Execution API: When receiving an action execution API, control the DJI RoboMaster to safely execute actions on the experimental site;

[0104] (5b) Numerical measurement of changes in the physical scene: Generate real-time positioning data of DJI RoboMaster in the physical scene. Here, a four-way sensor positioning algorithm is used to calculate the relative position of DJI RoboMaster on the experimental site in real time.

[0105] (2c) Location result synchronization and scene synthesis: The execution results of CPS applications in the hardware-in-the-loop scene are synchronized to the virtual scene and the comprehensive operating environment is displayed. The position change data of DJI RoboMaster during the operation of the action execution API is mapped to the virtual scene to generate a comprehensive operating environment that includes the action execution API results of hardware-in-the-loop and the perception API results of software-in-the-loop, so as to show the user the behavior of CPS applications in the current scene.

[0106] The software-in-the-loop runtime module in the environment synchronization execution framework is responsible for (2a) maintaining the virtual runtime environment of software-in-the-loop and (3a) processing perception-based APIs. Figure 9 This is a design structure diagram of the software-in-the-loop module.

[0107] Furthermore, the software-in-the-loop module is responsible for (2a) maintaining the software-in-the-loop virtual runtime environment, which includes two parts: virtual environment setup and virtual environment maintenance.

[0108] The virtual runtime environment will be built on the third-party virtual engine Unity, and the setup content is as follows: Figure 10 As shown, the virtual running scene is a virtual field containing obstacles (left), and the virtual running hardware is modeled by the MechMaster simulator (right). Here, the virtual running scene is not fixed to a specific map and can be quickly switched and reset.

[0109] The maintenance of the virtual runtime environment is achieved by defining the coupling interface with third-party platforms. Figure 9 This interface, implemented by DJI RoboMaster, ensures that any virtual engine platform can connect to the cyber-physical testbed architecture (hereinafter referred to as the testbed) after simple configuration and obtain the execution results of the CPS software in the testbed, reducing the complexity of building and updating the virtual environment.

[0110] Taking the integration of the Unity 3D virtual engine as an example, the unified configuration interface was written in C# script on the Unity side and in plaintext format for socket communication on the testbed side. The interface includes two types: scene setting interface and data reading interface.

[0111] The scene setting interface accepts scene setting data. Taking the pathfinding software development scene in the above-mentioned venue as an example, the scene setting data is maze map data, including the number of obstacles and the real-time vehicle position. Using this interface, it is possible to quickly switch running scenes at low cost, while displaying the execution environment currently faced by the CPS application.

[0112] The data reading interface generates environmental perception data. Taking the perception data from the infrared ranging sensor in front as an example, the environmental perception data is a floating-point variable that stores the distance from the vehicle model to the obstacle model in front in the virtual scene. The perception interface predefines a large number of sensor perception data formats, including data from the camera (stream data), ranging sensor (floating-point number), and angle sensor (integer) used by DJI RoboMaster. Using this interface, the scene in the virtual engine can be converted into the input of the CPS application in real time.

[0113] Furthermore, the software-in-the-loop module is responsible for (3a) processing perception APIs; see Figure 9 The software-in-the-loop module processes the received perception-related APIs as follows:

[0114] • The software-in-the-loop module receives a sensing API subscription request from the sensing processing module and reads the maximum sensing frequency set in the subscription parameters;

[0115] ● Periodically call the data reading interface coupled with the third-party virtual engine at the maximum perception frequency to obtain environmental perception data in the virtual environment as environmental output;

[0116] ●Based on the obtained environmental output information, adjust the data (e.g., add ranging noise) according to the uncertainty model of the current environment to generate input information for CPS applications;

[0117] The hardware-in-the-loop module in the environment synchronization execution framework is responsible for (2b) maintaining the virtual runtime environment of the software-in-the-loop, (5a) processing action execution APIs, and (5b) numerically measuring changes in the physical scene. Figure 11 This is a design structure diagram of the hardware-in-the-loop module.

[0118] Furthermore, the hardware-in-the-loop module is responsible for (2b) maintaining the physical operating environment of the hardware-in-the-loop, which includes two parts: physical environment setup and physical environment maintenance.

[0119] The environment synchronization execution framework includes a software-in-the-loop (Software-in-the-loop) module, a hardware-in-the-loop (HIL) module, and a scene synchronization module. The software-in-the-loop module is responsible for (2a) maintaining the virtual runtime environment of the software in the loop and (3a) processing perception APIs. This module models the execution scene (including the initial position of the DJI RoboMaster vehicle and scene obstacles) on a third-party virtual engine, processes perception API requests, reads perception data from the environment, and generates inputs for the CPS application. The software-in-the-loop module ensures that any third-party virtual engine platform can be integrated into the platform after simple configuration by defining a coupling interface with the third-party platform, thus reducing the cost of scene construction.

[0120] Furthermore, the hardware-in-the-loop module is responsible for (2b) maintaining the physical operating environment of the hardware-in-the-loop and (5a) processing action execution APIs. This module is connected to a DJI RoboMaster vehicle and controls the chassis hardware to process action execution API requests in a square, open experimental field and output the execution results to the environment. The hardware-in-the-loop module reduces the probability of physical collisions of APIs through an attitude adjustment mechanism and adds operating environment monitoring during execution through a boundary monitoring mechanism to terminate interactive behaviors that may cause physical collisions in advance, thereby ensuring the safety of the hardware-in-the-loop interaction process.

[0121] Furthermore, the scene synchronization module is responsible for (5b) quantifying the changes in the physical scene and (2c) synchronizing the positioning results and scene synthesis. This module locates the DJI RoboMaster robot car (hereinafter referred to as the robot car) in real time, defines the changes in the physical scene as the changes in the robot car's position during operation, measures and maps them into the virtual scene, thereby displaying a comprehensive operating environment that includes both hardware-in-the-loop action execution API results and software-in-the-loop perception API results, showing the user the behavior of the CPS application in the current scene; the scene synchronization module calculates the position of the robot car in the physical field in real time through four-way sensing positioning technology, combines the execution flow data of the environment synchronization execution framework to restore the execution results of the current action execution API, synchronizes it to the virtual environment, and shows the user the behavior of the CPS application in the current scene, thereby ensuring observability during operation.

[0122] The fundamental difference between this architecture and existing CPS application testbeds lies in the fact that, compared to pure software-in-the-loop testbeds, the CPS applications in this architecture interact with real hardware, avoiding the insufficient understanding of hardware behavior caused by simulation or emulation methods in pure software-in-the-loop environments. Compared to pure hardware-in-the-loop testbeds, the CPS applications in this architecture use virtual scenarios as the application input source, avoiding the huge overhead and equipment damage risk of building real scenarios when running hardware-in-the-loop.

[0123] like Figure 12 As shown, the construction of the physical operating environment relies on manual labor. The structure of the environment is as follows: the operating scene is a simple square experimental field with partitions as boundaries, and the operating hardware is a car equipped with a four-way infrared ranging sensor. The maintenance of the physical operating environment requires ensuring that the local area network connection between the car and the computer is stable, and that the ranging information of the infrared sensor can be transmitted to the computer.

[0124] Furthermore, the hardware-in-the-loop module is responsible for (5a) handling action execution APIs;

[0125] The key point of the hardware-in-the-loop module (5a) in handling action execution APIs is to ensure the safety of the vehicle's hardware interaction behavior. The existing hardware interaction behavior is unsafe. Due to the lack of protection mechanisms, the vehicle hardware may collide with the boundary of the field during the execution of the action execution API, causing damage to the vehicle hardware. Boundary detection mechanism and attitude adjustment mechanism can improve the safety of interaction behavior.

[0126] The boundary detection mechanism is a pessimistic safety assurance mechanism. This mechanism identifies areas near the edge of the experimental site as dangerous areas. At any time, when the API-operated vehicle touches this area, the module will stop the vehicle's movement, thus proactively preventing a collision, even if the vehicle can stop in time near the boundary.

[0127] The attitude adjustment mechanism is an optimistic safety guarantee mechanism. This mechanism delineates a safe zone at the center of the experimental site. APIs that start executing from within this zone are highly likely to avoid collisions, or will definitely not collide for a period of time after the API execution begins.

[0128] Figure 13 This demonstrates the execution process of an action execution class API that combines boundary detection and attitude adjustment mechanisms:

[0129] Suppose that when the car is at point P, the hardware-in-the-loop module receives an API call representing a forward movement.

[0130] According to the attitude adjustment mechanism, when the action begins, the car should be in the safe area (grid area). Therefore, the car is scheduled to move from point P to point Q to start the action. At this time, the car starts to move along the QR direction.

[0131] According to the boundary monitoring mechanism, the operation will be suspended when the vehicle reaches the danger zone (grid area).

[0132] The hardware-in-the-loop module will report the interruption reason and execution progress to the API initiator;

[0133] At this point, the execution processing module in the application support framework will realize that this is an incomplete API execution. After generating an action execution class API representing the remaining progress, it will submit it to the hardware-in-the-loop module for execution again.

[0134] refer to Figure 11 The hardware-in-the-loop module consists of a data stream processing module, an instruction parsing module, an execution state control module, a boundary monitoring module, an attitude scheduling module, and a positioning computer module.

[0135] The data stream processing module accepts the incoming action execution class API and forwards the request to the instruction parsing module;

[0136] The instruction parsing module accepts action execution API calls from the data stream processing module, boundary monitoring module, and attitude scheduling module, and finally parses them into low-level instructions for DJI RoboMaster, and interacts with DJI RoboMaster. If the API call request comes from the data processing module, the parsing module will return the execution status and execution progress after the interaction is completed.

[0137] The execution state control module controls the overall execution state of the hardware-in-the-loop module to either the execution state or the scheduling state. The execution state indicates that the module is executing API calls from the CPS application, at which point the boundary monitoring mechanism performs execution monitoring. The scheduling state indicates that the module is executing its own action APIs, at which point the attitude adjustment mechanism is used to adjust the vehicle's position. The execution state module receives the vehicle's location data from the physical environment to determine whether the vehicle is currently in a safe or dangerous area and performs corresponding module transitions.

[0138] The boundary monitoring module implements a boundary monitoring mechanism. When the module is in the execution state, if it detects that the car is in a dangerous area, the module will send a stop command to stop the current hardware device's operation and switch to the scheduling state.

[0139] The attitude scheduling module implements an attitude adjustment mechanism. When the module is in the scheduling state, if the vehicle is not in the safe area, the module will send a sequence of scheduling instructions until the vehicle reaches the safe area and then switch to the execution state.

[0140] Furthermore, the hardware-in-the-loop module is responsible for (5b) numerically measuring changes in the physical scene;

[0141] This function is supported by the positioning calculator module; numerical measurement of changes in the physical scene requires real-time and rapid calculation of the vehicle's position in the physical field; four-way sensing positioning technology is used for position measurement, which sets up a system of equations for solving the target (vehicle positioning), optimizes the solution process to combat the problem of solution failure caused by noisy input data, and selects the optimal solution from multiple candidate solutions using historical positioning data and current action tendencies.

[0142] Figure 14 Specifically, the data format and equation-solving process for four-way sensing and positioning technology are defined:

[0143] Figure 14 The left side illustrates the input of this technology. To use this technology, four binocular infrared ranging sensors need to be installed on the car, and the ranging values ​​of the four sensors are the input.

[0144] Figure 14 The middle section describes the output of this technology. The positioning technology calculates the vehicle's position in the physical field in real time. The output position information consists of the vehicle's coordinates (x0, y0) and orientation θ. The specific definition of the triple is as follows: Given that the experimental field is a square field with a side length of E meters, a coordinate system is established with the center of the square field as the origin and the x-axis and y-axis parallel to the boundary. x0 and y0 are the horizontal and vertical coordinates of the vehicle's center point in this coordinate system, and θ is the angle between the vehicle's forward orientation and the positive x-axis.

[0145] Figure 15The right side illustrates the process of using a system of simultaneous equations with input and output data in four-way sensing positioning technology; in the coordinate system, the four boundaries of the square can be regarded as four line segments, and the ranging process can be regarded as drawing two straight lines parallel or perpendicular to the direction θ of the vehicle through the vehicle's coordinates (x0, y0):

[0146]

[0147] Since the car is inside the square-shaped area, the two straight lines passing through coordinates (x0, y0) and the four boundary line segments will inevitably generate four intersection points, and the distance values ​​Dist in the four directions of front, back, left, and right will be... dir Equal to the coordinates of the intersection point (x) in that direction dir y dir The Euclidean distance from the centroid of the vehicle (x0, y0) to the coordinates (x0, y0) is:

[0148] (x dir -x0) 2 +(y dir -y0) 2 =Dist dir

[0149] If the boundary line segment containing each intersection point is known, the coordinates of the intersection points (x, y) dir y dir The equations can be further represented by (x0, y0) and θ, thus enabling the simultaneous solving of the system of equations.

[0150] The specific process of numerically measuring scene changes in the location calculator module (5b) within the hardware-in-the-loop module is as follows:

[0151] • Solve the system of equations G, read in the ranging values ​​from the four sensors on the vehicle, read in the environmental side lengths, enumerate the intersection points for coordinates (x0, y0) and orientation θ, and then solve the system of equations G. i Where i refers to the i-th intersection point; the system of equations has four sub-equations, using G... i，1 ~G i，4 serial number;

[0152] Solve in groups, grouping the system of equations G i Split into {G′ i，k |G′ i，k =G i -G i，k The four sub-equations {k = 1, 2, 3, 4} are solved by reducing the constraints. The sub-equations can still be solved successfully even when the four ranging input values ​​are noisy. After removing outliers from the solutions of the four sub-equations, they are combined into multiple feasible solutions S.

[0153] • Optimal solution determination: If at least one feasible solution exists, the best candidate location is selected by weighted sorting; otherwise, proceed to the no-solution processing step. This step first retrieves historical location information (x) from the historical location database. h y h θ h Get the current action tendency (x) from HiL Wrapper δ y δ θ δ ), generating ideal positioning (x α y α θ α )for:

[0154] (x α y α )=(x h +x δ y h +yδ)

[0155] θ α =θ h +θ δ

[0156] Then for each candidate position (x) i y i θ i Calculate the deviation from the ideal positioning, where v and u are two positive numbers representing the threshold values ​​for distance and angle deviations:

[0157] S i =v*dist((x a y α ), (x i y i ))+u*|θ α -θ i |

[0158] The smaller the deviation value Si, the closer the candidate position is to the ideal position, and the higher the score of the solution;

[0159] Finally, the solution with the highest score is updated to the historical location database as the optimal solution, and the scene change result is the difference between the results of the two adjacent timestamp calculations.

[0160] • If no solution is found in the current solution process, repeat the complete solution process k times (k is the number of retries set by the user) to try to see if the solution can be found successfully under different input noise conditions; if no solution is found in k times, reset the real physical environment, and backtrack in the virtual scene based on historical positioning data to return to the last successfully calculated positioning node.

[0161] By solving in groups and handling cases with no solution, the positioning calculator module generates a series of positioning records of the physical car relative to the square experimental field on the timeline, which serve as the observation results of the scene synchronization module on the hardware-in-the-loop execution environment.

[0162] The scene synchronization module of the environment synchronization execution framework is responsible for (2c) synchronizing positioning results and scene synthesis; Figure 15 This is a design architecture diagram for the scene synchronization module.

[0163] The scene synchronization module is further responsible for (2c) synchronizing the positioning results and compositing the scene; this includes two parts: calculating the execution output of the CPS application on the real physical environment and mapping the execution output into the virtual scene.

[0164] The scene synchronization module consists of a scene synchronization data filter and a comprehensive scene update module.

[0165] The scene synchronization data filter interacts with the hardware-in-the-loop execution module to collect statistics on the output results of CPS application execution in the real physical environment. The input data are the vehicle positioning results and the module execution status. The filtering rules only retain the position change results during the execution state and discard the position changes during the scheduling state. This separates the scene changes caused by CPS application action execution API from the scene changes caused by platform scheduling.

[0166] The integrated scene update module and the software-in-the-loop execution module interact to map the execution results of CPS applications to the virtual environment built by the virtual engine, thus displaying the complete CPS application behavior to the user.

[0167] The scene synchronization module synchronizes the execution results of a certain action API through the following steps:

[0168] Retain the positioning change information when the hardware-in-the-loop module is in the execution state, including position change information (Δx). p Δy p and angle variation information Δθ p Generate location update data for CPS applications:

[0169] (Δx p Δy p )=(x p，i+1 -x p，i y p，i+1 -y p，i )

[0170] Δθ p =θ p，i+1 -θ p，i

[0171] Where (x) p，i , y p，iθ p，i )) represents the i-th positioning result of the positioning calculator for the physical car p.

[0172] In the virtual scene execution module, the latest location information P of the virtual vehicle s is read from a third-party engine using a uniformly configured data reading interface. s，k Including location information (x s，k y s，k ) and current vehicle angle information θ s，k ;

[0173] Position change information (Δx) p Δy p The car's orientation is corrected by angle. The change information vector (Δx) is then used to correct the angle. p Δy p Rotate counterclockwise around the origin by θ p，i Then rotate clockwise by θ s，i ; Generate relative virtual location information P s，i Position change (Δx) s Δy s );

[0174] Calculate the vehicle localization result (x) for the (k+1)th iteration in the virtual environment. s，k+1 y s，k+1 θ s，k+1 The unified scene settings interface is used to update the virtual scene.

[0175] (x s，k+l y s，k+1 )=(x s，i +Δx s y s，i +Δy s )

[0176] θ s，k+1 =θ s，k +Δθ s

[0177] The execution flow of this invention is further illustrated using an exemplary CPS application. This application is a car maze navigation program. The application reads distance measurements from sensors at the front and left sides of the car, makes movement decisions, and controls the chassis operation. The scene is a maze composed of horizontal and vertical wooden plank obstacles. In the virtual scene, the maze is modeled using Unity, and the car model is initially placed in the lower right corner of the maze. In the physical scene, no maze model is created, and the experimental site is a square bounded area with sides of 2.4m, inside which a DJI RoboMaster car is placed. After the CPS application starts running…

[0178] For CPS applications such as the front sensor reading API, the testbed reads the distance from the vehicle model to the front obstacle model through a data reading interface configured uniformly with Unity, and adds uncertainty noise; at this time, the CPS application obtains scene input from the virtual scene. Compared with the construction of a physical maze, the virtual scene has a lower scene configuration cost.

[0179] For API actions such as chassis forward movement in CPS applications, the testbed first controls the vehicle to move forward in the experimental field, calculates the forward distance, and moves the vehicle's position in the virtual scene. Due to the uncertainty of execution in the physical world, the actual execution result of the vehicle deviates from the user's expectations. For example, assuming that the developer did not consider forward inertia when writing the CPS application (i.e., the actual forward distance is usually greater than the set forward distance parameter), this movement will cause the vehicle to collide with virtual obstacles in the virtual scene and report an error to the developer, ensuring the display of real software interaction behavior. In the above process, the physical vehicle and the experimental field do not collide, making the platform safer to run.

[0180] In the testbed of this invention, the interaction process with CPS applications is similar to the software-in-the-loop method, acquiring application input in a virtual environment, while simultaneously executing application output on real hardware, similar to the hardware-in-the-loop method. By combining the software-in-the-loop and hardware-in-the-loop runtime environments in the CPS application development process, the real software-hardware interaction process of hardware-in-the-loop is preserved, while the low cost and secure operation of the software-in-the-loop virtual environment are combined to ensure the security, observability, and operability of the software execution process during CPS development.

[0181] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent substitutions, and improvements made by those skilled in the art to the above embodiments without departing from the scope of the technical solution of the present invention, based on the technical essence of the present invention, shall still fall within the protection scope of the technical solution of the present invention.

Claims

1. A cyber-physical integrated testbed architecture based on MechMaster, characterized in that: The system includes an application support framework and an environment synchronization execution framework, which provide execution support for the software support components and the runtime environment in the cyber-physical fusion system, respectively. The application support framework accepts CPS applications uploaded by users, provides software support components for application execution, and simulates execution. During the simulation execution, the software support components divide the execution tasks into two categories: perception tasks and action execution tasks. Perception tasks are passed to the software-in-the-loop (MIL) operation in the environment synchronization execution framework for processing, while action execution tasks are passed to the hardware-in-the-loop (HIL) operation in the environment synchronization execution framework for processing. The environment synchronization execution framework maintains the virtual environment for software-in-the-loop operation and the real physical environment for hardware-in-the-loop operation. It uses the execution scene in the virtual environment for software-in-the-loop operation to execute perception tasks as input to the CPS application, and uses the real hardware execution results of action execution tasks in the real physical environment for hardware-in-the-loop operation as output to the CPS application. By synchronizing the scenes in the virtual environment and the real physical environment, a comprehensive runtime environment between hardware-in-the-loop and software-in-the-loop is generated.

2. The cyber-physical integrated testbed architecture based on MechMaster as described in claim 1, characterized in that: The application support framework includes a simulation operation module, a perception processing module, and an execution processing module. The simulation operation module is responsible for loading the CPS application and the application execution support environment SDK during initialization configuration, simulating the execution of the user-uploaded CPS application in conjunction with the application execution support environment SDK, and recording decisions locally. The perception processing module is responsible for encapsulating perception-type task call requests and sending them to the software-in-the-loop virtual environment. The execution processing module is responsible for sending action execution-type task call requests to the hardware-in-the-loop real physical environment and parsing and encapsulating the return value information of the real physical environment as the execution result.

3. The cyber-physical integrated testbed architecture based on MechMaster according to claim 2, characterized in that: The environment synchronization execution framework includes a software-in-the-loop (Software-in-the-loop) module, a hardware-in-the-loop (HIL) module, and a scene synchronization module. The Software-in-the-loop module maintains the virtual environment of the software and, upon receiving a perception-type task, uses the execution scene within the virtual environment as input to the CPS application. The Hardware-in-the-loop module maintains the real physical environment of the hardware and, upon receiving an action-execution task, uses the actual hardware execution result of the action-execution task within the real physical environment as output to the CPS application. The scene synchronization module calculates the actual hardware execution result in the real physical environment and synchronizes the scene generation between the virtual and real physical environments to create a comprehensive operating environment that bridges the gap between the hardware-in-the-loop and software-in-the-loop environments.

4. The cyber-physical integrated testbed architecture based on MechMaster as described in claim 3, characterized in that: The user CPS application execution process includes the following specific steps: (1) Initialization configuration: Users upload the CPS application to be executed through the application support framework; (2) Execution environment synchronization: Synchronize the virtual environment and the real physical environment in the execution framework; (3) Software-in-the-loop execution: The virtual environment performs perception-related tasks and obtains input information from CPS applications; (4) Application execution simulation: Provide input information to the application support framework CPS application, parse and generate action execution class tasks as the output of CPS application; (5) Hardware-in-the-loop execution: Executing CPS application output in a real physical environment; Among them, step (1) initialization configuration and step (4) application execution simulation are executed by the application support framework; step (2) execution environment synchronization, (3) software-in-the-loop execution and (5) hardware-in-the-loop execution are executed by the environment synchronization execution framework.

5. The cyber-physical integrated testbed architecture based on MechMaster according to claim 4, characterized in that: The perception-type tasks are perception-type APIs, and the action execution-type tasks are action execution-type APIs.

6. The cyber-physical integrated testbed architecture based on MechMaster according to claim 5, characterized in that: The software support component is an extended SDK that performs secondary development on the original SDK of Mech Master. It modifies and encapsulates the key APIs for the interaction between the CPS application and the external environment, separates the API call request and execution process, and realizes the seamless switching of API call requests of the CPS application between the virtual environment and the real physical environment.

7. The cyber-physical integrated testbed architecture based on MechMaster according to claim 6, characterized in that: The execution process of the application support framework is as follows: Loading CPS Application and Execution Environment: The application support framework initializes the execution environment and uses the Python module dynamic loading function to load the latest CPS application submitted by the user into the execution environment. The execution environment is the extended SDK after introducing the patch library for the native SDK of Mech Master. Execute CPS application: The application support framework executes the statements in the CPS application sequentially. If the statement is a perception API or an action execution API, it jumps to the extension SDK for execution and processing. Execute the perception class API in the CPS application: Call the perception class API code in the extended SDK. This code reads virtual sensor results from the virtual world, provides a unified interface for interacting with the virtual environment, and ensures flexible virtual environment configuration; the interface mentioned is a data reading interface. Execute the action execution class API in the CPS application: Use the action execution class API code in the extended SDK. This code provides task splitting and execution functionality to handle API context switching or interruption triggering during hardware execution, ensuring a safe hardware interaction process. Local records: Record context data of the interaction between the CPS application and the integrated runtime environment that combines virtual and real physical environments in the local database. For API calls in the extended SDK, record the call parameters, execution data and return values.

8. The cyber-physical integrated testbed architecture based on MechMaster according to claim 7, characterized in that: The execution process of the environment synchronization execution framework is as follows: Maintain the software-in-the-loop virtual runtime environment: Build virtual scenes and Mechmaster vehicle models in a third-party virtual engine, and provide scene setting interfaces for scene updates; Maintain the physical operating environment of hardware in the loop: Set up experimental sites in the physical world and connect them to the Mech Master to obtain operable hardware components; Processing perception APIs: When a perception request is received, the current scene data is read from the virtual scene through the data reading interface to generate CPS application input from the virtual environment; Handling Action Execution APIs: When an action execution API is received, control the Mech Master to safely execute actions on the experimental field; Numerical measurement of changes in the physical scene: real-time positioning data of the RoboMaster in the physical scene is generated. Here, a four-way sensing positioning algorithm is used to calculate the relative position of the RoboMaster in the experimental field in real time. Location result synchronization and scene synthesis: The execution results of CPS applications in the hardware-in-the-loop scene are synchronized to the virtual scene and the comprehensive operating environment is displayed. The position change data of the robot master during the operation of the action execution API is mapped to the virtual scene, generating a comprehensive operating environment that includes the hardware-in-the-loop action execution API results and the software-in-the-loop perception API results, so as to show the user the behavior of the CPS application in the current scene.

9. The cyber-physical integrated testbed architecture based on MechMaster according to claim 3, characterized in that: The Hardware-in-the-Loop (HIL) module employs a boundary detection mechanism combined with an attitude adjustment mechanism when processing action execution APIs. The HIL module consists of a data stream processing module, an instruction parsing module, an execution state control module, a boundary monitoring module, an attitude scheduling module, and a positioning computer module. The data stream processing module receives incoming action execution APIs and forwards the requests to the instruction parsing module. The instruction parsing module receives action execution API calls from the data stream processing module, the boundary monitoring module, and the attitude scheduling module, ultimately parsing them into low-level instructions for the robot master and interacting with the robot master. The execution state control module controls the overall execution state of the HIL module to either the execution state or the scheduling state. The boundary monitoring module implements the boundary monitoring mechanism. The attitude scheduling module implements the attitude adjustment mechanism. The positioning computer module supports numerical measurement of changes in the physical scene.

10. The cyber-physical integrated testbed architecture based on MechMaster according to claim 3, characterized in that: The scene synchronization module consists of a scene synchronization data filter and a comprehensive scene update module. The scene synchronization data filter interacts with the hardware-in-the-loop execution module and is responsible for statistically analyzing the output results of CPS application execution in the real physical environment. The integrated scene update module and the software-in-the-loop execution module interact to map the execution results of the CPS application's actions onto the virtual environment built by the virtual engine.

Images