Adjusting and testing device and method for automatic driving application and computer program product

Abstract

The invention relates to the technical field of automatic driving application testing, and provides an automatic driving application commissioning and testing device which comprises a commissioning and testing plug-in, a commissioning and testing development kit, a commissioning and testing container and a visual interaction interface. Wherein the commissioning and testing plug-in is used for executing a commissioning and testing task for the automatic driving application; the commissioning and testing development kit is used for managing plug-in configuration of the commissioning and testing plug-in, managing the life cycle of the commissioning and testing plug-in, realizing data communication between the commissioning and testing plug-in and commissioning the commissioning and testing plug-in; the adjusting and testing container is used for bearing the adjusting and testing plug-in; and the visual interaction interface is used for displaying the operation information of the debugging and testing plug-in and receiving a control instruction of a user for the debugging and testing device. According to the technical scheme provided by one or more embodiments, the development and test process of the automatic driving application can be accelerated.

Description

Technical Field

[0001] The present disclosure relates to the technical field of testing of autonomous driving applications, and in particular to a debugging device, method and computer program product for autonomous driving applications. Background Art

[0002] With the development of smart car technology, the autonomous driving applications applied to smart cars are also constantly updated. These autonomous driving applications may involve automatic cruising, automatic following, automatic lane changing, automatic parking and other aspects.

[0003] Currently, there are some graphical development tools for the development of autonomous driving applications, which improve the development efficiency of autonomous driving applications. However, autonomous driving applications created by these graphical development tools still need to be debugged before they can form finished applications and meet actual application scenarios.

[0004] However, most of the existing debugging toolboxes are built around a single functional module and lack sufficient flexibility to cope with the updates and iterations of autonomous driving application products. Summary of the invention

[0005] In view of this, one or more embodiments of the present disclosure provide a debugging device for an autonomous driving application, which has flexibility and scalability and can accelerate the development and testing process of the autonomous driving application.

[0006] In a first aspect, the present disclosure provides a debugging device for an autonomous driving application, the debugging device comprising a debugging plug-in, a debugging development toolkit, a debugging container and a visual interactive interface; wherein the debugging plug-in is used to perform debugging tasks for the autonomous driving application; the debugging development toolkit is used to manage the plug-in configuration of the debugging plug-in, manage the life cycle of the debugging plug-in, realize data communication between the debugging plug-in and the debugging container, and debug the debugging plug-in; the debugging container is used to carry the debugging plug-in; the visual interactive interface is used to display the running information of the debugging plug-in and receive user control instructions for the debugging device.

[0007] The plug-in architecture allows the separation of the debugging plug-in and the debugging container. The debugging development toolkit can simplify the development, integration and debugging process of the debugging plug-in. The interaction between the debugging plug-in and the debugging container is carried out through the debugging development toolkit, which ensures the high compatibility and low coupling between the debugging plug-in and the debugging container. In addition, the plug-in architecture allows developers to flexibly add, remove or update different debugging function modules in the debugging device according to their needs. This means that new debugging functions can be easily integrated into the debugging device without modifying the core code of the debugging device or redesigning the entire debugging device. The running information of the debugging plug-in can be displayed through a visual interactive interface, which enhances the user experience.

[0008] In one embodiment, the management of the plug-in configuration of the debugging plug-in includes: accessing the debugging container to obtain a data template, the data template including basic information items, area configuration items, rendering function items and functional function items; for a target plug-in in the debugging plug-in, obtaining the plug-in parameters indicated by the data template; using the plug-in parameters, configuring the data template to generate the plug-in configuration of the target plug-in.

[0009] The commissioning development toolkit obtains the data template from the commissioning container and creates the plug-in configuration of the target plug-in according to the data template, thereby ensuring that the plug-in configuration has a consistent data structure and facilitating the management and use of different commissioning plug-ins.

[0010] In one implementation, managing the life cycle of the debugging plug-in includes at least one of registering, loading, updating, and unloading the debugging plug-in.

[0011] The life cycle of the debugging plug-in is uniformly managed by the debugging development toolkit, which can conveniently install, uninstall and update the debugging plug-in, ensuring the stable operation of the debugging plug-in.

[0012] In one embodiment, the implementation of data communication between the debugging plug-in and the debugging container includes: loading the debugging plug-in into the debugging container and maintaining a data interaction channel between the debugging container and the debugging plug-in; monitoring a first message sent by the debugging container through a first event handler, and transmitting the first message to the debugging plug-in; monitoring a second message sent by the debugging plug-in through a second event handler, and transmitting the second message to the debugging container.

[0013] The debugging development toolkit provides a message transmission channel between the debugging container and the debugging plug-in. The message transmission method is event-driven, so that the debugging plug-in and the debugging container can perform corresponding operations in time according to each other's state changes. In this way, the communication between the debugging plug-in and the debugging container is efficient and low-coupling. The debugging plug-in and the debugging container can process data and events independently.

[0014] In one embodiment, the debugging of the debugging plug-in includes: obtaining a debugging instruction sent by a user, and debugging the debugging plug-in according to the debugging instruction; recording a debugging operation log of the debugging plug-in; monitoring the debugging operation status of the plug-in, and reporting operation error information of the debugging plug-in.

[0015] The debugging development toolkit provides support for rapid integration, development, and debugging of debugging plug-ins, and supports performance testing, integration testing, and semantic testing of debugging plug-ins, ensuring the high quality and stability of debugging plug-ins. Users can separate different debugging tasks for autonomous driving applications and have them executed by different independent debugging plug-ins, making the development, maintenance, and update of debugging devices more efficient. Each debugging plug-in can focus on optimizing a part of the application functions of the autonomous driving application, reducing the complexity of the debugging project.

[0016] In one embodiment, the visual interactive interface is specifically used to: adjust the interface layout of the plug-in identifier of the debugging plug-in in the visual interactive interface in response to a user's drag command; change the interface theme of the visual interactive interface in response to a user's theme command; and manage the interface window of the debugging plug-in in response to a user's window command, wherein the interface window is used to display the running information of the debugging plug-in.

[0017] The debugging device provides a flexible interface layout adjustment function, allowing users to customize the visual interface according to their needs. The debugging device also supports multiple interface theme switching (such as dark mode, light mode, etc.), allowing users to choose the appropriate interface theme according to their personal preferences to match the current working environment. The running information of the debugging plug-in can be displayed in the interface window, which can provide information on changes in the plug-in running status and running results.

[0018] In one embodiment, the debugging container is also used to: create, open or close a debugging project in response to a user's engineering instruction, wherein the debugging project management includes multiple plug-in instances of the debugging plug-in; and manage the engineering data of the debugging project in response to a user's data instruction.

[0019] The commissioning container provides a unified commissioning project management function, which can support the storage of multiple project files and efficiently manage the commissioning plug-ins involved in each commissioning project. The configuration or data of each project file and plug-in instance can be managed independently, thus meeting the needs of multi-task collaboration.

[0020] In one embodiment, the debugging device includes an application layer, a support layer and an operation layer; the application layer obtains the user's business request through the visual interactive interface and the input interface of the debugging development toolkit; the support layer is used to analyze the business request and obtain the business resources and business services indicated by the business request from the operation layer; the operation layer is used to run the debugging plug-in, the debugging development toolkit and the debugging container to provide the business resources and the business services.

[0021] The application layer focuses more on achieving user needs and business goals, while the operation layer provides basic business services and business resources. The support layer connects the application layer and the operation layer. After analyzing user needs and business goals, it can dispatch business services and business resources from the operation layer to achieve user needs and business goals. The three-level design helps to better design, develop, and maintain the debugging equipment for autonomous driving applications.

[0022] In a second aspect, the present disclosure provides a method for debugging an autonomous driving application, which uses the debugging device for the autonomous driving application provided in the first aspect to debug a target autonomous driving application.

[0023] In a third aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the debugging method for the autonomous driving application corresponding to the second aspect.

[0024] In a fourth aspect, the present invention provides a computer program product, comprising computer instructions, which are used to enable a computer to execute the debugging method of the autonomous driving application corresponding to the second aspect.

[0025] The technical solution provided by one or more embodiments of the present disclosure introduces a plug-in architecture to support the independent development, rapid integration and debugging of debugging plug-ins, so that the debugging device of the autonomous driving application has higher flexibility and scalability. The plug-in architecture allows developers to flexibly add, remove or update different modules according to needs, which improves the development efficiency of the adjustment device and further improves the development efficiency of the autonomous driving application. Through the plug-in architecture, developers and testers can select and integrate functional modules according to their needs, thereby accelerating the development and testing process of the autonomous driving application. In addition, the visual interactive interface of the debugging device can provide a flexible interface customization mechanism, real-time plug-in interaction and notification mechanism, further improving the development efficiency and user experience of the debugging device.

[0026] The technical solution provided by one or more embodiments of the present disclosure allows users to separate different debugging tasks for autonomous driving applications and execute them by different independent debugging plug-ins, making the development, maintenance and update of debugging devices more efficient. Each debugging plug-in can focus on optimizing a part of the application functions of the autonomous driving application, reducing the complexity of the debugging project. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The features and advantages of the embodiments of the present disclosure may be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present disclosure in any way. In the accompanying drawings:

[0028] Figure 1 A schematic diagram showing the composition structure of a debugging device for an autonomous driving application in one embodiment of the present disclosure is shown;

[0029] Figure 2 A schematic diagram of a process for managing plug-in configuration in one embodiment of the present disclosure is shown;

[0030] Figure 3 A schematic diagram of a process of loading a debugging plug-in in one embodiment of the present disclosure is shown;

[0031] Figure 4 A schematic diagram of a process of uninstalling a debugging plug-in in one embodiment of the present disclosure is shown;

[0032] Figure 5 A schematic diagram of a process of integrating a debugging plug-in in one embodiment of the present disclosure is shown;

[0033] Figure 6 A schematic diagram of a hierarchical design of a debugging device for an autonomous driving application in one embodiment of the present disclosure is shown;

[0034] Figure 7 A schematic diagram of the development process of a debugging device for an autonomous driving application in one embodiment of the present disclosure is shown;

[0035] Figure 8 A schematic diagram showing program generation of a debugging device for an autonomous driving application in one embodiment of the present disclosure is shown;

[0036] Fig. 9 A schematic diagram of the operation process of a debugging device for an autonomous driving application in one embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0038] The present disclosure finds that during the debugging stage of autonomous driving applications, there may be a variety of debugging task requirements such as data recording and playback, visual debugging, system monitoring and debugging, and performance evaluation. In some actual application scenarios, the debugging process of autonomous driving applications may process various types of massive sensor data, including video data, radar echo data, lidar point cloud data, global positioning system (GPS) data, etc. The debugging process of autonomous driving applications may need to provide data visualization capabilities, including point cloud visualization, image processing, radar data display, map annotation, etc. Under normal circumstances, the debugging process of autonomous driving applications may also need to evaluate the performance of autonomous driving products, such as response time, path planning accuracy, system stability, computing load, etc.

[0039] In related technologies, some integrated debugging toolboxes can monitor and debug autonomous driving products and verify various modules of autonomous driving products, such as perception, planning, control, and decision-making modules. However, when faced with a new autonomous driving product or when adding new debugging tasks to an autonomous driving product, it is often necessary to modify the core code of the debugging toolbox, resulting in a long development cycle and high maintenance costs for the debugging toolbox.

[0040] In related technologies, debugging toolboxes usually require large-scale updates to introduce new debugging functions or debugging frameworks. However, with the rapid development of autonomous driving technology, new autonomous driving algorithms and new autonomous driving products are emerging in an endless stream, which requires the debugging toolbox to be able to flexibly integrate new debugging functions and debug these new autonomous driving algorithms or new autonomous driving products.

[0041] In the related art, most commissioning toolboxes provide standardized functions and interfaces, but lack personalized customization support for commissioning personnel. However, commissioning personnel often want to customize different interfaces, plug-ins, test scenarios, etc. according to their own needs.

[0042] In the related art, most debugging toolboxes are monolithic structures with tightly coupled functions and modules. When expanding or updating a functional module, a large amount of code often needs to be modified, which leads to the instability of the debugging toolbox and increases the difficulty of maintaining the debugging toolbox.

[0043] In view of this, the debugging device of the autonomous driving application provided by one or more embodiments of the present disclosure can support the plug-in development and management of the debugging device. The user can disassemble different debugging tasks for the autonomous driving application and execute them by different independent debugging plug-ins, making the development, maintenance and update of the debugging device more efficient. Each debugging plug-in can focus on optimizing a part of the application functions of the autonomous driving application, reducing the complexity of the debugging project.

[0044] See also Figure 1 , an embodiment of the present disclosure provides a debugging device for an autonomous driving application, the debugging device comprising a debugging plug-in, a debugging development toolkit, a debugging container and a visual interactive interface. The debugging plug-in is used to perform debugging tasks for the autonomous driving application. The debugging development toolkit is used to manage the plug-in configuration of the debugging plug-in, manage the life cycle of the debugging plug-in, realize data communication between the debugging plug-in and the debugging container, and debug the debugging plug-in. The debugging container is used to carry the debugging plug-in. The visual interactive interface is used to display the running information of the debugging plug-in and receive control instructions from the user for the debugging device.

[0045] In this embodiment, the debugging device of the automatic driving application can optimize the automatic driving products of the smart car, such as optimizing the adaptive cruise function products of the automatic driving system (including the vehicle speed control function and the vehicle distance control function), the advanced driving assistance function products (including the intelligent lane change function, the intelligent overtaking function, the traffic congestion warning function, the lane keeping function, the automatic avoidance function), and the automatic parking function products (including the parking space recognition function and the parking space entry function). In some practical application examples, the automatic driving product can be generated by a graphical developer.

[0046] In this embodiment, the debugging plug-in can perform one or more debugging functions for the autonomous driving product, including autonomous driving data acquisition, autonomous driving data recording, autonomous driving data visualization, autonomous driving data analysis, etc. According to the actual application scenario, different debugging plug-ins can be developed to collect various types of data related to autonomous driving, such as video data, radar echo data, lidar point cloud data, global positioning system (GPS) data, etc. According to the actual application scenario, different debugging plug-ins can be developed to visualize autonomous driving related data, such as point cloud visualization, image processing, radar data display, map annotation, etc. According to the actual application scenario, different debugging plug-ins can be developed to test different autonomous driving algorithms, such as scene perception algorithm, path planning algorithm, driving decision algorithm, vehicle control algorithm, etc. According to the actual application scenario, different debugging plug-ins can be developed to evaluate different aspects of the performance indicators of autonomous driving products, such as response time, path planning accuracy, system stability, computing load, etc.

[0047] In this embodiment, the debugging development toolkit can be in the form of a software development kit (SDK), which can assist in the development of debugging devices for autonomous driving applications. The debugging development toolkit can realize data communication between the debugging plug-in and the debugging container, and can manage and debug the debugging plug-in. The debugging development toolkit can receive management instructions issued by the user through a visual interactive interface, and manage and debug the debugging plug-in. In some practical application examples, the debugging development toolkit can also provide an application program interface (API), so that users can directly manage and debug the debugging plug-in through the application program interface.

[0048] In this embodiment, the debugging container can carry various debugging plug-ins to ensure the stable operation of the debugging plug-ins and complete the stable debugging of the autonomous driving application. The debugging container can also manage debugging projects. Each debugging project can contain multiple debugging tasks for autonomous driving products. These debugging tasks can be executed by different debugging plug-ins. These debugging tasks can verify the capabilities of autonomous driving products in perception, planning, control, and decision-making. These debugging tasks include but are not limited to the verification of autonomous driving algorithms, functional debugging of autonomous driving products, performance evaluation of autonomous driving products, and data analysis of autonomous driving products.

[0049] In this embodiment, the visual interactive interface can display the running information of the debugging plug-in. The running information of the debugging plug-in can include two aspects. On the one hand, these debugging information can display the running status of the debugging plug-in. On the other hand, these debugging information can display the debugging progress and debugging results of the autonomous driving product.

[0050] In this embodiment, the plug-in architecture allows the debugging plug-in to be separated from the debugging container. The debugging development toolkit can be used to simplify the development, integration and debugging process of the debugging plug-in. The interaction between the debugging plug-in and the debugging container is carried out through the debugging development toolkit, which ensures the high compatibility and low coupling between the debugging plug-in and the debugging container. In addition, the plug-in architecture allows developers to flexibly add, remove or update different debugging function modules in the debugging device according to needs. This means that new debugging functions can be easily integrated into the debugging device without modifying the core code of the debugging device or redesigning the entire debugging device. The running information of the debugging plug-in can be displayed through a visual interactive interface, which enhances the user experience.

[0051] In some embodiments, the management of the plug-in configuration of the debugging plug-in includes: accessing the debugging container to obtain a data template, the data template including basic information items, area configuration items, rendering function items and functional function items; for the target plug-in in the debugging plug-in, obtaining the plug-in parameters indicated by the data template; using the plug-in parameters, configuring the data template to generate the plug-in configuration of the target plug-in.

[0052] Specifically, the debugging container can define a set of standardized data templates (Schema) to manage the static configuration of the debugging plug-in. The data template can include template sub-items such as basic information, area configuration, rendering function, and function function. Under these template sub-items, you can further define configuration items such as the plug-in name, plug-in icon, plug-in version, the display effect of the plug-in in the visual interactive interface, and the event response function of the plug-in.

[0053] After obtaining the data template defined by the debugging container, the debugging development toolkit can obtain the plug-in parameters from the target plug-in. Using the plug-in parameters, the debugging development toolkit can modify or update the configuration items in the data template and generate a plug-in configuration file for the target plug-in. The debugging development toolkit can also synchronize the plug-in configuration file or the plug-in configuration information of the plug-in configuration file to the debugging container.

[0054] The commissioning development toolkit obtains the data template from the commissioning container and creates the plug-in configuration of the target plug-in according to the data template, thereby ensuring that the plug-in configuration has a consistent data structure and facilitating the management and use of different commissioning plug-ins.

[0055] In an actual application example, the data template may include a basic information template sub-item, under which the initial information of configuration items such as the plug-in name, plug-in icon, plug-in version, and plug-in publisher may be defined. The data template may include an area configuration template sub-item, under which the initial information of configuration items such as the plug-in sidebar icon and plug-in title may be defined. The data template may include a rendering function template sub-item, under which the initial rendering function of the plug-in may be defined. The data template may include a function function template sub-item, under which the processing functions for when the plug-in is started, closed, and ready may be defined. For example, a notification when the debugging plug-in is opened, and a prompt to the user to save the data in the plug-in when the debugging plug-in is closed.

[0056] See also Figure 2In an actual application example, SDK Builder is the main management unit of the debugging development toolkit. The data template obtained by the debugging development toolkit from the debugging container can be stored by SDK Builder. The configuration management unit of the debugging development toolkit can obtain the latest data template from SDK Builder and update the data of the data template. The configuration management unit of the debugging development toolkit can read the plug-in parameters from the target debugging plug-in, and use the plug-in parameters to update the initial information of the data template to generate the plug-in configuration file of the target plug-in. The configuration management unit of the debugging development toolkit can feed back the completed plug-in configuration information to SDK Builder for storage.

[0057] In some implementations, managing the life cycle of the debugging plug-in includes at least one of registering, loading, updating, and unloading the debugging plug-in.

[0058] Specifically, users can issue management instructions to the debugging development tool through the visual interactive interface of the debugging device to control the life cycle of the debugging plug-in. Users can also directly control the life cycle of the debugging plug-in through the API provided by the debugging development toolkit. The life cycle of the debugging plug-in is uniformly managed by the debugging development toolkit, which can conveniently install, uninstall and update the debugging plug-in. The debugging plug-in is registered through the debugging development tool, which can be registered with information such as the plug-in name, plug-in version, and plug-in configuration. The debugging development toolkit can load debugging plug-ins from local resources, and can also load debugging plug-ins from remote resources. The debugging development toolkit can support automatic and manual updates of debugging plug-ins. The debugging development toolkit can uninstall the debugging plug-in and delete the configuration and files related to the debugging plug-in.

[0059] Manage the life cycle of the debugging plug-in. You can use the file path, plug-in ID and other information of the debugging plug-in to quickly determine the debugging plug-in that needs to be managed.

[0060] See also Figure 3 In an actual application example, the debugging development toolkit (SDK) can receive instructions issued by the user and start loading the candidate debugging plug-in into the debugging container. After the debugging development toolkit determines the file path of the candidate debugging plug-in, it loads the plug-in instance of the candidate debugging plug-in. The plug-in instance needs to check whether the startup parameters and startup conditions are met in turn. When the plug-in instance meets the startup conditions, the debugging development toolkit can associate the plug-in instance and establish a data interaction channel between the plug-in instance and the debugging container to complete the loading process.

[0061] See also Figure 4In an actual application example, the debugging development toolkit (SDK) can receive instructions issued by the user and start uninstalling the candidate debugging plug-in from the debugging container. After the debugging development toolkit determines the plug-in identifier of the candidate debugging plug-in, it can query the candidate debugging plug-in and uninstall it (for example, through the uninstall interface of the SDK). When the candidate plug-in is uninstalled, the debugging development toolkit can notify the debugging container of the processing result so that the debugging container can update the loaded plug-in list.

[0062] In some embodiments, implementing data communication between the debugging plug-in and the debugging container includes: loading the debugging plug-in into the debugging container and maintaining a data interaction channel between the debugging container and the debugging plug-in; listening to a first message sent by the debugging container through a first event processor, and transmitting the first message to the debugging plug-in; listening to a second message sent by the debugging plug-in through a second event processor, and transmitting the second message to the debugging container.

[0063] Specifically, the first event handler and the second event handler may be onmessage handlers for monitoring messages. The onmessage event handler may be used for cross-domain communication, allowing message transmission between different sources. In this way, the debugging plug-in may receive the first message distributed by the debugging container through the onmessage method; the debugging container may receive the second message distributed by the debugging plug-in through the onmessage method. In addition, the debugging container may distribute the first message to the debugging plug-in through the sendmessage method; and the debugging plug-in may distribute the second message to the debugging container through the sendmessage method.

[0064] The debugging development toolkit provides a message transmission channel between the debugging container and the debugging plug-in. The message transmission method is event-driven, so that the debugging plug-in and the debugging container can perform corresponding operations in time according to each other's state changes. In this way, the communication between the debugging plug-in and the debugging container is efficient and low-coupling. The debugging plug-in and the debugging container can process data and events independently.

[0065] In an actual application example, the debugging container and the debugging plug-in have their own servers and clients. The server is responsible for business processing, and the client is responsible for the user interface and interaction. The message transmission between the debugging container and the debugging plug-in can exist in the following three modes: the server of one party sends a persistent message to the client of the other party; according to the subscription topic, the debugging container and the debugging plug-in send or receive information related to the subscription topic; the client of one party performs regular polling to the server of the other party.

[0066] In some implementations, debugging the debugging plug-in includes: obtaining a debugging instruction sent by a user, and debugging the debugging plug-in according to the debugging instruction; recording a debugging operation log of the debugging plug-in; monitoring the debugging operation status of the plug-in, and reporting operation error information of the debugging plug-in.

[0067] Specifically, plug-in developers can use the debugging development toolkit to quickly integrate the debugging plug-in into the debugging container for debugging. When the debugging plug-in is loaded in the debugging container, the debugging development toolkit can automatically handle the data interaction and configuration synchronization between the debugging plug-in and the debugging container. Through the plug-in debugging tool provided by the debugging development toolkit, developers can directly view the behavior of the debugging plug-in during the development process and debug it to ensure that the debugging plug-in runs stably in the main container.

[0068] The built-in debugging function of the debugging development toolkit can also support logging and user-level trace tracking. Developers can use the debugging interface provided by the debugging development toolkit to track the status and error information of the debugging plug-in in real time to ensure that potential vulnerabilities (bugs) can be fixed before the debugging plug-in is released. The running log of the debugging plug-in can be automatically recorded, and log query and log export functions are provided, which is conducive to developers to optimize the performance and error analysis of the debugging plug-in. The error tracking function provided by the debugging development toolkit can monitor the running status of the debugging plug-in in real time and report when the debugging plug-in is abnormal. The error information of the debugging plug-in can be fed back to the developer through the notification mechanism to ensure the high quality of plug-in development.

[0069] The debugging development toolkit provides support for rapid integration, development, and debugging of debugging plug-ins, and supports performance testing, integration testing, and semantic testing of debugging plug-ins, ensuring the high quality and stability of debugging plug-ins. The debugging toolbox breaks down different debugging functions and executes them separately by different independent debugging plug-ins, making the development, maintenance, and updating of the debugging toolbox more efficient. Each debugging plug-in can focus on solving specific technical problems, reducing the complexity of the debugging project.

[0070] See also Figure 5In an actual application example, SDK Builder is the main management unit of the debugging development toolkit, which can be used to build and package the debugging development toolkit so that the debugging development toolkit can run in different software system environments. The data template obtained by the debugging development toolkit from the debugging container can be stored by SDK Builder. The configuration management unit of the debugging development toolkit can obtain the latest data template from SDK Builder and update the data of the data template. The configuration management unit of the debugging development toolkit can read the plug-in parameters from the target debugging plug-in, and use the plug-in parameters to update the initial information of the data template to generate a plug-in configuration file for the target debugging plug-in. Subsequently, the debugging container can start integrating the debugging plug-in and read the generated plug-in configuration file of the target debugging plug-in from the debugging development toolkit. The debugging container can verify the plug-in configuration file. After the verification passes, the debugging container completes the integration of the target debugging plug-in.

[0071] In some embodiments, the visual interactive interface is specifically used to: adjust the interface layout of the plug-in identifier of the debugging plug-in in the visual interactive interface in response to a user's drag command; change the interface theme of the visual interactive interface in response to a user's theme command; and manage the interface window of the debugging plug-in in response to a user's window command, wherein the interface window is used to display the running information of the debugging plug-in.

[0072] Specifically, the debugging device provides a flexible interface layout adjustment function, allowing users to customize the visual interface as needed. The debugging device also supports multiple interface theme switching (such as dark mode, light mode, etc.), allowing users to choose a suitable interface theme according to personal preferences to match the current working environment. The running information of the debugging plug-in can be displayed in the interface window, and the change information of the plug-in running status and the running result information can be provided. In some actual application scenarios, the plug-in instance of the debugging plug-in can be displayed on the main interface to realize the notification and reminder function of the plug-in status change. The interaction between plug-in instances can be carried out in the form of floating windows, public views, etc., which enhances the user experience.

[0073] In an actual application example, users can freely adjust the layout of different modules on the interactive interface through the drag and drop function. For example, users can freely adjust the interface layout of the visual interactive interface by dragging modules such as the tab bar, sidebar, and plug-in window. When dragging the tab bar, users can drag and drop different debugging plug-ins to the interactive interface, and multiple debugging plug-ins can be switched and displayed through the tab bar. When dragging the sidebar and plug-in window, the sidebar and plug-in window can be freely scaled and moved, supporting multi-column and single-column display, providing a more flexible interface display method.

[0074] In an actual application example, the debugging device provides a variety of theme options. Users can choose different themes such as dark mode and light mode to meet the user's personalized visual needs. Each theme is designed to be consistent in color, font, control style, etc. to ensure visual coordination. The theme switching function is implemented through the built-in theme management module of the debugging development toolkit. Users can easily select and switch themes in the visual interactive interface. For the page layout configured by the user, the debugging device can persist it, and the layout will be automatically loaded and displayed the next time the same debugging project is opened.

[0075] In a practical application example, the commissioning device supports opening multiple window instances in parallel, allowing users to process multiple project files and plug-in instances at the same time. Each project can open different tool instances, supporting cross-task collaboration and independent management of project data. Through multi-window mode, users can handle multi-task scenarios more efficiently and avoid frequent switching and interruptions.

[0076] In one embodiment, the debugging container is also used to: create, open or close a debugging project in response to a user's engineering instruction, wherein the debugging project management includes multiple plug-in instances of the debugging plug-in; and manage the engineering data of the debugging project in response to a user's data instruction.

[0077] Specifically, the commissioning container provides a unified commissioning project management function, which can support opening multiple project files at the same time, and can efficiently manage the commissioning project and the commissioning plug-ins involved in the commissioning project. The configuration or data of each project file and plug-in instance can be managed independently, so as to meet the needs of multi-task collaboration.

[0078] In a practical application example, the debugging device provides a unified project management function, and users can create, open, and close project files on a visual interactive interface. Each project file can contain independent configurations and data, and the data between multiple project instances does not interfere with each other. Through the visual interactive interface of the debugging device, users can create new project files and perform related configurations. The structure and data of each project file can be defined and managed through a preset data framework. The debugging device can automatically manage the storage path of the project file to ensure data isolation between different versions and different users. The content of each project file can be edited independently and can be saved as different versions.

[0079] In a practical application example, the debugging device can provide module-based data management functions through the Projectize module to perform operations such as adding, deleting, modifying, and checking engineering data. All engineering data can be managed through standardized data structures to ensure consistent data structures and formats. The debugging device supports persistent storage of engineering data and provides flexible data query and retrieval functions. Each data item can be defined through a standardized data structure to ensure data consistency and integrity. The debugging device can manage access rights to engineering data to ensure that users with different permissions can access and modify the corresponding data content.

[0080] In some embodiments, see Figure 6 The debugging device includes an application layer, a support layer and an operation layer; the application layer obtains the user's business request through the visual interactive interface and the input interface of the debugging development toolkit; the support layer is used to analyze the business request and obtain the business resources and business services indicated by the business request from the operation layer; the operation layer is used to run the debugging plug-in, the debugging development toolkit and the debugging container to provide the business resources and the business services.

[0081] Specifically, the Use application layer is used for business encapsulation and implementation of the tool plug-in itself. Among them, the plug-in view (plug-in View) is used to display the user interface of the plug-in. The plug-in server (plug-in Server) is used to process the logic and data of the plug-in. The configuration module (Config) is generated by the data template (Schema) of the main container and is used to manage and apply the configuration of the plug-in. SDKBuilder is used to build and package the SDK so that it can run in different environments.

[0082] The Providers support layer is used to provide framework-level support for tool plug-ins. Among them, the client SDK (SDKClient) is used to communicate with the server. The server SDK (SDK Server) is used to process client requests. SDKSchema is responsible for verifying plug-in information.

[0083] Runtime is the implementation layer, which is used to provide the underlying operating environment and basic implementation of the plug-in. Among them, the SDK runtime environment (SDK Runtime) is used to run the SDK. The main container is used to carry and manage different plug-ins. The virtual main container (Nil) is a plug-in that is packaged and run independently of the main container, such as logging, monitoring, calibration and other tools.

[0084] In this implementation, the application layer focuses more on achieving user needs and business goals, and the operation layer provides basic business services and business resources. The support layer connects the application layer and the operation layer, and can analyze user needs and business goals, and then dispatch business services and business resources from the operation layer to achieve user needs and business goals. The three-level design helps to better design, develop, and maintain the debugging device of the autonomous driving application.

[0085] In this embodiment, the debugging device is divided into two parts, the main container and the plug-in, through the plug-in architecture design. The plug-in and the main container communicate through the SDK. The main container is responsible for unified project management and user interface (UI) display, while the plug-in provides extensions for specific functions. The SDK provides complete support including plug-in loading, plug-in configuration management, plug-in life cycle control, etc., to ensure high compatibility between the plug-in and the main container.

[0086] In some embodiments, see Figure 7 , with the help of the unified UI rendering engine used by the toolbox SDK, the consistency of UI interface presentation on different platforms is ensured. In the development and adjustment phase, the Node runtime is integrated, and NativeAPI adaptation is provided to the application layer, which ensures the consistency of interface calls at the business layer and isolates the differences of Native APIs on different platforms.

[0087] In some embodiments, see Figure 8 , using SDK Builder can support multi-platform packaging and generate installation packages for different platforms (.exe, .dmg, .deb, etc.).

[0088] In some embodiments, see Fig. 9 ,Through the above mechanisms, the debugging device can cleverly solve the ,complexity of cross-platform development, allowing developers to quickly build desktop ,applications using the toolbox SDK.

[0089] In these implementations, the plug-in toolbox can support cross-platform development, and multiple plug-ins can be developed independently and adapted to different operating environments. This makes it feasible to use a unified toolbox on different autonomous driving hardware platforms. The toolbox SDK plug-in architecture can be easily opened to third-party developers, who can add new features to the toolbox or optimize existing features. This is a current trend in the field of autonomous driving, especially driven by open source tools and ecosystems.

[0090] The technical solution provided by one or more embodiments of the present disclosure introduces a plug-in architecture to support the independent development, rapid integration and debugging of debugging plug-ins, so that the debugging device of the autonomous driving application has higher flexibility and scalability. The plug-in architecture allows developers to flexibly add, remove or update different modules according to needs, which improves the development efficiency of the adjustment device and further improves the development efficiency of the autonomous driving application. Through the plug-in architecture, developers and testers can select and integrate functional modules according to their needs, thereby accelerating the development and testing process of the autonomous driving application. In addition, the visual interactive interface of the debugging device can provide a flexible interface customization mechanism, real-time plug-in interaction and notification mechanism, further improving the development efficiency and user experience of the debugging device.

[0091] The technical solution provided by one or more embodiments of the present disclosure allows users to separate different debugging tasks for autonomous driving applications and execute them by different independent debugging plug-ins, making the development, maintenance and update of debugging devices more efficient. Each debugging plug-in can focus on optimizing a part of the application functions of the autonomous driving application, reducing the complexity of the debugging project.

[0092] It should be noted that the technical solutions provided by one or more embodiments of the present disclosure can be used to adjust and test the autonomous driving system and can be applied to smart cars. It is foreseeable that the autonomous driving adjustment device provided by one or more embodiments of the present disclosure can also be applied to smart devices with autonomous driving logic, such as drones and smart robots.

[0093] The method for developing an autonomous driving debugging device provided in one embodiment of the present disclosure can be implemented by the debugging device of the autonomous driving application provided above to complete the debugging work of the autonomous driving application.

[0094] One embodiment of the present disclosure also provides a computer-readable storage medium, and the debugging method of the autonomous driving application provided according to the embodiment of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be downloaded through a network and originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0095] A portion of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide a debugging method for an autonomous driving application provided according to an embodiment of the present disclosure through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in a computer-readable medium includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which the computer program instructions are executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0096] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device, equipment, and storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0097] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

[0098] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A debugging device for an autonomous driving application, characterized in that: The debugging device includes a debugging plug-in, a debugging development toolkit, a debugging container and a visual interactive interface; wherein, The debugging plug-in is used to perform a debugging task for the autonomous driving application; The debugging development toolkit is used to manage the plug-in configuration of the debugging plug-in, manage the life cycle of the debugging plug-in, realize data communication between the debugging plug-in and the debugging container, and debug the debugging plug-in; The debugging container is used to carry the debugging plug-in; The visual interaction interface is used to display the operation information of the debugging plug-in and receive the user's control instructions for the debugging device.

2. The device according to claim 1, characterized in that The managing the plug-in configuration of the debugging plug-in includes: Access the debugging container to obtain a data template, where the data template includes a basic information item, an area configuration item, a rendering function item, and a functional function item; For a target plug-in in the debugging plug-in, obtaining plug-in parameters indicated by the data template; The plug-in parameters are used to configure the data template to generate the plug-in configuration of the target plug-in.

3. The device according to claim 1, characterized in that The managing the life cycle of the debugging plug-in includes at least one of registering, loading, updating and unloading the debugging plug-in.

4. The device according to claim 1, characterized in that The implementing data communication between the debugging plug-in and the debugging container includes: Loading the debugging plug-in into the debugging container, and maintaining a data interaction channel between the debugging container and the debugging plug-in; monitoring, through a first event processor, a first message sent by the debugging container, and transmitting the first message to the debugging plug-in; The second event handler monitors the second message sent by the debugging plug-in, and transmits the second message to the debugging container.

5. The device according to claim 1, characterized in that The debugging of the debugging plug-in includes: Obtaining a debugging instruction sent by a user, and debugging the debugging plug-in according to the debugging instruction; Record the debugging operation log of the debugging plug-in; Monitor the debugging running status of the plug-in and report the running error information of the debugging plug-in.

6. The device according to any one of claims 1 to 5, characterized in that: The visual interaction interface is specifically used for: In response to a user's drag command, adjusting the interface layout of the plug-in identifier of the debugging plug-in in the visual interactive interface; In response to a user's theme instruction, changing the interface theme of the visual interactive interface; In response to a window instruction from a user, an interface window of the debugging plug-in is managed, where the interface window is used to display the running information of the debugging plug-in.

7. The device according to any one of claims 1 to 5, characterized in that: The debugging container is also used for: In response to a user's engineering instruction, create, open or close a commissioning project, wherein the commissioning project management includes a plurality of plug-in instances of the commissioning plug-in; In response to a data instruction from a user, the project data of the commissioning project is managed.

8. The device according to any one of claims 1 to 5, characterized in that: The debugging device includes an application layer, a support layer and an operation layer; The application layer obtains the user's service request through the visual interactive interface and the input interface of the debugging development toolkit; The support layer is used to obtain the business resources and business services indicated by the business request from the operation layer after analyzing the business request; The operation layer is used to run the debugging plug-in, the debugging development toolkit and the debugging container, and provide the business resources and the business services.

9. A method for debugging an autonomous driving application, characterized in that: Utilize the debugging device provided in any one of claims 1 to 8 to debug the target autonomous driving application.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the method for debugging an autonomous driving application as described in claim 9.

Images