Debugging system and debugging method

Abstract

A debugging system and a debugging method. The debugging system includes an environment simulator, a synchronizer and a component simulator. The environment simulator is configured to provide a simulation test environment, wherein the environment simulator includes multiple operation nodes, and the multiple operation nodes include a first operation node; the component simulator is configured to perform a first data processing, wherein the first operation node is a mapping of the first data processing performed by the component simulator in the environment simulator; the synchronizer is configured to establish a communication connection between the environment simulator and the first operation node of the component simulator, and is configured to adjust the environment clock frequency of the environment simulator to be consistent with the simulation clock frequency of the component simulator, and synchronize the communication between the component simulator and the environment simulation simulator. The debugging system can avoid the problem of data backlog, and both the environment simulator and the component simulator can work normally during the debugging process, and can better test the performance and functional safety of the first operation node.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a debugging system and a debugging method. Background Art

[0002] Autonomous driving, also known as unmanned driving, computer driving or wheeled mobile robots, is a cutting-edge technology that relies on computers and artificial intelligence technology to complete complete, safe and effective driving without human control. The functions of autonomous driving include positioning and path planning, environmental perception, behavioral decision-making and control, etc. After the environmental perception devices such as sensors provide information, the main control device will process this information according to the algorithm, make corresponding decisions and take actions. Summary of the invention

[0003] At least one embodiment of the present disclosure provides a debugging system, including an environment simulator, a synchronizer and a component simulator, wherein the environment simulator is configured to provide a simulation test environment, wherein the environment simulator includes multiple operation nodes, and the multiple operation nodes include a first operation node; the component simulator is configured to perform a first data processing, wherein the first operation node is a mapping of the first data processing performed by the component simulator in the environment simulator; the synchronizer is configured to establish a communication connection between the first operation node of the environment simulator and the component simulator, and is configured to adjust the environment clock frequency of the environment simulator to be consistent with the simulation clock frequency of the component simulator, and to synchronize the communication between the component simulator and the environment simulation simulator.

[0004] For example, in the debugging system provided by at least one embodiment of the present disclosure, a plurality of hardware debugging programs are run on the component simulator, and the plurality of hardware debugging programs include a first hardware debugging program, and the first hardware debugging program is configured to execute the first data processing in the component simulator; the plurality of operation nodes correspond one-to-one to the plurality of hardware debugging programs, and the first operation node is a mapping of the first hardware debugging program in the environment simulator.

[0005] For example, in the debugging system provided in at least one embodiment of the present disclosure, the environment simulator includes an environment simulation unit and a computing unit, the environment simulation unit is configured to simulate a simulation environment for autonomous driving testing, the computing unit includes the multiple operation nodes, and is configured to perform data processing through the multiple operation nodes and utilize the multiple hardware debugging programs to achieve vehicle control.

[0006] For example, in the debugging system provided by at least one embodiment of the present disclosure, the first operation node is configured to: receive the data to be processed sent by the environment simulation unit, forward the data to be processed to the synchronizer, so as to transmit the data to be processed to the first hardware debugging program through the synchronizer, and receive the calculation result of the first hardware debugging program through the synchronizer, and forward the calculation result to other operation nodes, wherein the other operation node is any operation node among the multiple operation nodes except the first operation node.

[0007] For example, in the debugging system provided by at least one embodiment of the present disclosure, the synchronizer is implemented in the form of a server, and the synchronizer is configured to forward the message received from the message sender to the message receiver after synchronization processing, wherein the message sender and the message receiver are the environment simulator or the component simulator, and the message sender and the message receiver are different.

[0008] For example, in the debugging system provided by at least one embodiment of the present disclosure, the synchronization processing performed by the synchronizer is completed through a synchronous communication protocol, and the synchronous communication protocol uses a preset storage area for data caching to complete the synchronization of message passing across clock domains between the environment simulator and the component simulator.

[0009] For example, in the debugging system provided by at least one embodiment of the present disclosure, before starting debugging, the synchronizer is configured to calculate the ratio of the current environment clock frequency of the environment emulator and the simulation clock frequency, and send the ratio to the environment emulator so that the environment emulator adjusts the environment clock frequency of the environment emulator according to the ratio to be consistent with the simulation clock frequency.

[0010] For example, in the debugging system provided by at least one embodiment of the present disclosure, the synchronizer is also configured to perform signal conversion, and the signal conversion includes parsing the received message according to the protocol format of the message sender, reorganizing the parsed message content according to the protocol format of the message receiver, and sending the reorganized message to the message receiver, and the message sender and the message receiver are the environment simulator or the component simulator.

[0011] For example, in the debugging system provided by at least one embodiment of the present disclosure, the component simulator includes a processing core, the processing core is configured to control and schedule the multiple hardware debugging programs to implement the vehicle control, and the processing core communicates with the multiple hardware debugging programs through an on-chip bus.

[0012] For example, in the debugging system provided by at least one embodiment of the present disclosure, the component simulator is a hardware simulation accelerator or a field programmable gate array.

[0013] At least one embodiment of the present disclosure provides a debugging method, comprising: using an environment simulator to collect data to be processed, and sending the data to be processed to a synchronizer, wherein the environment simulator includes multiple operation nodes, and the multiple operation nodes include a first operation node; processing the data to be processed by the synchronizer, and transmitting the processed data to be processed to a component simulator, wherein the environment clock frequency of the environment simulator and the simulation clock frequency of the component simulator are adjusted to be consistent by the synchronizer, and the processing at least includes synchronization processing; using the component simulator to perform first data processing on the received data to be processed, obtain a calculation result, and send the calculation result to the synchronizer; processing the calculation result by the synchronizer, and transmitting the processed calculation result to the first operation node, wherein the first operation node is a mapping of the first data processing performed by the component simulator in the environment simulator.

[0014] For example, in the debugging method provided in at least one embodiment of the present disclosure, the environment simulator includes an environment simulation unit and a computing unit, the environment simulation unit is used to simulate the simulation environment of the autonomous driving test, the computing unit includes the multiple operation nodes, and is configured to perform data processing through the multiple operation nodes and utilize multiple hardware debugging programs to achieve vehicle control, the debugging method also includes: forwarding the calculation results after the synchronous processing to other operation nodes through the first operation node, wherein the other operation nodes are any operation nodes among the multiple operation nodes except the first operation node.

[0015] For example, in the debugging method provided in at least one embodiment of the present disclosure, before using the environment simulator to collect the data to be processed, the debugging method also includes: calculating, through the synchronizer, the ratio of the current environment clock frequency of the environment simulator to the simulation clock frequency of the component simulator; and adjusting the environment clock frequency of the environment simulator according to the ratio so that the environment clock frequency of the environment simulation unit is consistent with the simulation clock frequency of the component simulator.

[0016] For example, in the debugging method provided in at least one embodiment of the present disclosure, collecting the data to be processed by using an environment simulator includes: collecting the data to be processed according to the adjusted environment clock frequency.

[0017] For example, in the debugging method provided in at least one embodiment of the present disclosure, a plurality of hardware debugging programs are run on the component simulator, and the plurality of hardware debugging programs include a first hardware debugging program, and the first hardware debugging program is configured to execute the first data processing in the component simulator; the component simulator includes a processing core, and the processing core is configured to control and schedule the plurality of hardware debugging programs to realize the vehicle control, and the processing core communicates data with the plurality of hardware debugging programs through an on-chip bus, and the component simulator is used to perform a first data processing on the received data to be processed to obtain a calculation result, and the calculation result is sent to the synchronizer, including: receiving the data to be processed through the processing core, and transmitting the data to be processed to the first hardware debugging program via the on-chip bus; performing the first data processing on the data to be processed by the first hardware debugging program to obtain the calculation result; transmitting the calculation result to the processing core through the on-chip bus, and forwarding it to the synchronizer by the processing core. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, but are not intended to limit the present disclosure.

[0019] Figure 1 It is a schematic diagram of the interface of an environmental simulation unit;

[0020] Figure 2 A schematic block diagram of a debugging system provided for at least one embodiment of the present disclosure;

[0021] Figure 3 A structural diagram of an environment simulator provided for at least one embodiment of the present disclosure;

[0022] Figure 4 A schematic structural diagram of a component simulator provided for at least one embodiment of the present disclosure;

[0023] Figure 5 A schematic flow chart of a debugging method provided for at least one embodiment of the present disclosure;

[0024] Figure 6 A data flow diagram of a debugging method provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0026] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0027] In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits detailed descriptions of some known functions and known components.

[0028] At present, the algorithms deployed on autonomous vehicles can be implemented with the help of the Robot Operating System (ROS).

[0029] The Robot Operating System provides a series of program libraries and tools to help software developers create robot application software. It is a highly flexible software architecture for writing robot software programs. It provides many functions such as hardware abstraction, device drivers, function libraries, visualization tools, messaging and package management. It also provides the tools and library functions needed to acquire, compile, write, and run code across computers.

[0030] An important concept in the robot operating system is the operation node (ROS node), which is the smallest process unit in the robot operating system. For example, a software package can have multiple executable files. After the executable file is run, it becomes a process (process), which is called an operation node in the robot operating system. From a program perspective, an operation node is an executable file (usually an executable file compiled by C++, a Python script) that is executed and loaded into the memory; from a functional perspective, an operation node is usually responsible for a single function. Because the functional modules of the autonomous driving algorithm are very complex, multiple operation nodes are usually set up in a distributed manner, and each operation node communicates point-to-point through topics.

[0031] For example, suppose an operation node needs a certain type of message (message). The operation node can be called a receiver or subscriber (subscribe). The operation node sends a request to the node manager (ROS Master). The node manager looks for a node that is publishing this type of topic. This node is called a publisher (talker). After the node manager finds it, it will help the subscriber and publisher establish a connection, and then the subscriber and publisher can communicate with each other.

[0032] Before an autonomous driving algorithm is deployed on a real vehicle, it usually needs to be verified by simulation, which can be completed using an environmental simulation unit running on a computer (such as a stand-alone machine, server, or cloud service).

[0033] Figure 1 Figure 1 is a schematic diagram of the interface of an environmental simulation unit. Figure 1 As shown, the environment simulation unit running in the computer can simulate the real traffic environment, pedestrian behavior, car sensor signals, etc. The environment simulation unit supports the development, training and verification of the autonomous driving system. For example, the environment simulation unit may include an open source simulator CARLA, which is an open source architecture that can support flexible configuration of sensor kits and environmental conditions. Of course, the environment simulation unit can also be any other platform that can simulate the simulation environment of the autonomous driving test, and the present disclosure does not make specific restrictions on this.

[0034] For example, the environment simulation unit can communicate with the robot operating system through the carla-ros-bridge software package, for example, communicate with the operation node in the robot operating system. Thus, the operation node receives the environment perception information collected by the environment simulation unit through various sensors attached to the vehicle as the simulation object, processes the environment perception information, obtains the calculation result (such as decision) and feeds it back to the environment simulation unit, and the environment simulation unit displays the action status of the vehicle according to the calculation result.

[0035] At present, software is usually used to construct operation nodes, for example, some open source frameworks for autonomous driving (such as Apollo or Autoware) are used to construct operation nodes. However, the speed of constructing operation nodes by software, such as executing operation nodes such as image processing, is limited by resources, resulting in low computing efficiency, which in turn affects the real-time nature of decision-making, which is very important for autonomous driving. To speed up calculations, hardware programs can be used to construct operation nodes. For example, some calculations of operation nodes are changed from CPU (Central Process Unit), GPU (Graphic Process Unit), AI accelerator, etc. to hardware processor implementation, for example, by FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), etc.

[0036] However, when debugging a hardware program, the simulation clock frequency of the simulation platform running the hardware program and the environmental clock frequency of the environmental simulation unit may be inconsistent, or even very different. For example, the simulation clock frequency of the simulation platform may be only 10 MHz (megahertz), but the environmental clock frequency of the environmental simulation unit may be in GHz (Ghertz), such as 2 GHz. The environmental simulation unit will collect environmental perception information according to the environmental clock frequency and send the environmental perception information to the simulation platform according to the environmental clock frequency. For example, the environmental simulation unit may collect environmental images around the vehicle at a frequency of several GHz and send the environmental images to the simulation platform in GHz, while the simulation platform can only receive and process images at a speed of 10 MHz. This will generate a large amount of data backlog and even cause subsequent programs to not work properly, making it impossible to debug the hardware program.

[0037] At least one embodiment of the present disclosure provides a debugging system, including an environment simulator, a synchronizer and a component simulator, wherein the environment simulator is configured to provide a simulation test environment, wherein the environment simulator includes multiple operation nodes, and the multiple operation nodes include a first operation node; the component simulator is configured to perform a first data processing, wherein the first operation node is a mapping of the first data processing performed by the component simulator in the environment simulator; the synchronizer is configured to establish a communication connection between the environment simulator and the first operation node of the component simulator, and is configured to adjust the environment clock frequency of the environment simulator to be consistent with the simulation clock frequency of the component simulator, and to synchronize the communication between the component simulator and the environment simulation simulator.

[0038] In the above embodiment, the debugging system includes a synchronizer, which can make the environment clock frequency of the environment simulator consistent with the simulation clock frequency of the component simulator, so that the environment simulator can collect environmental perception information as data to be processed according to the simulation clock frequency of the component simulator, and send the data to be processed to the component simulator according to the simulation clock frequency. The component simulator will not generate data backlogs, and both the environment simulator and the component simulator can work normally, and can better test the performance and functional safety of the first operation node.

[0039] Furthermore, in at least one example, the clock frequency of the environment simulator at the physical layer may not be completely synchronized with the component simulator. The synchronizer can also implement synchronous processing of communications and achieve communication synchronization across clock domains, further avoiding data backlogs and ensuring the real-time and accuracy of data communications.

[0040] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments.

[0041] Figure 2 A schematic block diagram of a debugging system provided for at least one embodiment of the present disclosure.

[0042] like Figure 2 As shown, the debugging system 100 includes an environment simulator 101 , a synchronizer 103 and a component simulator 102 .

[0043] For example, the environment simulator 101 is configured to provide a simulation test environment. For example, the environment simulator includes a plurality of operation nodes, and the plurality of operation nodes includes a first operation node. Figure 2 The first operation node is shown. Of course, the environment simulator 101 may also include a second operation node, a third operation node, etc., and the present disclosure does not impose any specific limitation on this.

[0044] For example, an environment simulator can provide something like Figure 1 The simulation test environment shown is used for the development, training and verification of the autonomous driving system. At the same time, the environment simulator can also include an operating system for running the autonomous driving algorithm, such as a robot operating system, to complete data processing and response strategy generation through multiple operation nodes, so that the operation status of the autonomous driving algorithm can be intuitively seen through the simulation test environment.

[0045] For the concept of operation nodes, please refer to the above content and will not be repeated here.

[0046] For example, the component simulator 102 is configured to perform a first data processing. For example, the first operation node is a mapping of the first data processing performed by the component simulator in the environment simulator.

[0047] For example, the first operation node can be implemented by pure software. A software program runs in the component simulator to perform the first data processing. The first operation node provides an interface to send the calculation results of the first data processing to other operation nodes or environment simulators through the communication method between operation nodes (such as topic).

[0048] For example, the first operation node can be implemented by a hardware program. For example, multiple hardware debugging programs are running on the component simulator, and the multiple hardware debugging programs include a first hardware debugging program, and the first hardware debugging program is configured to perform the first data processing in the component simulator. The multiple operation nodes correspond to the multiple hardware debugging programs one by one, and the first operation node is a mapping of the first hardware debugging program in the environment simulator.

[0049] That is to say, at this time, the operation node is not constructed by pure software, but is implemented by a hardware processor or chip, etc., and at this time, a hardware debugging program constitutes an operation node. Similarly, taking the first operation node as an example, the first operation node provides an interface, and uses the provided interface to send the calculation result when the first hardware debugging program executes the first data processing to other operation nodes or environment simulators through the communication method (such as topic) between operation nodes.

[0050] Compared with constructing operation nodes in a pure software manner, constructing operation nodes using hardware debugging programs can effectively utilize the powerful computing power of the hardware itself, improve computing efficiency, achieve accelerated computing, and improve the real-time performance of data processing. It is especially suitable for fields such as autonomous driving that have high real-time requirements. Operation nodes constructed using hardware methods can quickly and in real time obtain calculation results, get response strategies, and promptly control the vehicle to perform corresponding actions.

[0051] For example, the synchronizer 103 is configured to establish a communication connection between the first operation node of the environment simulator and the component simulator, and is configured to adjust the environment clock frequency of the environment simulator to be consistent with the simulation clock frequency of the component simulator, and to synchronize the communication between the component simulator and the environment simulator.

[0052] Therefore, by setting the synchronizer 103, the communication synchronization between the environment simulator and the component simulator is achieved. The environment simulator can collect environmental perception information as data to be processed according to the simulation clock frequency of the component simulator, and send the data to be processed to the component simulator according to the simulation clock frequency. The component simulator will not generate data backlog, and both the environment simulator and the component simulator can work normally, and can better test the performance and functional safety of the first operation node.

[0053] Of course, it should be noted that the synchronizer 103 also establishes communication connections between other operation nodes of the environment simulator and the component simulator. The specific content is similar to that of the first operation node and will not be repeated here.

[0054] The structures of the environment simulator, synchronizer and component simulator are described in detail below with reference to the accompanying drawings.

[0055] Figure 3 A structural diagram of an environment simulator provided for at least one embodiment of the present disclosure.

[0056] like Figure 3 As shown, the environment simulator 101 includes an environment simulation unit 1011 and a calculation unit 1012 .

[0057] For example, the environment simulation unit 1011 is configured to simulate the simulation environment of the autonomous driving test. For example, the environment simulation unit 1011 may be the open source simulator CARLA as described above, and the environment simulation unit 1011 may provide, for example Figure 1 The simulation environment for autonomous driving is shown in the figure, and a variety of traffic scenarios can be configured, such as pedestrians running red lights, so that the functional safety of the autonomous driving algorithm can be evaluated to confirm whether the vehicle can accurately analyze and adopt safe and reliable response strategies in various scenarios.

[0058] For example, the environment simulation unit 1011 may be provided with multiple simulated sensors, such as radar (e.g., laser radar or ultrasonic radar), camera (e.g., three-color camera or monochrome camera), ToF sensor, etc. These sensors may collect environmental perception information such as images of the simulated vehicle surrounding environment according to the environment clock frequency of the environment simulation unit 1011. The environment clock frequency of the environment simulator is the environment clock frequency of the environment simulation unit 1011, and the environment clock frequency of the environment simulation unit is the overall clock frequency of the environment simulation unit. The environment simulation unit completes operations such as collection and message transmission according to the environment clock frequency.

[0059] For example, the computing unit 1012 may use the robot operating system as described above. For example, the computing unit 1012 includes multiple operating nodes, and is configured to perform data processing through multiple operating nodes and multiple hardware debugging programs to achieve vehicle control. In other words, multiple operating nodes can be distributed in the robot operating system, and each operating node completes a different data computing task and cooperates to achieve vehicle control.

[0060] Figure 31 shows a first operation node among multiple operation nodes included in the computing unit 1012. For example, the first operation node is composed of a first hardware debugging program, that is, the first operation node is not constructed by pure software, but is implemented by relying on a hardware processor or chip, etc. Other operation nodes are similar to the first operation node and are not described here.

[0061] like Figure 3 As shown, when the environment simulation unit is CARLA, the environment simulation unit and the first operation node communicate through carla-ros-bridge. For carla-ros-bridge, please refer to the above content and will not be repeated here. Of course, if the environment simulation unit is a platform that can simulate the simulation environment of the autonomous driving test, other intermediaries can also be used to perform protocol conversion to complete the communication, or the first operation node can communicate directly with the environment simulation unit, and the present disclosure does not make specific restrictions on this.

[0062] For example, the environmental perception information collected in the environmental simulation unit 1011 is transmitted to the first operation node through the carla-ros-bridge as the data to be processed. The first operation node performs the first data processing on the data to be processed, and the obtained calculation result can be sent to the graphic interface server. The graphic interface server controls the vehicle in the environmental simulation unit to display the corresponding visual action state, so as to intuitively see the control effect of the vehicle.

[0063] For example, a graphic interface server is a display service provided by a Linux system. However, for other operating systems centered on an interface, such as a Windows operating system, a graphic interface server may not be provided.

[0064] For example, the component simulator 102 may be a hardware emulation accelerator or a field programmable gate array (FPGA).

[0065] For example, different hardware debugging programs can perform different data processing and complete different data computing tasks. For example, data computing tasks may include path planning tasks, image semantic segmentation tasks, image rendering tasks, driving judgment, etc. Depending on the task, the amount of calculation is different, and the scale of the hardware debugging program is also different. For example, when the scale of the hardware debugging program is relatively large, it can be run, simulated, and tested on a hardware simulation accelerator. The hardware simulation accelerator is a high-performance verification computing platform that integrates simulation (Simulation), acceleration (Acceleration) and emulation (Emulation) in a unified verification environment, which can accelerate the simulation process. For example, when the scale of the hardware debugging program is relatively small, you can directly use FPGA for running, simulation, and testing. For example, the bit file generated by the hardware debugging program is loaded into the FPGA for running to complete testing and debugging.

[0066] Figure 4 A schematic structural diagram of a component simulator provided for at least one embodiment of the present disclosure.

[0067] like Figure 4 As shown, the component simulator includes a processing core 1021, and the processing core 1021 is configured to control and schedule multiple hardware debugging programs to implement vehicle control.

[0068] like Figure 4 As shown, the multiple hardware debugging programs include a first hardware debugging program, a second hardware debugging program, ..., an Nth hardware debugging program, where N is a positive integer. Of course, it should be noted that the number of hardware debugging programs disclosed in the present invention is not limited.

[0069] For example, the processing core 1021 may be a hardware simulation accelerator or a central processing unit of an FPGA, or may be a central processing unit in the form of an instantiated software core, and the present disclosure does not impose any specific limitation on this.

[0070] For example, the processing core 1021 can be used as a main control unit to run the main program, and through instruction control and register configuration, it can schedule multiple hardware debugging programs to perform different calculations. For example, the hardware debugging program can perform different accelerated calculations as needed, such as completing accelerated calculations such as image segmentation and image rendering by the hardware debugging program, so as to make full use of hardware performance and improve calculation efficiency; and the calculation results are transmitted to the processing core, which distributes, transmits or further processes them.

[0071] For example, the processing core and the hardware debugger communicate via an on-chip bus, such as an AMBA (Advanced Microcontroller Bus Architecture) bus, to implement interrupt control and high-speed signal transmission.

[0072] For example, the processing core receives data sent by the synchronizer and transmits the data to the corresponding hardware debugging program through the on-chip bus. After the hardware debugging program processes and obtains the calculation result, it transmits the calculation result to the processing core through the on-chip bus, and then the processing core transmits it to the synchronizer.

[0073] For example, in some other embodiments, the component simulator 102 may not be provided with a processing core. In this case, all data processing and vehicle control are completely implemented by the hardware debugging program, and the present disclosure does not impose any specific restrictions on this.

[0074] For example, the second hardware debugger to the Nth hardware debugger may also have corresponding second operation nodes, ..., Nth operation nodes in the environment simulator 101 . Similarly, the operation nodes are mappings of the corresponding hardware debuggers in the environment simulator 101 .

[0075] The following takes the first operation node and the first hardware debugging program as an example to explain the meaning of “the operation node is the mapping of the corresponding hardware debugging program in the environment simulator 101” in the present disclosure.

[0076] During the debugging phase, the hardware debugger runs in the component simulator. The component simulator and the environment simulator belong to two different systems / devices. The difference here means that the two have different clock frequencies and belong to different clock domains. For example, this may be caused by cross-server communication between the component simulator and the environment simulator.

[0077] At this time, the first operation node is the mapping of the first hardware debugging program in the environment simulator 101, that is, the first hardware debugging program and the first operation node are physically the same object, but the specific operations, calculations, and processing are performed in the component simulator. The first operation node provides an interface in the environment simulator 101, so that the calculation results of the first hardware debugging program can be sent to other operation nodes or a graphical interface server through the communication method (e.g., topic) between operation nodes. For example, the other operation nodes are any operation nodes other than the first operation node among the multiple operation nodes. Therefore, during the debugging phase, the first operation node provides a communication interface as a mapping of the first hardware debugging program, and the communication interface is used to communicate with other operation nodes in the environment simulators of different systems / devices whose component simulators running the first hardware debugging program belong to.

[0078] For example, combined with Figure 1 The first operation node is configured to receive the data to be processed sent by the environment simulation unit, forward the data to be processed to the synchronizer, so as to transmit the data to be processed to the first hardware debugging program through the synchronizer, and the first operation node is also configured to receive the calculation result of the first hardware debugging program through the synchronizer, and forward the calculation result to other operation nodes. Thus, the first operation node, as a mapping of the first hardware debugging program, provides a communication interface for data communication of the first hardware debugging program in the environment simulator.

[0079] For example, before starting debugging, the synchronizer 103 is configured to calculate the ratio of the current environment clock frequency and simulation clock frequency of the environment simulator, and send the ratio to the environment simulator so that the environment simulator adjusts the environment clock frequency of the environment simulator according to the ratio to be consistent with the simulation clock frequency.

[0080] As mentioned above, usually the environment simulation unit in the environment simulator collects environmental perception information as data to be processed at a frequency of GHz, and sends it to the operation node at a frequency of GHz. In the debugging stage, the hardware debugging program corresponding to the operation node runs at a frequency of MHz, such as 10MHz, 20MHz, etc. Therefore, the synchronizer calculates the ratio n of the current environment clock frequency and the simulation clock frequency of the environment simulator, where n is a positive number, and sends the ratio n to the environment simulator. The environment simulation unit in the environment simulator can configure its own overall environment clock frequency. Therefore, the environment simulation unit in the environment simulator can adjust its own environment clock frequency according to the ratio n, slowing down its own environment clock frequency to n times the original, so as to be consistent with the simulation clock frequency of the component simulator.

[0081] Therefore, the environment simulator can collect environmental perception information according to the simulation clock frequency. For example, it can collect the environmental image around the vehicle according to the simulation clock frequency and send it to the synchronizer at the simulation clock frequency, so that the component simulator can receive the environmental image at the simulation clock frequency. The receiving frequency is consistent with the frequency of the component simulator, and no data backlog will occur. Both the component simulator and the environment simulator can work normally, so that the functions and performance of the hardware debugging program can be effectively debugged and tested.

[0082] Although the environment simulator and component simulator are adjusted to the same clock frequency, there may still be cross-server communication between the two. At the physical layer, the clock domains are still different. If messages are directly transmitted without synchronization, there may still be data backlogs, resulting in message errors, loss, etc. Therefore, synchronization processing is required to achieve synchronization of cross-clock domain message transmission between the environment simulator and the component simulator, that is, to synchronize the clock domains of the environment simulator and the component simulator, further avoid data backlogs, and ensure smooth and accurate communication between the component simulator and the environment simulator.

[0083] For example, the synchronizer 103 is configured to forward the message received from the message sender to the message receiver after synchronization processing. Here, the message sender can be the environment simulator 101, and the message receiver is the component simulator 102. Of course, the message sender can also be the component simulator 102, and the message receiver is the environment simulator 101.

[0084] For example, the synchronizer 103 can be implemented as a server, that is, the synchronizer 103 is implemented as a dedicated server that is physically independent of the environment simulator 101 and the component simulator 102. This method has higher processing efficiency and does not occupy the computing resources of the environment simulator 101 or the component simulator 102. In actual operation, there is no need to modify and configure the hardware too much, and the implementation is easier. For example, at this time, the synchronizer 103 and the component simulator 102 and the environment simulator 101 can be connected and communicated through an external bus.

[0085] Of course, the synchronizer 103 and the environment simulator can also be set in the same electronic device, such as a computer. The synchronizer 103 and the environment simulator 101 can be implemented in the form of two virtual machines set in the computer. For example, in this case, the synchronizer 103 and the component simulator 102 and the environment simulator 101 can be connected and communicated through the communication bus inside the computer. The present disclosure does not specifically limit the specific structure of the synchronizer 103.

[0086] For example, the synchronization process performed by the synchronizer 103 is completed through a synchronous communication protocol. For example, the synchronous communication protocol uses a preset storage area to cache data to achieve synchronization of message transmission across clock domains between the environment simulator and the component simulator.

[0087] In the present disclosure, communication synchronization is implemented through the protocol layer, for example, the Asyncio class provided in the Django architecture can be used to implement it, and the Asyncio class can provide asynchronous support, and use cache operations such as stacks and pushes to achieve high-speed asynchronous signal forwarding. Of course, the present disclosure is not limited to this, and other feasible synchronous communication protocols can also be used to complete the cross-clock domain synchronization of messages.

[0088] The signal protocol format defined by the component simulator and the signal protocol format defined by the environment simulator are likely to be different. Therefore, signal conversion is required to parse the valid content of the message sent by the message sender and reorganize the valid content according to the protocol format of the message receiver, so that the message receiver can quickly and accurately parse the message content and complete the corresponding operation.

[0089] For example, the synchronizer is further configured to perform signal conversion, which includes parsing the received message according to the protocol format of the message sender, reorganizing the parsed message content according to the protocol format of the message receiver, and sending the reorganized message to the message receiver.

[0090] For example, the signal protocol format sent by the environment simulator is the signal protocol format of the operation node, and the signal protocol format of the component simulator is the signal protocol format of the processing core in the component simulator. The synchronizer is used to establish a communication connection between the operation node and the processing core, so the synchronizer can perform signal conversion on the operation node and the processing core.

[0091] Therefore, in the above embodiment, the synchronizer can configure the environment clock frequency of the environment simulator to be consistent with the simulation clock frequency of the component simulator, so that the environment simulator can collect and transmit the data to be processed according to the simulation clock frequency, and there will be no data backlog in the communication process. During debugging, both the environment simulator and the component simulator can work normally; the synchronizer can also complete signal conversion, allowing the two to communicate without changing the protocol of the component simulator and the environment simulator; and the synchronizer can also complete the synchronization of message transmission across clock domains between the environment simulator and the component simulator, further avoiding data backlog.

[0092] At least one embodiment of the present disclosure also provides a debugging method. Figure 5 A schematic flowchart of a debugging method provided for at least one embodiment of the present disclosure.

[0093] For example, Figure 5 As shown, the debugging method provided by the embodiment of the present disclosure includes steps S10 to S40.

[0094] In step S10, the environment simulator is used to collect data to be processed, and the data to be processed is sent to the synchronizer.

[0095] For example, the environment simulator includes a plurality of operation nodes, and the plurality of operation nodes includes a first operation node.

[0096] For example, the environment simulator may include an environment simulation unit and a calculation unit, and may further include a setting unit as required, so as to interact with the user.

[0097] The environment simulation unit is used to simulate the simulation environment of the autonomous driving test. For the relevant content of the environment simulation unit, please refer to the relevant introduction of the environment simulation unit 1011 in the debugging system, which will not be repeated here.

[0098] The computing unit includes a plurality of operating nodes and is configured to perform data processing through the plurality of operating nodes and utilize a plurality of hardware debugging programs to realize vehicle control.

[0099] For the hardware debugging program, computing unit, operation nodes, etc., please refer to the relevant introduction of the computing unit 1012 in the aforementioned debugging system, which will not be repeated here.

[0100] For example, the environment simulation unit provides multiple different types of sensors, such as radars, cameras, etc., which can sense the simulated vehicle surrounding environment and collect images and other information around the vehicle at the environment simulation frequency as data to be processed.

[0101] For example, before executing step 10, the debugging method provided by at least one embodiment of the present disclosure also includes: calculating, through a synchronizer, the ratio of the current environment clock frequency of the environment simulator to the simulation clock frequency of the component simulator; and adjusting the environment clock frequency of the environment simulator according to the ratio so that the environment clock frequency of the environment simulation unit is consistent with the simulation clock frequency of the component simulator.

[0102] For the specific process of adjusting the environment clock frequency, please refer to the relevant content in the aforementioned debugging system, which will not be repeated here.

[0103] For example, step S10 may include: collecting data to be processed according to the adjusted environment clock frequency.

[0104] For example, a plurality of sensors of different categories are provided in the environment simulation unit, so that the environment simulation unit can collect environmental perception information as data to be processed at a simulation clock frequency. For example, the data to be processed can be an environmental image around the vehicle, and the data to be processed is sent to the operation node at the simulation clock frequency, which is forwarded to the synchronizer by the operation node to avoid data backlog and ensure the real-time and accuracy of communication.

[0105] In step S20, the data to be processed is processed by the synchronizer, and the synchronized data to be processed is transmitted to the component simulator.

[0106] For example, the environment clock frequency of the environment simulator and the simulation clock frequency of the component simulator are adjusted to be consistent through a synchronizer.

[0107] For example, the processing of a synchronizer may include a synchronization process.

[0108] For example, the synchronization process can be completed through a synchronous communication protocol, which uses a preset storage area for data caching to complete the synchronization of message transmission across clock domains between the environment simulator and the component simulator. For the specific content of the synchronization process, please refer to the relevant content of the synchronization process performed by the synchronizer in the aforementioned debugging system, which will not be repeated here.

[0109] For example, the processing of the synchronizer may also include signal conversion. For example, the synchronizer parses the received data according to the protocol format of the message transmission to obtain the effective content of the message, such as the effective content of the message is the data to be processed, and then reorganizes the data to be processed according to the protocol format of the component simulator to obtain the processed data to be processed, and the processed data to be processed has the signal protocol format of the component simulator.

[0110] In step S30, the component simulator is used to perform first data processing on the received data to be processed, to obtain a calculation result, and the calculation result is sent to the synchronizer.

[0111] For example, a plurality of hardware debugging programs are run on the component simulator, the plurality of hardware debugging programs include a first hardware debugging program, and the first hardware debugging program is configured to perform a first data process in the component simulator. The component simulator includes a processing core, the processing core is configured to control and schedule the plurality of hardware debugging programs to implement vehicle control, and the processing core communicates data with the plurality of hardware debugging programs via an on-chip bus.

[0112] For the relevant introduction of the component simulator, please refer to the relevant content of the component simulator 102 in the aforementioned debugging system, which will not be repeated here.

[0113] For example, step S30 may include: receiving the data to be processed through the processing core, and transmitting the data to be processed to the first hardware debugging program via the on-chip bus; performing first data processing on the data to be processed using the first hardware debugging program to obtain a calculation result; transmitting the calculation result to the processing core through the on-chip bus, and the processing core forwarding it to the synchronizer.

[0114] In step S40, the calculation result is processed by the synchronizer, and the processed calculation result is transmitted to the first operation node.

[0115] For example, the first operation node is a mapping of the first data processing performed by the component simulator in the environment simulator.

[0116] The relationship between the operation node and the hardware debugging program can be referred to the related introduction in the aforementioned debugging system, which will not be repeated here.

[0117] For example, the processing of the calculation result by the synchronizer here may include synchronization processing and signal conversion. The specific process is referred to the aforementioned content and will not be repeated here.

[0118] For example, the debugging method provided by at least one embodiment of the present disclosure may further include: forwarding the calculation results after synchronization processing to other operation nodes through the first operation node, wherein the other operation nodes are any operation nodes among the multiple operation nodes except the first operation node.

[0119] For example, after receiving the calculation result of the first data processing, the first operation node transmits it to other operation nodes through the communication method (such as topic) between the operation nodes. For example, the first operation node can also transmit it to the graphical interface server for displaying the action changes of the vehicle according to the calculation result on the visual interface.

[0120] In the above embodiment, the first operation node is taken as an example for description, but according to actual conditions, the multiple operation nodes may also include a second operation node and other operation nodes. The data flow of each operation node is the same as that of the first operation node. Each operation node communicates and exchanges data with the corresponding hardware debugging program through a synchronizer, a processing core, a bus, etc. The operation nodes communicate with each other through communication methods defined by the robot operating system such as topics. Each operation node and the environment simulation unit can communicate through intermediaries such as carla-ros-bridge, which will not be described in detail here.

[0121] Figure 6 A data flow diagram of a debugging method provided in one embodiment of the present disclosure.

[0122] Combine the following Figure 6 , specifically explain the data communication process of the debugging method.

[0123] For example, before starting debugging, the synchronizer calculates the ratio n between the current environment clock frequency and the simulation clock frequency of the environment simulator, and sends the ratio n to the environment simulator. The environment simulation unit in the environment simulator adjusts its own environment clock frequency according to the ratio n, slowing down its own environment clock frequency to n times the original, so as to be consistent with the simulation clock frequency of the component simulator.

[0124] For example, afterwards, the camera of the simulated vehicle in the environment simulation unit collects the environment image in front of the simulated vehicle according to the environment clock frequency, and sends the environment image to the first operation node at the environment clock frequency.

[0125] For example, afterward, the first operation node forwards the environment image to the synchronizer, and the synchronizer performs synchronization processing, signal conversion, and the like, and transmits a message containing the environment image to the component simulator.

[0126] For example, afterward, the processing core in the component simulator receives the environment image and sends it to the corresponding first hardware debugging program through the on-chip bus.

[0127] For example, afterward, the first hardware debugging program processes the environment image to obtain a calculation result, and transmits the calculation result back to the processing core through the on-chip bus.

[0128] For example, afterward, the processing core passes the calculation result through the synchronizer, undergoes synchronization processing, signal conversion, etc., and transmits a message containing the calculation result to the first operation node in the environment simulator.

[0129] For example, afterwards, the first operation node sends it to other operation nodes through topics or the like.

[0130] The above process is repeated in this way. Through the visualized movement changes of the vehicle in the environmental simulator, the performance and functional safety of the hardware debugging program are observed, and the performance evaluation and functional safety evaluation of the hardware debugging program are completed.

[0131] The above debugging method can achieve similar technical effects as the above debugging system, which will not be described in detail here.

[0132] Those skilled in the art will appreciate that the contents disclosed in this disclosure may be subject to various modifications and improvements. For example, the various devices or components described above may be implemented by hardware, or by software, firmware, or a combination of some or all of the three.

[0133] In addition, although the present disclosure makes various references to certain units in the system according to embodiments of the present disclosure, any number of different units can be used and run on the client and / or server. The units are only illustrative, and different aspects of the system and method can use different units.

[0134] Flowcharts are used in this disclosure to illustrate the steps of the method according to the embodiments of the present disclosure. It should be understood that the preceding or following steps are not necessarily performed precisely in order. On the contrary, various steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes.

[0135] The above is an explanation of the present disclosure and should not be considered as a limitation thereof. Although several exemplary embodiments of the present disclosure are described, it will be readily understood by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure as defined in the claims. It should be understood that the above is an explanation of the present disclosure and should not be considered to be limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present disclosure is defined by the claims and their equivalents.

Claims

1. A debugging system comprising an environment simulator, a synchronizer and a component simulator, The environment simulator is configured to provide a simulation test environment, wherein: The environment simulator includes a plurality of operation nodes, and the plurality of operation nodes includes a first operation node; The component simulator is configured to perform a first data process, wherein the first operation node is a mapping of the first data process performed by the component simulator in the environment simulator; The synchronizer is configured to establish a communication connection between the first operation node of the environment simulator and the component simulator, and is configured to adjust the environment clock frequency of the environment simulator to be consistent with the simulation clock frequency of the component simulator, and synchronize the communication between the component simulator and the environment simulation simulator; Wherein, a plurality of hardware debugging programs are running on the component simulator, the plurality of hardware debugging programs include a first hardware debugging program, and the first hardware debugging program is configured to execute the first data processing in the component simulator; The multiple operation nodes correspond to the multiple hardware debugging programs one by one, the first operation node is a mapping of the first hardware debugging program in the environment simulator, and provides a communication interface for data communication of the first hardware debugging program in the environment simulator; Wherein, the environment simulator comprises an environment simulation unit and a calculation unit, The environment simulation unit is configured to simulate the simulation environment of the autonomous driving test, The computing unit includes the plurality of operating nodes and is configured to execute data processing through the plurality of operating nodes and utilize the plurality of hardware debugging programs to implement vehicle control.

2. The debugging system according to claim 1, wherein: The first operation node is configured as: receiving the data to be processed sent by the environment simulation unit, forwarding the data to be processed to the synchronizer, so as to transmit the data to be processed to the first hardware debugging program through the synchronizer, and The calculation result of the first hardware debugging program is received through the synchronizer, and the calculation result is forwarded to other operation nodes, wherein the other operation nodes are any operation nodes among the multiple operation nodes except the first operation node.

3. The debugging system according to claim 1 or 2, wherein: The synchronizer is implemented in the form of a server. The synchronizer is configured to forward a message received from a message sender to a message receiver after synchronization processing, wherein the message sender and the message receiver are the environment simulator or the component simulator, and the message sender and the message receiver are different.

4. The debugging system according to claim 1 or 2, wherein: The synchronization process performed by the synchronizer is completed through a synchronous communication protocol. The synchronous communication protocol utilizes a preset storage area to cache data, thereby completing synchronization of message transmission across clock domains between the environment simulator and the component simulator.

5. The debugging system according to claim 1 or 2, wherein: Before starting debugging, the synchronizer is configured to calculate the ratio of the current environment clock frequency of the environment emulator and the simulation clock frequency, and send the ratio to the environment emulator so that the environment emulator adjusts the environment clock frequency of the environment emulator according to the ratio to be consistent with the simulation clock frequency.

6. The debugging system according to claim 1 or 2, wherein: The synchronizer is also configured to perform signal conversion, The signal conversion includes parsing the received message according to the protocol format of the message sender, reorganizing the parsed message content according to the protocol format of the message receiver, and sending the reorganized message to the message receiver. The message sender and the message receiver are the environment simulator or the component simulator.

7. The debugging system according to claim 1, wherein: The component simulator includes a processing core configured to control and schedule the plurality of hardware debuggers to implement the vehicle control, The processing core communicates with the plurality of hardware debuggers via an on-chip bus.

8. The debugging system according to claim 7, wherein: The component simulator is a hardware simulation accelerator or a field programmable gate array.

9. A debugging method, comprising: Using an environment simulator to collect data to be processed, and sending the data to be processed to a synchronizer, wherein the environment simulator includes a plurality of operation nodes, and the plurality of operation nodes includes a first operation node; Processing the data to be processed by the synchronizer, and transmitting the processed data to be processed to the component simulator, wherein the environment clock frequency of the environment simulator and the simulation clock frequency of the component simulator are adjusted to be consistent by the synchronizer, and the processing at least includes synchronization processing; Using the component simulator to perform first data processing on the received data to be processed, obtain a calculation result, and send the calculation result to the synchronizer; Processing the calculation result through the synchronizer, and transmitting the processed calculation result to the first operation node, wherein the first operation node is a mapping of the first data processing performed by the component simulator in the environment simulator, a plurality of hardware debugging programs are running on the component simulator, the plurality of hardware debugging programs include a first hardware debugging program, the first hardware debugging program is configured to execute the first data processing in the component simulator, and the first operation node provides a communication interface for data communication of the first hardware debugging program in the environment simulator; Wherein, the environment simulator comprises an environment simulation unit and a calculation unit, The environment simulation unit is used to simulate the simulation environment of the autonomous driving test. The computing unit includes the plurality of operating nodes and is configured to perform data processing by using the plurality of hardware debugging programs to realize vehicle control. The debugging method further comprises: The calculation result after the synchronization processing is forwarded to other operation nodes through the first operation node, wherein the other operation node is any operation node among the multiple operation nodes except the first operation node.

10. The debugging method according to claim 9, wherein: Before collecting the data to be processed by using the environment simulator, the debugging method further includes: Calculating, by means of the synchronizer, a ratio of a current environment clock frequency of the environment simulator to a simulation clock frequency of the component simulator; According to the ratio, the environment clock frequency of the environment simulator is adjusted so that the environment clock frequency of the environment simulation unit is consistent with the simulation clock frequency of the component simulator.

11. The debugging method according to claim 10, wherein: Use the environment simulator to collect data to be processed, including: The data to be processed is collected according to the adjusted environmental clock frequency.

12. The debugging method according to claim 9, wherein: A plurality of hardware debugging programs are running on the component simulator, the plurality of hardware debugging programs include a first hardware debugging program, and the first hardware debugging program is configured to execute the first data processing in the component simulator; The component simulator includes a processing core configured to control and schedule the plurality of hardware debuggers to implement the vehicle control, The processing core communicates data with the multiple hardware debugging programs via an on-chip bus. Using the component simulator to perform first data processing on the received data to be processed to obtain a calculation result, and sending the calculation result to the synchronizer, including: receiving the data to be processed by the processing core, and transmitting the data to be processed to the first hardware debugging program via the on-chip bus; Using the first hardware debugging program to perform the first data processing on the data to be processed to obtain the calculation result; The calculation result is transmitted to the processing core via the on-chip bus and forwarded to the synchronizer by the processing core.

Images